Using Big Data to Gain Knowledge


By Frank J. Ohlhorst

For many in the business world, big data has become the holy grail of business analytics, providing answers to questions that many analysts did not know they even had. However, the phenomenon known as big data is becoming harder and harder to classify, especially as the term evolves beyond its original meaning of data sets too large to process with traditional technologies.

Today, the term big data can be loosely defined as the processing of extremely large data sets that contain structured and unstructured data, which is used to uncover previously hidden insights. Simply put, big data is all about obtaining knowledge. Nevertheless, the type of knowledge delivered from the analysis of big data counts for much more than traditional business knowledge. Big data analytics has the ability to deliver predictions and insights and to expose relationships that were once unrealized, but only if the analysis is done correctly.

Therein lies the biggest challenge with big data–doing it right! From a technological standpoint, many of the original challenges associated with big data have been solved. After all, platforms such as Hadoop and applications such as MapReduce have mitigated the problems associated with the basic processing of large data sets, but are creating a different challenge: How does one derive value out of large amounts of data?

Deriving that value can be a tedious and troublesome process, relying on gut instincts and the overall availability to uncover information that has analytic application. It’s a challenge that depends heavily on batch processing and designing algorithms to uncover the value. It all comes down to knowing what to look for, using an approach that starts with a simple question, “What am I looking for?”

For most businesses, that question evolves into something that aligns with growth, such as “How do we increase sales in the Western states?” or “How can we predict product demand?” Ultimately, it will be the questions that need answers that will drive the big data process.

Frank J. Ohlhorst is a technology journalist, author, professional speaker, and IT business consultant with more than 25 years of experience in the technology arena. His latest book, Big Data Analytics: Turning Big Data into Big Money, offers valuable insights into the realm of big data is available at http://amzn.to/T33yh4.