Our modern information society has generated one thing above any other: data. A vast amount of very different data that flows through the Internet at a high speed. This is essentially Big Data and volume, variety and speed are the three axes upon which this concept rotates.
Data everywhere, and counting
All of us enter data in order to carry out everyday tasks, such as paying at the supermarket, shopping over the Internet, making a transfer through our bank’s Web portal or participating in social media. A huge amount of data is also extracted from mobile phones, gas and light meters, and satellites. In fact, virtually our entire life could be translated into data.
This has made technology firms develop more sophisticated data processing centers and computers with increased storage capacity. Just to see the spectacular evolution in this field, let’s take as an example the first 1-GB hard disk that was created in 1980: the IBM 3380. It occupied the same space as a fridge, weighed 250 kilos and cost 40,000 USD at the time. Nowadays, anyone can buy a memory card the size of a paperclip at an affordable price.
Facebook perfectly exemplifies the wide variety of data this type of platform can host from its millions of users: gender, age, marital status, personal preferences, etc., i.e. all highly valuable information, especially for major brands’ marketing departments.
Speeding up the Big Data analysis process
However, the quantum leap in terms of data management and use perhaps lies in speed. There are currently applications that allow running Big Data analysis processes in real time that previously had to be run in batches or during an entire night. This opens up a new world of possibilities for companies across all industries, without exception. For instance, car insurers can process customer claims in a matter of hours, whereas this procedure used to take several weeks. And health insurers can interpret complex data sets on customer health and thus forecast the health problems they may suffer from in just a few minutes. In particular, the insurance industry has substantially capitalized on the advantages offered by Big Data processing and analysis when it comes to detecting fraud, as they now can identify fraudulent claims and prescriptions of ‘ghost’ medicines.
The banking industry is another of the great beneficiaries of this phenomenon. Customer service centers of major financial institutions are implementing predictive modeling systems that allow them to embed Big Data from social media in real time. In this way, they have a comprehensive view of the customer on the other end of the line: they know their banking profile and personal background, their reaction to certain marketing campaigns, their opinion on the customer service rendered by the entity, their account status and transactions… This was simply unimaginable just some years ago.
Technology giants like Google, IBM, Microsoft, Oracle and SAP are already busy developing new business models and services based on Big Data processing. The document management software industry is not far behind either and has managed to apply Big Data to corporate paper-based documents. Specifically, they have developed text recognition software that makes it possible to capture data from physical documents, process it and enter it into Big Data systems from any kind of medium. In this sense, Big Data not only refers to information on the Web, but also to all printed documents and digital files that are in a company’s possession and contain critical information.
Big Data processing and analysis, an underestimated potential
So far, so good. Then, where does the challenge rise from? Precisely from the fact that, currently, only a few companies are capable of making the most out of all this information. According to Gartner, 85% of the companies included in the Fortune 500 list are not prepared to obtain competitive advantage from Big Data. And the potential is enormous.
The skepticism that reigns among some entrepreneurs should not be underestimated either. The question they usually ask themselves in this respect is: what’s the return on my investment? Nobody wants to wait for five years until the rewards from Big Data processing and analysis can be reaped. There is also some reluctance to the security breaches that might occur when handling so much and, most of the times, so sensitive data. Precisely for this reason, governments around the world have enacted data protections laws in order to mitigate the eventual impact and prevent as much as possible the wrongful or fraudulent use of information. After all, it shouldn’t be forgotten that gathering, storing and interpreting personal data may compromise users’ privacy. Those who handle this information not always ask permission to use it for their own benefit. Not to mention the much feared hackers, capable of accessing an entity’s bank accounts and stealing sensitive data.
In a nutshell, companies handle a vast amount of highly valuable data, but most of them don’t know how to leverage on it. Adequately used, Big Data processing and analysis allows companies to predict weather changes, improve crops, estimate capital market movements, know a product’s performance, etc. It can be most useful when designing a customized offering or making faster and better informed decisions. All we have to do now is wait a few more years and see whether or not companies have been able to overcome this challenge.
Julio Olivares
DocPath CEO
André Klein
Freelance Consultant for DocPath