There is a tsunami of data headed our way


Big Data, the term referring to data sets so large and complex traditional data processing applications are not sufficient enough to be able to process it. This is because we are saving data on a large scale beyond operational data which is mainly due to the rise of the internet. At this moment, Big Data is still quite manageable but there is an emerging technology that will make the current Big Data seem like Small Data.

I’m talking about the rise of the Internet of Things. Right now, this technology it is at the height of the Peak of Inflated Expectations of the Gartner Hype Cycle[1]. It is thus expected that the amount of ‘things’ will grow exponentially in the coming years because these ‘things’ can be any object with built-in sensors like cars, phones, watches, doors, shoes, lights switches, anything.

emerging-tech-hcIoT-2

Cisco expects that the number of connected devices will grow from 18 billion in 2015 up to 50 billion in 2020[2]. Each of these ‘things’ will generate a massive amount of data contributing to the already large influx of data. Right now, about 4.4 ZB (that is 4.4 trillion gigabytes) of data is generated each year[3]. This is expected to grow to more than 40 ZB in 2020. Companies saving data because of just saving data are at risk of being washed away by a tsunami of data headed their way.

NSA for example, an institution we have all heard of, is already drowning in data. They have adopted a strategy to filter out data that does not contain useful content in order to diminish the amount of data they have to save by about 20%.[4] Instead of just taking all there is, analysts are now trying figure out what is actually necessary. Ofcourse, it is very hard to know in advance what information is useful and what is not. But companies will have to start thinking about this topic and what to do what all that data headed their way.

According to Computerworld, the coming trend in Big Data and Business Intelligence will be to structure the collection of data and to know exactly what data is collected. [5] I think they are quite right, I have experienced on my own that companies just collect data for sake of collecting without knowing is actually important. Only after you have done some analysis you find out what is useful and what is not. But in the coming age of the internet of things, with so many gigabytes of data coming our way, this strategy is just not feasible anymore.

Tony Jordan – 400986


[1] http://www.gartner.com/newsroom/id/3114217

[2] http://newsroom.cisco.com/feature-content?type=webcontent&articleId=1208342

[3] http://www.zdnet.com/article/the-internet-of-things-and-big-data-unlocking-the-power/

[4] http://webwereld.nl/big-data/79671-nsa-verzuipt-in-big-data-door-datasleepnet

[5] http://computerworld.nl/big-data/79682-internet-of-things-veroorzaakt-datatsunami/pagina-3

Advertisements

One response to “There is a tsunami of data headed our way”

  1. 435718rv@eur.nl says :

    Very interesting post. The numbers that you present are a clear reflection of how interconnected and online we are as people. However, these messages about the enormous size of the data that we are saving, these predictions on future growth and the efforts we have to make to save them always makes me wonder if it is already useful to save these gigantic amounts of data. We are still scratching the surface of what we are able with to do with big data. Of course we need data to find out what we can do, but until we know what that is it seems somewhat useless in my eyes to save these gigantic amounts of data. The amount of data could easily be build up again seeing its high growth rate over the last years. Then there is also the problem that we need to change our decision making decision before we can even make use of the results big data analytics can generate (McAfee & Brynjolfsson). If we keep following our gut feeling these efforts that we put into big data will be somewhat useless.
    McAfee, A., and Brynjolfsson, E. 2012. Big Data: The Management Revolution. Harvard Business Review 90(10) 60-68.

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s

%d bloggers like this: