August 2013

SOCIAL BIG DATA
How the analysis of Social Big Data has radically changed the way in which we monitor social networks and deal with business intelligence

01 | What Big Data is
02 | What has changed with the Social Media
03 | 4Ps of Insight: a framework to get started
04 | How to use the 4Ps of Insights right now
04 | Big Data applications

What Big Data is
Big Data is what the market calls a large volume of data.
In the 1990s, large stores of data (Data Warehouses) were already used by companies like Walmart to identify consumer habits from their choices at the point of sale. This was how it was discovered that parents who bought beer were also buying disposable nappies, allowing retailers to devise tied promotions.
In the last decade, the Mayor of New York, Michael
Bloomberg, also implemented a data warehouse for the city. The aim was to improve decisions by the public authorities. For example, which properties – some 900,000 in the city – to inspect to prevent the occurrence of large fires.
The first step was to identify the different data sources that generate information about the city: on rented properties, medical emergencies, complaints about noise in the neighbourhood, crime reporting, even the sighting of rats.
This data was unstructured and in different silos
(different databases according to the municipal department dealing with the problem). It first went through a process of structuring to make manipulation possible. Then a statistical model was created to handle the data. Once the model was implemented, every new claim received by the city about a fire danger was checked using the algorithm of the new model.
The result: the rate of reports dealt with that actually

represented an imminent danger jumped from 30% to
70%. Decisive forecasting enabled New York City to make considerable resource savings, since it was able to avoid sending out fire crews to false alarms.
A similar problem also related to New York’s exploding manhole covers. The same problem has plagued other cities in the world such as Rio de Janeiro, and in the
Big Apple it was also treated with Big Data. To find out which manholes were on the point of blowing, a large volume of different data such as charts of gas, telephone, electricity ducts, as well as traffic in the immediate vicinity was cross-referenced.
Just like NYC, the American retailer Target also transformed its big data into insights, e.g. when it predicted whether its female consumers were pregnant a few months before giving birth. The process involved the analysis of a large volume of data on products consumed and buying habits, enabling Target to have a more personalised communication with the customer.
There is even a story that one irate father called Target’s
Marketing Department complaining that his teenage daughter had begun to receive offers of baby products by email. Some time later he discovered that his daughter actually was pregnant (and he didn’t know).
In both cases, New York City and Target, unstructured data formed a large volume of data (big data) that, after passing through statistical analyses, enabled important insights to be identified into both current and future consumer habits.

01

What has changed with the
Social Media
The social media present a new challenge: these days, companies have more external data on their consumers than internal data. There are more than
1 billion Facebook users worldwide. Twitter, with 200 million active users, generates over 400 million tweets daily. The volume of this information grows by the day. From
2008 to date, we at E.life have stored 600 million data points, or units of data, posts, comments and shares from Twitter and Facebook. More than half of this data has been stored since 2012. Just to mention some figures, there are more than 30 million Brazilian users
(and they’re growing exponentially).
The challenge is not only