Searching for Privacy in a World Without Secrets

“I am not a number, I am a free man”

IDC reported that we generated and replicated 1.8 zettabytes – that’s 1.8 trillion gigabytes – of data in 2011. To give you an example of scale you would need to stack CDs from Earth to the Moon and Back again – twice – to represent that amount of data and its expected to grow 50x by 2020. Interesting factoid: Through April of 2011 the Library of Congress had stored 235TBs of data. In 2011 15 out of 17 sectors in the US have more data per company than the US Library of Congress, much of that data is about you.

Facebook is preparing to raise $100 billion, yes a hundred billion, in a highly anticipated IPO next spring. Twitter is valued at $10 billion, and social media companies are pulling massive valuations. In terms of data, roughly 4 billion pieces of content are shared on Facebook every day, and Twitter registered 177 million tweets per day in March of 2011. The success of these companies, and many others, is trade in human commodity. There is an inherent value to your tweet, your wall post, becoming mayor at some DC cafe or posting your location to wherever people post those things, but the real value is simply in your existence as a number in a sea of other 1 and 0’s.

We are entering a world where every aspect of our lives, short of those thoughts we hold deep, will be processed, indexed, analyzed and archived forever. What we search for, our online activity, where and how we drive, what we buy; when and how often, our health, financial, and personal records digitized for quick sale to the highest bidder. Never before have we had the ability to implement systems to handle massive volumes of disparate data, at a velocity that can only be described as break-neck and with this ability comes the inevitable misuse.

The commercial implications for companies seeking access to this depth and breadth of customer intelligence is clear, but this same information federated with the analysis of unstructured video, picture, voice and text data in the hands of our government or one that meant us harm is truly frightening.

Social media is an interesting experiment in applying a large scale operant conditioning chamber to a mass population, the law of effect is a retweet, a friending, being listed on a top x most influential list, or whatever else elicits the desired response. We leap head first off the cliff of technology and only concern ourselves with the implications when they become a problem for us.

The irony is that in our search for identity and individuality in an increasingly digital world we have willingly surrendered that which we used to hold so dear – our privacy.

May future generations forgive us.

Needles in a Digital Hay Stack; Finding Value in Big Data

Big data is a scorching hot topic, currently capturing a lions share of the markets available stock of hyperbole and for good reason, data is growing at a meteoric rate.

As we continue to innovate, as business accelerates technology adoption, as the line bleeds between corporate and personal computing and as we interact more in digital mediums we are creating mountains of data. Much of this data is garbage, but some of it is gold (big-data-are-you-creating-a-garbage-dump-or-mountains-of-gold).

Unfortunately with all overly hyped technologies there is a lot of misinformation, failed expectations and the inevitable trough of disillusionment, but that doesn’t mean you have to spend months or years curled up in a fetal position, disillusioned and wondering what went so wrong. With a thoughtful approach you can venture through the murky swamp of your big data and find the insights that provide your company a significant competitive and market advantage.

Continue reading

Big Data; Are You Creating a Garbage Dump or Mountains of Gold

You’re not really sure how it happened, but some time between last year and the summer of 2011 you were suddenly facing a big data problem, or you were being told you were facing a big data problem, or more accurately you were being told that you needed a big data solution.

Funny thing was that you hadn’t really done anything drastic over the last couple of years that would seem to indicate a tsunami of data was about to breach your storage floodgates, but then again it wasn’t like you watched yourself going bald either.

Continue reading