Why I Suck at Blogging…and Twitter

So recently I posted some thoughts on big data and the increasing usage of Hadoop, the general theme was data management != data analysis…this caused confusion with some folks, as evidenced by the twitter exchange (tweets haven’t been altered but some extraneous ‘noise’ removed to maximize your reading pleasure)

@Beaker @amrittsering I’m confused by your last blog. Is your point that people are spending $$$ on data aggregation hoping it leads to analytics?

@Beaker @amrittsering I read/re-read your posts & it’s almost like u r suggesting majority of co’s deploying Hadoop (e.g) are clueless WRT why?

Needles in a Digital Hay Stack; Finding Value in Big Data

Big data is a scorching hot topic, currently capturing a lions share of the markets available stock of hyperbole and for good reason, data is growing at a meteoric rate.

As we continue to innovate, as business accelerates technology adoption, as the line bleeds between corporate and personal computing and as we interact more in digital mediums we are creating mountains of data. Much of this data is garbage, but some of it is gold (big-data-are-you-creating-a-garbage-dump-or-mountains-of-gold).

Unfortunately with all overly hyped technologies there is a lot of misinformation, failed expectations and the inevitable trough of disillusionment, but that doesn’t mean you have to spend months or years curled up in a fetal position, disillusioned and wondering what went so wrong. With a thoughtful approach you can venture through the murky swamp of your big data and find the insights that provide your company a significant competitive and market advantage.

Big Data; Are You Creating a Garbage Dump or Mountains of Gold

You’re not really sure how it happened, but some time between last year and the summer of 2011 you were suddenly facing a big data problem, or you were being told you were facing a big data problem, or more accurately you were being told that you needed a big data solution.

Funny thing was that you hadn’t really done anything drastic over the last couple of years that would seem to indicate a tsunami of data was about to breach your storage floodgates, but then again it wasn’t like you watched yourself going bald either.

The Good, Bad, and Ugly of Technology Acquisitions

It is the foundation for the free market system and capitalism and it is every entrepreneurs dream; build a great technology, execute and achieve excellence in GTM, deliver fantastic value to customers and take great pride in watching your passion grow – fast.

Then it happens; the exit, the liquidation event, the ‘golden ticket’ and in a blip of your time on this tiny little rock your life changes.

Last year, after spending almost four years as the CTO of BigFix, we were acquired by IT industry behemoth IBM (IBM to acquire BigFix) for what was the largest acquisition of a private software company in 2010 (second was CA’s acquisition of Nimsoft at around $380m) and my life changed…

Cloud-Computing is Dead, Turn the Internet Off, Amazon Failed – Again!

 

So it appears the Internet went down, or so many claimed when they were presented with 404 errors when attempting to watch “Georgia Hillbilly Massacre 17: The return of the Banjo Man” on Netflix  - Since Netflix is selective on what you can stream they certainly weren’t queuing up the latest and greatest new releases, but that is a totally different rant – or attempting to declare themselves the Mayor of “who gives a rats ass where you are right now” on Foursquare.

Last time this happened some started to claim that it rocked the very foundation of confidence in cloud-computing (here), yet they failed to juxtapose Amazon’s operational failures against the universe of enterprise operational failures, security compromises and general administrative stupidity that plagues nearly 99.98% of every organization on Earth (minus the DPRK’s website, really not more you can do to fudge that one up)

BigData, Hadoop, and the Impending Informationpocalypse

“Information is not knowledge” – Albert Einstein

I recently read a couple of posts about BigData from my friend Chris Hoff - “Infosec Fail: The Problem With BigData is Little Data” and “More on Security and BigData…Where Data Analytics and Security Collide”

In these posts Hoff posits that the mass centralization of information will benefit the industry and that monitoring tools will experience a boon, especially those that leverage a cloud-computing architecture…

This will bring about a resurgence of DLP and monitoring tools using a variety of deployment methodologies via virtualization and cloud that was at first seen as a hinderance but will now be an incredible boon.

As Big Data and the databases/datastores it lives in interact with then proliferation of PaaS and SaaS offers, we have an opportunity to explore better ways of dealing with these problems — this is the benefit of mass centralization of information.

Hoff then goes on to describe how new data warehousing and analytics technologies, such as Hadoop, would positively impact the industry…

Even when we do start to be able to integrate and correlate event, configuration, vulnerability or logging data, it’s very IT-centric.  It’s very INFRASTRUCTURE-centric.  It doesn’t really include much value about the actual information in use/transit or the implication of how it’s being consumed or related to.

This is where using Big Data and collective pools of sourced “puddles” as part of a larger data “lake” and then mining it using toolsets such as Hadoop come into play…

2011: The Death of Security As We Know IT or Operationalizing Security

There is a dull hum permeating the industry of late – security is dead some say, others think it to be too costly to maintain, others still believe that what is needed is a change of perspective, perhaps a radical shift in how we approach the problem. What underlies all of these positions is a belief that the status quo is woefully ineffective and the industry is slated for self-destruction or, as a whole, we will succumb to a digital catastrophe that would have been avoided if only we had just…well, just done something different from whatever it is we are doing at the time something bad happens.

As we go round and round on the never ending hamster wheels provided as best practice guidelines and securty frameworks by security vendors, consultants, and pundits, we find ourselves trapped in an OODA loop that will forever deny us victory against malicious actors because we will never become faster, or more agile than our opponents. But to believe one can win, implies that there is an end that can be obtained, a victory that can be held high as a guiding light for all those trapped in eternal security darkness. We are as secure as we need to be at any given moment, until we are no longer so – when that happens, regardless of what you may believe, is outside of of our control.

One of the biggest trends in security over the past 5-6 years has been its movement into mainstream IT. Traditionally IT security has been seen as outside of normal business processes. Organizations tended to react driven by a security incident or compromise, an audit or compliance event, or due to perceived changes in the threat landscape. For the most part security has been and still is an afterthought.