Recent Big Data Trends

So while the book has been consuming much of my free time, I’ve still been busy working with numerous clients on their big data journey.  And over the past few months, there have been some shifts, and some non-shifts, in the big data world.  Here are a few of my observations:

• Business stakeholders are getting serious about understanding the business potential of big data.
  I recently had a 30-minute meeting with a CEO and his senior executive staff – CEO, not CIO – about how big data could transform their key business processes.  The 30-minute meeting lasted 2 hours!  We didn’t talk about technology; Instead the focus of the meeting was helping the senior leadership team to understand, or envision, how they could leverage big data to gather new insights about their customers, products, and operations to increase customer traffic and improve cross-sell and up-sell effectiveness.  These senior business executive meetings, while uncommon for me a year ago, are starting to repeat themselves across a number of different clients across a variety of industries.  Yes, the business value of big data is starting to become very real to the business folks.
• Business leaders are partnering with IT to find the right big data opportunities.  It is a joint partnership, where the business and IT must be equal partners to identify the right business opportunities upon which to target big data technologies.  That discussion requires the business stakeholders to step up as to how they are going to leverage these new insights for material, quantifiable business value, and for the IT stakeholders to do a realistic access of the data, technology, and organization’s capabilities for successful execution.

Needless to say, that discussion or engagement with the business and IT stakeholders is absolutely illuminating, and sets the organization down the right big data path with its eyes wide open.

• No two organizations, even within the same industry, have the same big data starting point.  Every organization seems to have a different big data starting point because each organization has different strategic business initiatives.  From predictive maintenance to customer acquisition to personalized marketing optimization to infrastructure performance optimization to on-time pick up optimization; each organization is focused on different parts of their value chain.

Business users want insights, not data and analytics.  The conversations have matured beyond organizations worrying about what data to capture, to a realization that it’s the insights buried in those different data sources that are actually used to optimize their internal business processes and uncover new monetization opportunities.  Organizations are staring to realize that you don’t monetize data; you monetize insights that are teased out of the data.

Leverage your existing business intelligence and data warehouse investments.
For many organizations, the best big data starting point is optimizing their existing, internal business processes.  And this is where business intelligence and data warehouse investments can be leveraged.  Organizations – through the existing business intelligence and data warehouse development efforts – have already gone through the process of defining their key business processes, identified the key performance indicators and key metrics, scoped out the data necessary to support the key business questions and the key business decisions, and built reports and dashboards that front those business processes.  Existing business processes, and the business intelligence and data warehouse work that has already been done, may be a good big data starting point for many organizations.

• The cloud is going to be a big deal.  The ability to store massive amounts of data – even data that you are not sure how you are going to use – and bring to bear vast amounts of computing power against that data will power new types of analysis and enable next generation, analytics-enabled applications.

• Lots of business value exists within your organizational “dark” data.  For many organizations, the starting point is the under-leveraged, detailed transactional data that gets aggregated and sampled to fit into their existing data warehouses.  This under-leveraged transactional data, the “dark data” of the organization, provides a “low-hanging fruit” starting point for many organizations’ big data journey.  There is lots of value in those TBs of customer loyalty data that resides on mag tape!!
• Organizations don’t grasp the importance of the economics of big data.
  Today’s data management and analytic environments are 20x cheaper than today’s data warehouse environments.  20x cheaper!  Leveraging these new economics of data and analytics requires organizations to think differently about what data they store at what level of detail, how they build data and analytic models, what sort of data transformation and enrichment algorithms they can build and execute, and what sort of user experiences they can provide.  Today’s debilitating SLAs can be cast aside in many situations, leveraging new architectural approaches that free the organization to ask questions of the business more frequently, in more real-time, and at a level of granularity never considered before.  Think differently.

I’m as excited as ever to be in the data and analytics business, because organizations are increasing realizing that everyone needs to be in the data and analytics business.  Organizations are quickly coming to the realization that if they don’t immerse themselves into the data and analytics business, their key competitors and new startups just might beat them to the punch.

Think differently.