Dispatch from the Strata Front

An Apache Hadoop leader, IoT pioneer, chipmaker and independent consultant walked into a bar…

Oops, wrong story.

In truth, employees representing this eclectic group convened for today’s Birds of a Feather session at the Strata + Hadoop conference in San Jose, CA. My colleague Marie Couch and I digested a few key takeaways as we hosted this lively lunch, which focused on Best Practices for Data Consolidation for Hadoop Analytics.

Big Data initiatives are still in the early stage. I noted last week that Big Data and the Internet of Things (IoT) are near the Peak of Inflated Expectations, on their way to the Trough of Disillusionment, on Gartner’s Hype Cycle. The consultant that joined us noted that more than half his clients are only beginning to get their arms around Big Data opportunities and their strategy. Others noted the lack of experienced Data Scientists as a major inhibitor. A lot of potential Big Data remains untapped.

Ingestion still causes distress. Many show attendees seek to consolidate data onto Hadoop from many sources, and struggle with the manual ETL coding required to make that happen. No news here, but we are pleased that Attunity solutions continue to address what remains a major pain point in today’s 80-20 model.

The Hadoop Data Lake is the preferred option. New BI tools such as ClearStory Data seek to help customers analyze data across platforms, selectively pre-integrating some data sets and feeding in others via live APIs. Our table prefers the established method of centralizing all data points into a single repository, ideally Hadoop, to open as wide a range of analytical options as possible. Hadoop will not be the authoritative source of financial transactions, but often it will be the single version of truth for analytics.

Customers need a dashboard. Specifically, enterprises seek to monitor multiple concurrent data feeds as closely as possible, and a dashboard such as Maestro is the preferred method.

This was an educational discussion with some sharp Big Data practitioners. We’ll keep you posted on tomorrow’s session!