Gilbane Conferences & Advisor

Curated content for content, computing, and digital experience professionsals

Category: Big data

Customer experiences, communications, and analytics

three epicenters of innovation in modern marketing
I recently discovered Scott Brinker’s Chief Marketing Technologist blog and recommend it as a useful resource for marketers. The Venn diagram above is from a recent post, 3 epicenters of innovation in modern marketing. It was the Venn diagram that first grabbed my attention because I love Venn diagrams as a communication tool, it reminded me of another Venn diagram well-received at the recent Gilbane Conference, and most of the conference discussions map to someplace in the illustration.

As good as the graphic is on its own, you should read Scott’s post and see what he has to say about the customer experience “revolution”.

Lest you think Scott is a little too blithe in his acceptance of the role of big data, see his The big data bubble in marketing — but a bigger future, where the first half of the (fairly long) post talks about all the hype around big data. But you should read the full post because he is right on target in describing the role of big data in marketing innovation, and in his conclusion that data-driven organizations will need to make use of big data though these data-driven and data-savvy organizations will take some time to build.

So don’t let current real or perceived hype about the role of big data in marketing lead you to discount its importance – it’s a matter of when, not if. “When” is not easy to predict, but will certainly be different depending on an organizations’ resources and ability to deal with complexity, and organizational and infrastructure changes.

Enterprise Search Strategies: Cultivating High Value Domains

At the recent Gilbane Boston Conference I was happy to hear many remarks positioning and defining “Big Data” and the variety of comments. Like so much in the marketing sphere of high tech, answers begin with technology vendors but get refined and parsed by analysts and consultants, who need to set clear expectations about the actual problem domain. It’s a good thing that we have humans to do that defining because even the most advanced semantics would be hard pressed to give you a single useful answer.

I heard Sue Feldman of IDC give a pretty good “working definition” of big data at the Enterprise Search Summit in May, 2012. To paraphrase is was:

  • > 100 TB up to petabytes, OR
  • > 60% growth a year of unstructured and unpredictable content, OR
  • Ultra high streaming content

But we then get into debates about differentiating data from unstructured content when using a phrase like “big data” and applying it to unstructured content, which knowledge strategists like me tend to put into a category of packaged information. But never mind, technology solution providers will continue to come up with catchy buzz phrases to codify the problem they are solving, whether it makes semantic sense or not.

What does this have to do with enterprise search? In short, “findability” is an increasingly heavy lift due to the size and number of content repositories. We want to define quality findability as optimal relevance and recall.

A search technology era ago, publishers, libraries, content management solution providers were focused on human curation of non-database content, and applying controlled vocabulary categories derived from decades of human managed terminology lists. Automated search provided highly structured access interfaces to what we now call unstructured content. Once this model was supplanted by full text retrieval, and new content originated in electronic formats, the proportion of human categorized content to un-categorized content ballooned.

Hundreds of models for automatic categorization have been rolled out to try to stay ahead of the electronic onslaught. The ones that succeed do so mostly because of continued human intervention at some point in the process of making content available to be searched. From human invented search algorithms, to terminology structuring and mapping (taxonomies, thesauri, ontologies, grammar rule bases, etc.), to hybrid machine-human indexing processes, institutions seek ways to find, extract, and deliver value from mountains of content.

This brings me to a pervasive theme from the conferences I have attended this year, the synergies among text mining, text analytics, extractor/transformer/loader (ETL), and search technologies. These are being sought, employed and applied to specific findability issues in select content domains. It appears that the best results are delivered only when these criteria are first met:

  • The business need is well defined, refined and narrowed to a manageable scope. Narrowing scope of information initiatives is the only way to understand results, and gain real insights into what technologies work and don’t work.
  • The domain of content that has high value content is carefully selected. I have long maintained that a significant issue is the amount of redundant information that we pile up across every repository. By demanding that our search tools crawl and index all of it, we are placing an unrealistic burden on search technologies to rank relevance and importance.
  • Apply pre-processing solutions such as text-mining and text analytics to ferret out primary source content and eliminate re-packaged variations that lack added value.
  • Apply pre-processing solutions such as ETL with text mining to assist with content enhancement, by applying consistent metadata that does not have a high semantic threshold but will suffice to answer a large percentage of non-topical inquiries. An example would be to find the “paper” that “Jerry Howe” presented to the “AMA” last year.

Business managers together with IT need to focus on eliminating redundancy by utilizing automation tools to enhance unique and high-value content with consistent metadata, thus creating solutions for special audiences needing information to solve specific business problems. By doing this we save the searcher the most time, while delivering the best answers to make the right business decisions and innovative advances. We need to stop thinking of enterprise search as a “big data,” single engine effort and instead parse it into “right data” solutions for each need.

Why Big Data is important to Gilbane Conference attendees

If you think there is too much hype, and gratuitous use of the term, big data, you haven’t seen anything yet. But don’t make the mistake of confusing the hype with how fundamental and how transformational big data is and will certainly be. Just turn your hype filter to high and learn enough about it to make your own judgements about how it will affect your business and whether it is something you need to do something about now, or monitor for future planning.

As I said yesterday in a comment on a post by Sybase CTO Irfan Khan Gartner dead wrong about big data hype cycle (with a response from Gartner):

However Gartner’s Hype Cycle is interpreted I think it is safe to say that most, including many analysts, underestimate how fundamental and how far-reaching big data will be. How rapidly its use will evolve, and in which applications and industries first, is a more difficult and interesting discussion. The twin brakes of a shortage of qualified data scientist skills and the costs and complexities of IT infrastructure changes will surely slow things down and cause disillusionment. On the other hand we have all been surprised by how fast some other fundamental changes have ramped up, and BDaaS (Big Data as a Service) will certainly help accelerate things. There is also a lot more big data development and deployment activity going on than many realize – it is a competitive advantage after all.

There is also a third “brake” which is all the uncertainty around privacy issues. There is already a lot of consumer data that is not being fully used because of fear of customer backlash or new regulation and, one hopes, because of a degree of respect for consumer’s privacy.

Rob Rose expanded on some specific concerns of marketers in a recent post Big Data & Marketing – It’s A Trap!, including the lack of resources for interpreting even the current mostly website analytics data marketers already have. It’s true, and not just for smaller companies. In addition there are at least four requirements for making big data analytics accessible to marketers that are largely beyond the reach of most current organizations.

Partly to the rescue is Big Data as a Service BDaaS (one of the more fun-sounding acronyms). BDaaS is going to be a huge business. All the big technology infrastructure firms are getting involved and all the analytics vendors will all have cloud and big data services. There are also many new companies including some surprises. For example, after developing its own Hadoop-based big data analytics expertise Sears created subsidiary MetaScale to provide BDaaS to other enterprises. Ajay Agarwal from Bain Capital Ventures predicts that the confluence of big data and marketing will lead to several new multi-billion dollar companies and I think he is right.

But while big data is important for the marketers, content managers, and IT who attend our conference because of the potential for enhanced predictive analytics and content marketing. The reach and value of big data applications is far broader than marketing – executives need to understand the potential for new efficiencies, products and businesses. The well-known McKinsey report “Big Data: The Next Frontier for Innovation, Competition, and Productivity” (free) is a good place to start. If you are in the information business I focus on that in my report Big-Data: Big Deal or Just Big Buzz? (not free).

Big data presentations at Gilbane Boston

This year we have six presentations on big data, two devoted to big data and marketing and all chosen with an eye towards the needs of our audience of marketers, content strategists, and IT. You can find out more about these presentations, including their date and time on the conference program.

Keynote

Bill Simmons, CTO, DataXu
Why Marketing Needs Big Data

Main Conference Presentations

Tony Jewitt, VP Big Data Solutions at Avalon Consulting, LLC
“Big Data” 101 for Business

Bryan Bell, Vice President, Enterprise Solutions, Expert System
Semantics and the Big Data Opportunity

Brian Courtney, General Manager of Operations Data Management, GE Intelligent Platforms
Leveraging Big Data Analytics

Darren Guarnaccia, Senior VP, Product Marketing, Sitecore
Big Data: What’s the Promise and Reality for Marketers?

Stefan Andreasen, Founder and Chief Technology Officer, Kapow Software
Big Data: Black Hole or Strategic Value?

Update: There is now a video of me being interviewed on big data by CMS-Connected.

IBM Unveils Big Data Software

IBM unveiled new software for managing and analyzing big data to the workplace. The new offerings span a wide variety of big data and business analytics technologies across multiple platforms from mobile devices to the data center to IBM’s SmartCloud. Now employees from any department inside an organization can explore unstructured data such as Twitter feeds, Facebook posts, weather data, log files, genomic data and video, and make sense of it as part of their everyday work experience. IBM is also placing the power of mobile analytics into the hands of iPad users with a free download in Apple’s iTunes Store. The new software is designed to help employees in industries such as financial services, healthcare, government, communications, retail, and travel and transportation use and benefit from business analytics on the go. IBM is delivering new analytics and information management offerings: New Hadoop-based analytics software on the cloud, which helps employees tap into massive amounts of unstructured data from a variety of sources including social networks, mobile devices and sensors; New mobile analytics software for iPad users; and new predictive analytics software with a mapping feature that can be used across industries for marketing campaigns, retail store allocation, crime prevention, and academic assessment. http://www.ibm.com/