Recently in Definitions Category

This one almost slipped right past me but I see we are in another shoot-out in the naming of search market segments. Probably it is because we have too many offerings in the search industry. When any industry reaches a critical mass, players need to find a way to differentiate what they sell. Products have to be positioned as, well, "something else."

In my consulting practice "knowledge management" has been hot (1980s and 90s), dead (late '90s and early 2000s), relevant again (now). In my analyst role for "enterprise search" Gilbane has been told by experts that the term is meaningless and should be replaced with "behind the firewall search," as if that clarifies everything. Of course, marketing directories might struggle with that as a category heading.

For the record, "search" has two definitions in my book. The first is a verb referring to the activity of looking for anything. The second, newer, definition is a noun referring to technologies that support finding "content." Both are sufficiently broad to cover a lot of activities, technologies and stuff. "Enterprises" are organizations of any type in which business, for-profit, non-for-profit, or government, is being conducted. Let us quibble no more.

But I digress; Endeca has broadened its self-classification in any number of press releases to referring to its products that were "search" products last year, as "information access software." This is the major category used by IDC to include "search." That's what we called library systems in the 1970s and 80s. New products still aim for accessing content, albeit with richer functions and features but where are we going to put them in our family of software lists? One could argue that Endeca's products are really a class of "search," search on steroids, a specialized form of search. What are the defining differentiators between "search software" and "information access software?" When does a search product become more than it was or narrower, refined in scope? (This is a rhetorical question but I'm sure each vendor in this new category will break-it out for me in their own terms.)

Having just finished reviewing the market for enterprise search, I believe that many of the products are reaching for the broader scope of functionality defined by IDC as being: search and retrieval, text analytics, and BI. But are they really going to claim to be content management and data warehousing software, as well? Those are included in IDC's definition of "information access software." May-be we are going back to single-vendor platforms with everything bundled and integrated. Sigh... it makes me tired, trying to keep up with all this categorizing and re-redefining.

Steve Arnold's Beyond Search report is finally launched and ready for purchase. Reviewing it gave me a different perspective on how to look at the array of 83 search companies I am juggling in my upcoming report: Enterprise Search Markets and Applications. For example, technological differentiators can channel your decisions about must haves/have nots in your system selection. Steve codifies considerations and details 15 technology tips that will help you frame those considerations.

We are getting ready for the third Gilbane Conference in which "search" has been a significant part of the presentation landscape in San Francisco, June 17 - 20th.Six sessions will be filled with case studies and enlightening "how-to-do-it-better" guidance from search experts with significant "hands-on" experience in the field. I will be conducting a workshop, immediately after the conference, How to Successfully Adopt and Deploy Search. Presentations by speakers and the workshop will focus on users' experiences and guidance for evaluating, buying and implementing search. Viewing search from a usage perspective begs a different set of classification criteria for divvying up the products.

In February, Business Trends published an interview I gave them in December, Revving up Search Engines in the Enterprise. There probably isn't much new in it for those who routinely follow this topic but if you are trying to find ways to explain what it is, why and how to get started, you might find some ideas for opening the discussion with others in your business setting. The intended audience is those who don't normally wallow in search jargon. This interview pretty much covers the what, why, who, and when to jump into procuring search tools for the enterprise.

For my report, I have been very pleased with discussions I've had with a couple dozen people immersed in evaluating and implementing search for their organizations. Hearing them describe their experiences guides other ways to organize a potpourri of search products and how buyers should approach their selection. With over eighty products we have a challenge in how to parse the domain. I am segmenting the market space into multiple dimensions from the content type being targeted by "search" to the packaging models the vendors offer. When laying out a simple "ontology" of concepts surrounding the search product domain, I hope to clarify why there are so many ways of grouping the tools and products being offered. If vendors read the report to decide which buckets they belong in for marketing and buyers are able to sort out the type of product they need, the report will have achieved one positive outcome. In the meantime, read Frank Gilbane's take on the whole topic of enterprise tacked onto any group of products.

As serendipity would have it, a colleague from Boston KM Forum, Marc Solomon, just wrote a blog on a new way of thinking of the business of classifying anything, "Word Algebra." And guess who gave him the inspiration, Mr. Search himself, Steve Arnold. As a former indexer and taxonomist I appreciate this positioning of applied classification. Thinking about why we search gives us a good idea for how to parse content for consumption. Our parameters for search selection must be driven by that WHY?

Called to account for the nomenclature “enterprise search,” which is my area of practice for The Gilbane Group, I will confess that the term has become as tiresome as any other category to which the marketplace gives full attention. But what is in a name, anyway? It is just a label and should not be expected to fully express every attribute it embodies. A year ago I defined it to mean any search done within the enterprise with a primary focus of internal content. “Enterprise” can be an entire organization, division, or group with a corpus of content it wants to have searched comprehensively with a single search engine.

A search engine does not need to be exclusive of all other search engines, nor must it be deployed to crawl and index every single repository in its path to be referred to as enterprise search. There are good and justifiable reasons to leave select repositories un-indexed that go beyond even security concerns, implied by the label “search behind the firewall.” I happen to believe that you can deploy enterprise search for enterprises that are quite open with their content and do not keep it behind a firewall (e.g. government agencies, or not-for-profits). You may also have enterprise search deployed with a set of content for the public you serve and for the internal audience. If the content being searched is substantively authored by the members of the organization or procured for their internal use, enterprise search engines are the appropriate class of products to consider. As you will learn from my forthcoming study, Enterprise Search Markets and Applications: Capitalizing on Emerging Demand, and that of Steve Arnold (Beyond Search) there are more than a lot of flavors out there, so you’ll need to move down the food chain of options to get it right for the application or problem you are trying to solve.

OK! Are you yet convinced that Microsoft is pitting itself squarely against Google? The Yahoo announcement of an offer to purchase for something north of $44 billion makes the previous acquisition of FAST for $1.2 billion pale. But I want to know how this squares with IBM, which has a partnership with Yahoo in the Yahoo edition of IBM’s OmniFind. This keeps the attorneys busy. Or may-be Microsoft will buy IBM, too.

Finally, this dog fight exposed in the Washington Post caught my eye, or did one of the dogs walk away with his tail between his legs? Google slams Autonomy – now, why would they do that?

I had other plans for this week’s blog but all the Patriots Super Bowl talk puts me in the mode for looking at other competitions. It is kind of fun.

It has been a week since the annual Gilbane Boston 2007 Conference closed and I am still searching for the most important message that came out of Enterprise Search and Semantic Web Technology sessions. There were so many interesting case studies that I’ll begin with a search function that illustrates one major enterprise search requirement – aggregation.

Besides illustrating a business case for aggregating disparate content using search, the case studies shared three themes:
> Search is just a starting point for many business processes
> While few very large organizations present all of their organization’s content through a single portal, the technology options to manage such an ideal design are growing and up to supporting entire enterprises
> All systems were implemented and operational for delivering value in less than one year, underscoring the trend toward practical and more out-of-the box solutions

Here is a brief take on what came out of just the first two of seven sessions.
Small-medium solutions:
> Use of ISYS to manipulate search results and function as a back-office data analysis tool for DirectEDGAR, the complete SEC filings, presented by Prof. Burch Kealey of the University of Nebraska. Presentation
> Support for search by serendipity across the shareable content domains of members of a trade association (ARF) by finding results that satisfy the searcher in his pursuit of understanding with Exalead, presented by Alain Heurtebise CEO of Exalead. Presentation
> A knowledge portal enabling rapid and efficient retrieval of the complete technical documentation for field service engineers at Otis Elevator to meet rapid response goals when supporting customers using a customized implementation of dtSearch, presented by project consultant Rob Wiesenberg of Contegra Systems, Inc. Presentation

Large solutions calling for search across multi-million record domains:
> Hosted Vivisimo solution federating over 40 million documents across 22,000 government web sites accessible with search results clustered; it records over a half million page views per day on http://USA.gov and was deployed in 8 weeks, presented by Vivisimo co-founder Jerome Pesenti. Presenation
> Intranet knowledge portal for improving customer services by enabling access to internal knowledge assets (over half a million customer cases with all their associated documents) at USi (an AT&T company) using Endeca, a search product USi had experience deploying and hosting for very large e-commerce catalogs, presented by development leader Toby Ford of USi. With one developer it was running in six months. Presentation
> Within a large law firm (Morrison Foerster) and the legal departments of two multi-national pharmaceutical companies (Pfizer and Novartis), Recommind aggregates and indexes content for numerous internal application repositories, file shares and external content sources for unified search across millions of documents, contributing a direct ROI in saved labor by ensuring that required documents are retrieved in a single search process. Presentation

In each of these cases, content from numerous sources was aggregated through the crawling and indexing algorithms of a particular search engine pointed at a bounded and defined corpus of content, with or without associated metadata to solve a particular business problem. In each case, there were surrounding technologies, human architected design elements, and interfaces to present the search interface and results for a predefined audience. This is what we can expect from search in the coming months and years, deployments to meet specialized enterprise needs, an evolving array of features and tools to leverage search results, and a rapid scaling of capabilities to match the explosion of enterprise content that we all need to find and manipulate to do our jobs.

Next week, I will reconstruct more themes and messages from the conference.

Structured search (noun) was rooted firmly in the enterprise when publishers of print index resources (e.g. Chemical Abstracts, Index Medicus from the National Library of Medicine, GRA&I from the National Technical Information Service) became available on-line in the early 1970s. The Systems Development Corporation launched ORBIT developed by a team lead by Carlos Cuadra. Orbit was a command driven search tool accessible to professional searchers. In those days searchers were usually special librarians in corporations, large public libraries, government agencies and major universities. Using the ORBIT command language through a terminal connected by a phone line to remote large computers, librarians would type search commands to find data in specific structured fields. These remote computers held electronic versions of paper indices. Citations resulting from a query for specific chemical compounds, diseases, or government reports, would contain information needed to retrieve articles, patents or books from library shelves.

Corporations spent hundreds of thousands of dollars each year to access external specialized, and structured indices, and the journals, conference proceeding, patents and government documents to which the indices pointed. Hard copy (paper or microform) was the only practical way to read content. Computer screens were not accessible to most researchers and even if they had been, content could not be rendered on them in easily readable forms. Also, until computer storage technologies became cheap, indexing large amounts of text (full-text, or unstructured content) was not affordable.

Even with the advent of graphical interfaces, searching for non-specialists made only minor advances in the early-1980s when library systems offered index browsing to find citations. Library users still needed to read content in hard copy. It was only in the late 1980s and early 90s that full-text content began to be searchable by large numbers of library users on CD-ROMs. Users would go to a library computer, which held multiple CD-ROMs containing journals and other subscriptions, and use a menu to find content on the CD-ROMs by typing keywords that would look through all the content to find matches. This was the first routine use of full-text searching by library users.

These technologies are just memories for a few of us, and unknown to most, but they do point to the differentiation between structured and unstructured searching. Both have been around for a couple of decades but it has taken Web search engines to put search in the hands of everyone. Only recently is frustration with retrieving buckets of unfiltered content pushing enterprises to reconfirm the added value of structured searching.

Technical and business users are appreciating the value of being able to search for a precise title, all documents contributed to a specific project, or all presentations delivered by the CEO in the past two years. Each of these searches requires a defined set of data points, stored with the content and retrievable with a search interface that can support the “structured” query.

Yes, librarians have been here before but, just now, the rest of the organization is learning how they managed to get such good search results all along. Structured searching is now a lot simpler than it was in the 1970s. It is only one aspect in enterprise search but it is an important requirement for most enterprise users when they need reliable and clearly defined search results. And, by the way, Carlos is still around building systems for enterprises to manage and search their critical proprietary content.

If you signed up for feeds from this site, new posts have been slow coming. Gilbane's announcement of an Enterprise Search Practice has not gone unnoticed. The past two weeks have resulted in more good will than this analyst could easily digest and filter. The good news is that ideas for posting on "enterprise search" are already accumulating faster than they can get written, and the number of enthusiastic well-wishers is encouraging. It looks like we have an audience and community of practice in the making. Thank you to all who have sent their support and good cheer.

Quite a number of responses have come from companies who want to discuss their technology offerings and positioning. At Gilbane we are following up on those requests and beginning to schedule time for discussions and presentations. With the recognition that vendors/suppliers of technologies want ink, and plenty of it, comes a responsibility of which I am acutely aware because I was one of that community for over 20 years. Having founded, in 1980, and lead an integrated library automation firm in the corporate arena, I know how industry press coverage can make or break the fortunes of even the best offerings. While blogs are intended to launch and promote discussions, even play devil's advocate, I don't take this role lightly. Every good intention and hard work by vendors deserves thoughtful and unbiased consideration. It deserves to have analysts who know what they are talking about, and those that would present what they can fairly assess in a useful context. The very definition of analyst (noun) supposes a responsible action, to analyze (verb) the offerings. While my analysis may not focus on what a vendor wants me to consider, it will try to present information that is both helpful and thought-provoking without being mean-spirited or dismissive, and content that helps potential users of the technology focus their own choices and decisions.

Now it's time to get down to business and start making this a more frequent happening. Based on a number of comments, let's begin with clarifying what we mean by enterprise search at Gilbane. While the marketplace often categorizes enterprise search as a specific kind of search product, we at Gilbane don’t. Any technology that serves any type of enterprise by helping it find electronic or physical content through an electronic search interface is fair game. Enterprise search is about looking for content in the organization or for the organization. It may be embedded in a specialized application, may be a platform designed to collectively search and aggregate content from many internal silos, or it may combine search of desktops, enterprise hard drives and the Internet. There is a very big universe of content out there; enterprises need all the search tools they can (afford to) leverage to harvest what they need and when they need it.

Now this analyst’s job is to give you a balance between what the vendors are saying and offering, and what the users really need, and get the two engaging more effectively with each other.