Curated for content, computing, and digital experience professionals

Category: Enterprise search & search technology (Page 31 of 60)

Research, analysis, and news about enterprise search and search markets, technologies, practices, and strategies, such as semantic search, intranet collaboration and workplace, ecommerce and other applications.

Before we consolidated our blogs, industry veteran Lynda Moulton authored our popular enterprise search blog. This category includes all her posts and other enterprise search news and analysis. Lynda’s loyal readers can find all of Lynda’s posts collected here.

For older, long form reports, papers, and research on these topics see our Resources page.

Welcome Fred Dalrymple

Fred is our newest contributor, and has already posted his first blog entry. Fred pokes at the challenging tension in search between intent and context, especially over time as context (or intent) changes. Lynda has also posted about intent, and the subject also came up in discussions of search quality around Udi Manber’s talk at our conference in this past June.

Fred brings the welcome perspective of a serious software developer, and will be blogging on a few different topics, so he may be posting here on on one of our other blogs. Welcome Fred!

Researching Search with Intent Firmly in Control

I have hit on intent before and our latest member of the Gilbane blog team, Fred Dalrymple has joined the theme with his entry this week. Welcome Fred! You have given me an opening for an already planned topic, how to conduct research for enterprise search tools, those that go beyond the search box. Actually, this guidance is appropriate for the selection of any technology applications.

Getting intent solidly defined is important for so many reasons, many of them relating to solving a business problem and the expected outcomes. Knowing what these are will give you the framework for isolating likely candidates, efficiently. A second critical reason for having strong intent is to stave off project scope creep. As a former vendor, and now consultant, I see this play out repeatedly as product research ensues. Weak backbones in selection team members or flimsiness of their business case leaves openings for vendors to promote additional features, which often distracts from what is really needed.

So, armed with the right skeleton, a strong framework, a core scaffolding you are ready to approach your research systematically. Four paths are open to a study team; I recommend using all of them, in overlapping passes. Discovery about products, product performance in real-world scenarios, vendor business relationships with their clients, and the user community you will be joining are all targets that need to be exploited.

Discovering a user community on-line that might have expressed a potential problem with a vendor or product, should drive you back to do more research to discover potential limitations or why a user might be having a problem that they brought on through inappropriate implementation. Iteration in research for technology requires perseverance and patience. A comment on each path to research might be helpful:

  • Online research – This requires creativity and the most persistence to verify and validate what you find. I am amazed at how superficially many people read any content. We may be taught that good business writing requires a clear statement in the first paragraph of what follows with a solid summary at the end, but most content does not follow “good” business writing practices. You need to read between the lines, think about what is not being said and ask yourself why, follow every link on the sites of vendors under serious consideration. Look at vendor news notes and press releases to see how much activity is going on with product advances or new installations, and read descriptions of customer implementations to see how closely those deployments match your business need. Finally, search those customer names on the Internet in conjunction with the product name. This may retrieve public content that sheds more light on user experiences.
  • Professional groups – Professional organizations in which you participate are fertile ground for asking about what others in similar situations to yours are using. As you get closer to a final choice, go back to others you know personally or professionally to get answers to the direct question, “have you had any problems with this product or vendor?” and “what is the benefit of this product for you?”
  • Societies and academic institutions – These organizations publish content that may have a cost associated. When you consider thousands your organization spends on a selection process (in people time), contracting, licensing, implementation and deployment, it is wise to have a budget of several hundred dollars for reports that give detailed product evaluations. Get recommendations of librarians and peers as to publications’ authoritativeness.
  • User and analyst blogs and industry publications – The same guidance holds for industry publications as for societies and academic publishers but you will also want to pursue blogs of users and analysts. Users are a great source of discovering tidbits about products and vendors but continue beyond what you discover to see if the comments are isolated or follow a pattern.

This is a longer commentary than I intended but the core of my intent needed flesh, so there it is.

Beyond Intent

Intent, hidden within a search click, lies at the intersection of Search and Business, as in “let’s do some business”. That search click has extra-ordinary value because of the intent to buy — that’s why we’re searching, right?

Perhaps, or maybe we’re just browsing, or surfing, and we’re not in the mood for advertisements. It could be more militant than that; perhaps we’re still trying to research our choices and would see a sales pitch as tainting the honesty of the information. At least that’s what the founders of Google originally believed.

Although the model of the web was a set of stateless pages, and a Google search box certainly fits that appearance, people’s intent is not stateless. It ebbs and flows, from entertaining looking around, to researching choices and comparing possibilities, through sourcing a chosen product (now we’re talking about a qualified buyer), to selecting fulfillment options, and possibly all the way to figuring out how to return a product that we’re dissatisfied with. That last one is probably not the best time to present an ad claiming how wonderful that product is.

This is a “long running transaction,” a series of steps that fit together and flow towards (and past) a purchasing decision, but with back-currents and eddies. And it really is a transaction in the database sense where a failure during one step can cause the entire sequence to be discarded as if it never happened. Though if you believe Sergey and Larry, it will be worse than never happening, you may lose trust in your guide through that transaction.

Has the intent changed? Depends on what that means. On one hand, what has changed across those steps is the mode of the intent. If the intent was to purchase a product, then the research, comparison, purchase, and fulfillment were clearly pieces of that intent, though they call for different approaches: organic search for the research, product focused responses for the purchase, perhaps service-oriented for the fulfillment, and some combination for the comparison.

But what about that “I need to return this product because I hate it” step? The intent has clearly changed, but it is more necessary than ever to connect this new intent to the previous steps. If not, perhaps the search engine will continue to suggest that product to a disgruntled customer with very counter-productive results.

So, what is the unifying concept? Is it intent, organized by modes? Not if what is being unified is a complete user’s story about their purchasing experience.

Before You Start a List of Vendors: Map Your Course

There is a pattern in how many small to mid-sized enterprises go about researching technology applications, one that does not serve them well. As I interact with colleagues, business affiliates and professional peers, I play into this behavior unwittingly. For example, how many times have you been on the asking or answering side of this question: “My organization is planning to procure a search system this year, what systems should we be looking at?” Too often, I make a quick judgment based on what little I know about the asker and toss out a few potential candidate vendor names.

This scenario plays out frequently and now I am uncomfortable because, as a consultant and search analyst, I know that there is a lot more I need to know before offering off-handed advice to that question. Here are some ideas for questions that you should be asking first so that, when someone like me wants more context, you have ready answers.

Your first step is to survey your internal landscape and clearly document the following:

  • What are the business outcomes you expect to derive from the search product, who will be using it, under what circumstances and for what purpose?
  • What is the scope of the content that will be indexed for retrieval? Create a content map that explicitly illustrates: What, Where, Who, When. This means capturing what the content is in terms of document types and formats, numbers and size, and topic, and where it is being created, stored and managed. You need to know who created it, owns it, and will have access to it. Finally, it helps to document when it was created and information about retention.
  • Who will be involved in product selection and evaluation, who needs to sign off at every stage of selection and procurement, who will be involved in installation and deployment, and who will maintain the system on an ongoing basis?
  • What is your IT infrastructure and who controls it? If a schematic is not in place that depicts at least the portion of the computing infrastructure that will be integral to your search support, it is time to make sure one is prepared. You cannot make an informed decision about appropriate and workable search solutions without this information.

You will also be wasting the time of vendors when you seek product and licensing information if you do not have all of these issues sorted out. Much of the packaging of search products is dependent on numbers of documents or size of the corpus to be indexed, how the software will be installed, and who and how many will be accessing it. Pricing information will be vague until you have concrete content “demographics” to share with prospective vendors. You can’t even establish a budget without answering the questions above, and you need a ballpark budget figure to help narrow your choices.

So, I am resolving to be more thoughtful in my responses when queried by friends and colleagues. Before answering I will be asking you for some meaningful data in advance of reeling off a list of products. It is time for you to do some preliminary research in-house before establishing the lineup of suitors. More on the next steps, next time up.

Taking Measure of Search on Vendor Sites

As I was developing concepts put forth in the report Enterprise Search Markets and Applications – Capitalizing on Emerging Demand I bounced around the Internet a lot to verify information I had previously noted about products listed in the vendor directory. As I did so, evidence began to emerge about the ease with which I could resurrect an earlier retrieved bit of content. It mystified me that vendors of products to aid retrieval of content would make it so difficult to find information on their own web site. One assumption of mine has been completely debunked, that vendors would use their own search product to help site visitors discover more about their products and services. It made me wonder why they would not be showcasing the full flavor of their offerings.

The report was not written to evaluate specific products but rather to give a more holistic view of how the markets for products break down and how products themselves can be categorized. In order to do the latter, it required reading about many products with which I had no hands on experience. I wanted to understand how vendors were positioning their products, what markets they felt their products are most suited to satisfy, and what search problems were best solved with their technologies. Coming up with generalizations, trends, and differentiators was one purpose for my research. When I realized how difficult it was to dig out specifics from many vendor Web sites, I moved on, probably leaving stones unturned but time was not on my side.

Now I am going back to learn more about the problem with researching search, something complained about by a number of buyers I interviewed. Vendors are not making it easy for buyers to narrow their search for search, and shame on them. This should be a “no brainer.” If you are a vendor pushing a product that is easy to install, implement and deploy, there is no better way than to put it to work on your own site. On the other hand, if you have products that are more sophisticated in terms of offering complex retrieval by leveraging refined ontologies or rules, you had better take the time to make it work well for finding nuggets on a few hundred pages of your Web site.

I am going to be writing more about this because the deeper I dig, the more interesting the results. For starters, of the first 28 vendor on my list, twelve have no site search. Of those that do, several use a third-party search engine, not their own. One major vendor’s search result count displayed nearly a hundred records that matched the search while also displaying the breakdown of records by category. The trouble was the category numbers totaled less than 20. Hmmm!

Perhaps the trouble in “searchland” is that no one wants to take the time to implement, deploy and maintain search to satisfy the user. I keep saying, “it’s not the technology; it’s the thought and skill that goes into the back room implementation.” Or is it? Stay tuned.

Search is Not Taking a Summer Break & Call for Papers

Amidst post Gilbane San Francisco business. I have been reading what everyone else has been writing about search the past couple of months. While there continues to be much speculation and gossip about the Microsoft acquisition of FAST, and which companies may soon be absorbed into larger entities, there also continues to be interesting activity among the mid-tier and start-up search vendors. Meanwhile, I advise those who aspire to acquire a search solution for “behind the firewall,” don’t wait for the “big players” to come up with the definitive solution to all your search needs because it will never happen. I’m in good company with other analysts who advise moving on with point search solutions for specific business needs. You will save money, and time because most of the new products are optimized for rapid deployment, in weeks or months, not years.

If you check out my new research report, Enterprise Search Markets and Applications; Capitalizing on Emerging Demand, June, 2008, you will find a directory to companies offering search solutions with choices for what Steve Arnold refers to as “beyond search.” Deep test drives of many of these products can be found in his report, as well. Meanwhile, new releases of products listed, and new products both continue to be announced. ISYS, Coveo and Expert System (Cogito) have brought new offerings to market in the last month and Collexis, a relative newcomer, is drawing attention to itself by demonstrating its products at numerous meetings this year.

So, keeping reading and checking out the possibilities. While you are at it, be sure to put the Gilbane Boston Conference on your calendar for December 3 – 4. We are all busy rounding out the program right now.

I am particularly interested in hearing from those of you who have participated in the selection of a search product in the past two years, implementing or deploying a system anywhere within your own enterprise. Please consider sending me a brief proposal for a presentation at the conference. For your effort, you will get to attend all the conference sessions, as well as help the audience with the needed reality checks on what it takes to conduct a selection process and follow through with implementation. I particularly want you to share your learning experiences: the good, the frustrating, and the lessons you have accrued. Professional speaking experience is not required – we want stories. [You’ll find my email on the “Contacts” page of the Gilbane site and you should also look at the speakers guidelines for additional information.]

Afterthoughts on the State of Search

Search for the whole enterprise vs. point solutions was the subject of some discussion, especially since our keynote speaker, Stephen Arnold gave strong guidance that you can’t think about one search solution (“product”) for the entire enterprise and all content. This is something with which I pretty much agree, in most cases.

Just emerging from the Gilbane San Francisco conference, six sessions on search and a workshop I conducted, I want to share a couple of general impressions. Details and expanded reflections will follow in the days and weeks to come.

Search for the whole enterprise vs. point solutions was the subject of some discussion, especially since our keynote speaker, Stephen Arnold gave strong guidance that you can’t think about one search solution (“product”) for the entire enterprise and all content. This is something with which I pretty much agree, in most cases. However, a question arose in one of the sessions in which a couple of presentations talked about a single search engine for what appeared to be the entire enterprise. A member of the audience asked for clarification in view of Arnold’s earlier comments.

I chose to intercede so as not to put our speakers on the defensive about what, for their organizations were very reasonable choices. Both of the cases were for research or professional services organizations with a high incidence of uniformity in the scope and type of content. They are relatively flat in structure with the bulk of the population being researchers: consultants, engineers, scientists. The applications were for intranets that were being leveraged to connect content and experts, so that from either direction (finding an expert and then looking at their content, or finding content to reveal expertise) other professionals could leverage organizational knowledge. It is a safe bet that other search does exist elsewhere in these companies, even if it is in stealth mode or embedded in other applications. Still, in general, large organizations with highly differentiated personnel with functional and disparate content requirements will find value in point search solutions that may only have purpose in a single internal domain.

To that point, if you are a finance professional or business manager you might want to sign up for a webinar this Thursday, June 26th, when I will be laying out a business case for a particular kind of search solution that is targeted at your demographic. This Apps Associates sponsored webinar also describes a solution leveraging Oracle enterprise search, but the ideas in it will give you a sense of what search can provide in your domain.

Judging from the topics presented on search, the reasons and ways in which it is being applied are more diverse than even I imagined. Opinions about what is good/bad, appropriate or not, and how to approach search technology ran the gamut of simple to complex. Two strong points of view were expressed about taxonomy vs. just tagging or letting the search engine categorize. Neither side would give an inch to the other as having an approach that is often “good enough.” It is pretty clear that hybrid solutions offering both a structured approach to search where a taxonomy is applied through metadata, and auto-categorization by the search engine without a supporting taxonomy in the background will be applied in many enterprises.

What is Semantic Technology Anyway?

Meaning is a very large concept in every aspect of search technology and dozens of search product sites include either the words “semantic” or “meaning” as a key element of the offered technology. This is not as far fetched as search product claims to “know” what the searcher wants to find, as if “knowing” can be attributable to non-human operations. However, how well a search engine indexes and retrieves content to meet a searcher’s intent, is truly in the eyes of the beholder. I can usually understand why, technically speaking, a piece of content turns up in a search result, but that does not mean that it was a valid scrap for my intent. My intent for a search cannot possibly be discernible by a search engine if, as is most often the case, I don’t explicitly and eloquently express what, why, and other contextual facts when entering a query.

The session we have set aside at Gilbane San Francisco for a discussion on current activity related to semantic technologies will undoubtedly reveal more meaning about technologies and art of leveraging tools to elicit semantically relevant content. I suspect that someone will also stipulate that what works requires a defined need and clear intent during the implementation process – but what about all those fuzzy situations? I hope to find out.

This is the last posting before the conference this week so I hope you will add this enterprise search session (EST-6: Semantic Technology – Breakdown or Breakthrough) being moderated by Colin Britton to your agenda on June 19th. He will be joined by speakers: Steve Carton, VP Content Technologies, Retrieval Systems Corp., Folksonomies: Just Good Enough for all Kinds of Thing, Prakesh Govindarajulu, President, RealTech Inc, Building Enterprise Taxonomies from the Ground Up, and Jack Jia, Founder & CEO, Baynote.

See you in San Francisco in person or virtually thereafter.

« Older posts Newer posts »

© 2024 The Gilbane Advisor

Theme by Anders NorenUp ↑