This plays out in countless variations but the bottom line is that impatience with human processing time and the assumption that a search engine "ought to be able to" solve this problem without human intervention cripple possibilities for success faster than anything else.
It is time for search implementation teams to get realistic about the tasks that must be executed and milestones to be reached. Teams must know how they are going to measure success and reliability, then to stick with it, demanding that everyone agrees on the requirements before throwing the towel in at the first executive anecdote that the "dang thing doesn't work."
There are a lot of steps to getting even an out-of-the-box solution working well. But none is more important than paying attention to these:
• Know your content
• Know your search audience
• Know what needs to be found and how it will be looked for
• Know what is not being found that should be
The operative verb here is to know and to really know anything takes work, brain work, iterative, analytical and thoughtful work. When I see these reactions from IT upon setting off a search query that returns any results: "we're done" OR "no error messages, good" OR "all these returns satisfy the query," my reaction is:
• How do you know the search engine was really looking in all the places it should?
• What would your search audience be likely to look for and how would they look?
• Who is checking to make sure these questions are being answered correctly?
• How do you know if the results are complete and comprehensive?
It is the last question that takes digging and perseverance. It is pretty simple to look at search results and see content that should not have been retrieved and figure out why it was. Then you can tune to make sure it does not happen again.
To make sure you didn't miss something takes systematic "dog work" and you have to know the content. This means starting with a small body of content that it is possible for you to know, thoroughly. Begin with content representative of what your most valued search audience would want to find. Presumably, you have identified these people through establishing a clear business case for enterprise search. (This is not something for the IT department to do but for the business team that is vested in having search work for their goals.) Get these "alpha worker" searchers to show you how they would go about trying to find the stuff they need to get their work done every day, to share with you some of what they consider some of the most valuable documents they have worked with over the past few years. (Yes, years - you need to work with veterans of the organization whose value is well established, as well as with legacy content that is still valuable.)
Confirm that these seminal documents are in the path of the search engine for the index build; see what is retrieved when they are searched for by the seekers. Keep verifying by looking at both content and results to be sure that nothing is coming back that shouldn't and that nothing is being missed. Then double the content with documents on similar topics that were not given to you by the searchers, even material that they likely would never have seen that might be formatted very differently, written by different authors, and more variable in type and size but still relevant. Re-run the exact searches that were done originally and see what is retrieved. Repeat in scaling increments and validate at every point. When you reach points where content is missing from results that should have been found using the searcher's method, analyze, adjust, and repeat.
A recent project revealed to me how willing testers are to accept mediocre results when it became apparent how closely content must be scrutinized and peeled back to determine its relevance. They had no time for that and did not care how bad the results were because they had a pre-defined deadline. Adjustments may call for refinements in the query formulation that might require an API to make it more explicit, or the addition of better category metadata with rich cross-references to cover vocabulary variations. Too often this type of implementation discovery signals a reason to shut down the project because all options require human resources and more time. Before you begin, know that this level of scrutiny will be necessary to deliver good-to-great results; set that expectation for your team and management, so it will be acceptable to them when adjustments are needed for more work to be done to get it right. Just don't blame it on the search engine - get to work, analyze and fix the problem. Only then can you let search loose on your top target audience.
]]>There is one aspect of search output (results) that always makes an impression when I search. Sometimes the display is clear and obvious and other times the first thing that pops into my mind is "what the heck am I looking at" or "why did this stuff appear?" In most cases, no matter how relevant the content may end up being to my query, I usually have to plow through a lot (could be dozens) of content pieces to confirm the validity or usefulness of what is retrieved.
Admittedly, much of my searching is research or helping with a client's intranet implementation, not just looking for a quick answer, a fact or specific document. When I am in the mode for what I call "quick and dirty" search, I can almost always frame the search statement to get the exact result I want very quickly. But when I am trying to learn about a topic new to me, broaden my understanding or collect an exhaustive corpus of material for research, sifting and validating dozens of documents by opening each and then searching within the text for the piece of the content that satisfied the query is both tedious and annoyingly slow.
That is where Documill could enrich my experience considerably for it can be layered on any number of enterprise search engines to present results in the form of precise thumbnails that show where in a document the query criterion/criteria is located. In their own words, "it enhances traditional search engine result list with graphically accurate presentation of the content."
Here are some ideas for its application:
I highly recommend that you ask your search engine solution provider about incorporating Documill into your enterprise search architecture. And, if you have, please share your experiences with me through comments to this post or by reaching out for a conversation.
This week the predictable happened, it was an article by Information Week's Sr. VP focusing on Autonomy's terrific business run in a tough economy. Fair enough - it happens all the time for big spenders.
I just want to remind readers that if you are a small unit in a large organization or a small or medium business, there are dozens of enterprise search solutions that will serve you extremely well, with much lower cost of ownership and startup effort than Autonomy. You do not need the biggest or fastest growing company's products to get good or even excellent solutions. Furthermore, the chances of getting superior customer support and services from a more modest company, which is focused exclusively on search excellence, are much better.
Be sure to check out the offerings at the Gilbane Conference in San Francisco next week. A lot more guidance and good case studies will give you an earful of what else to consider. The search headliners at the conference with Hadley Reynolds moderating are:
E8. Search Survival Guide: Delivering Great Results
Speakers: Randy Woods, Co-founder & Executive VP, non-linear creations, Best Practices for Tuning Enterprise Search and Miles Kehoe, President, New Idea Engineering
E9/I5. The Next Big Thing: Tomorrow's Search Revealed
Speakers: Stephen Arnold, ArnoldIT, What You Need to Know About Google Dataspaces and Jeff Fried, Senior Product Manager, Microsoft
E10/I6. Bringing it All Together: Perils and Pitfalls of Search Federation
Speakers: Helen Mitchell Curtis, Senior Program Director of Enterprise Solutions, MacFadden, Federated Search in a Disparate Environment, Larry Donahue, Chief Operating Officer & Corporate Counsel, Deep Web Technologies, Federated Search: True Enterprise Search and Jeff Fried, Senior Product Manager, Microsoft
E11/I7. The Special Case of Categories - and Where To Find Them
Speakers: Joseph Busch, Founder, Taxonomy Strategies, Taxonomy Validation, and Arje Cahn, CTO, Hippo, Find What You Need in Unstructured Content with the Help of Others (and your CMS): Demo of Wikipedia with Faceted Search
E12/I8. It's Easier with Structure: Leveraging Markup for Better Search
Speakers: Dianne Burley, Industry Specialist, Nstein Technologies, Semantic Search and J. Brooke Aker, CEO, Expert System, A 3-Step Walk Through ECM Using Semantics
E13/I9. Improving SharePoint Search & Navigation with a Taxonomy and Metadata
]]>Here is the April news:
Marketwire. Bell Mobility and Coveo Partner to create Enterprise Search from Bell, an Exclusive Enterprise-Grade Mobile Search Solution, March 31, 2009.
My diminished manual dexterity and old eyes have discouraged me from embracing smart mobile devices but this announcement demonstrates that Coveo does understand the future of search. Coveo is releasing many enhanced functions and new options but the emphasis on search on a handheld is compelling for all road warriors. From what I understand, setup is pretty quick and productivity is immediate. This is a smart move to support Coveo's current customer base and a draw for new ones.
Velten, Carlo. Official Qitera Blog: Qitera Enterprise - Search as a Service, April 2, 2009
There is a 15-day free trial being offered for this new software as a service for aggregating and sharing Internet search results with colleagues within the enterprise. As a consultant in knowledge management, helping people with collaboration and sharing technologies for KM processes, I find this an intriguing option. If any readers have checked it out, please let us know your impressions. http://www.qitera.com/corp/products/overview
Knovel Enhances Engineering Reference Offering, Adding Works from Five New Publishers, 04/06/2009
As a former technical librarian in a Fortune 500 chemical company, I contributed to a massive manually built index to technical information to support our research scientists. We ensured that all property data for any materials we developed, or developed by our competitors was indexed for rapid retrieval (e.g. what is the thermal conductivity of ABC grade of graphite). The overhead for scanning journals, government documents, patents and conference papers to harvest and categorize that information was enormous. Since I learned about Knovel in 2002, I have been a huge fan of the value of the content they codify and make accessible through their proprietary retrieval tools. This content is re-purposed through licenses with publishers who understand the increased value of being able to manipulate tables, charts and graphs as well has being able to compare data from various sources of reference books. Engineers and scientists need to be able to find data expressed in the most relevant form for their purpose. Knovel provides an aggregating and retrieval engine, and researchers can then normalize search results dynamically themselves through simple re-sorting operations. If you want to be kept abreast of the wealth of content that continues to come on-line from Knovel, be sure to visit their site and sign up for announcements. I recommend a subscription to their services to every scientific and engineering library in my client organizations.Endeca Technologies, Inc. ... today announced a formalized partnership to deliver Endeca's Digital Asset Navigator solution on Open Text Digital Media Group's Enterprise Media Management Solution.
April 16, 2009. Endeca's Digital Asset Navigator offers an unprecedented access and discovery experience, combining Endeca's market leading search, Guided Navigation and Content Spotlighting capabilities. It integrates related data from Open Text's enterprise Digital Asset Management solution, as well as databases, file servers, enterprise applications and other source systems... The joint solution also takes advantage of Endeca's advanced security capabilities to ensure that users only have access to data they are approved to see and use...
This is an interesting alliance, to be sure. Digital asset management is an arena ripe for growth and it has not gotten the wide-spread traction I believe it deserves. For publishers and R & D operations the productivity gains can be huge and this combined offering may intensify focus on an under-leveraged technology by ensuing high security and excellent retrieval. Looking at the headline, I would just advise that they pare down the labeling to something pithy and memorable.
X1 is gaining some serious traction and I know it appeals to engineers who like to maintain good order with their piles of data, particularly the flow of large quantities that show up in email and feeds. I spent a few hours four years ago with an engineer, an X1 devotee, who had tagged his email in text files scrupulously and then used X1 to index them every night. He swore by its value and reliability. This looks like they recognize the service they provide to customers who live in email and need to master their desktop domain. Theirs is a niche with a large audience to capture.
ISYS Search Software Announces Release of ISYS:sdk 9. April 28, 2009. Newest Version of Company's Integration Kit Offers Dramatic Performance and Scalability Enhancements, Intelligent Content Analysis and Parametric Search.
ISYS made some significant management changes over the past few months and they are clearly moving along with their marketing efforts as they recognize the value of expanding the re-seller partnership options. Customer comments that come my way continue to be favorable and they have a good story to tell.
Data Search Technology Used by FBI Makes its Way to Enterprises. eWeek New York, NY, April 29, 2009
Add Chiliad to the list of platform search engines, now that they are being highlighted for their value at the FBI. With deep (20+ years) roots in government programs (including early DARPA research and later the SBIR) and a burst of interest and investment by the government after 9/11, Chiliad is taking its venture into more commercial opportunities. We will see how they stack up against the big players in the marketplace. They must have learned something as they worked to "connect the dots" for the FBI.
There have been a lot more stories this past month, but these notices are the ones that kept me engaged in contemplating the enterprise search marketplace that just keeps putting up more options.
]]>One of my own discoveries is that whether I am conducting a meeting, moderating or just attending, seeking out people who might have experiences that could be educational for me is both a way to get into a nice business relationship but it also helps break the ice. It can be awkward going to meetings where we know nobody in advance. Having an agenda that involves meeting people is the ultimate networking model. You might notice that a lot of social networking sites, like LinkedIn, have included a function for asking questions. This has proven popular and I know several people who have leveraged it in beneficial ways.
I have just come from two days at the Infonortics Search Engine meeting and many of you will soon be attending the Enterprise Search Summit in New York, The Gilbane Group conference in San Francisco or SemTech 2009 in San Jose. Here are a few suggestions on how to go shopping for great insight on search tools while establishing a relationship could nurture both you and those you engage for many years to come. Any one of these can start the conversation but think ahead about what you want to ask next once you have your initial answer:
Q: Hi, are you at this conference because you are just beginning to look for a search engine or to find answers about one you are already using? Depending on the answer you will want to find out what they have used, looked at, tested or are researching and what they have learned in the process.
Q: Hi, I see you are from ABC Corporation. How are you involved with search technology there? The answer will give you an idea what line of questioning you might pursue based on the person's presumed experience and knowledge. IT people, developers, content managers or expert searchers will each have a different view of the technologies they have or would like to use. Any role offers a unique perspective for you to draw out and understand for your own institution. Knowing how different professionals view search in other organizations can give you insight into the people you may have to team with in your own organization.
Q: Have you heard any talks at this meeting that have been particularly helpful for you? What have you learned that you didn't know about before? Follow up, and if you sense that some expertise you have might be interesting, sharing it can begin to build a trusted exchange that might prove helpful to you both.
Q: What are a couple of mandatory requirements for a search engine in your organization? Have you been using anything recently that you feel is serving you well or are you having problems? Any time you get a response from another attendee that indicates they are experienced and engaged with specific products, learn everything you can about their: selection process, implementation, deployment and user experiences. Talk to them about what their objectives were and whether and how those were met.
Going to meetings, chatting up attendees, asking questions, and sharing what you know are great ways to build a community of practice outside your internal communities. This brings fresh insights and gives you a valuable networking resource. Don't leave without contact information so you can continue the dialogue. Continue it with online exchanges based on their preference for communication.
Finally, the expense of going to meetings is increasingly hard to justify. But the benefit of finding key vendors and others with a common purpose in one place where you can quickly coalesce around the topic of search (or any other topic) gives you an easy sociability that can then be sustained. To solidify what you have learned and from whom, write a trip report; broadly disseminate it to all those in your enterprise network or team, as well as your boss. This sharing will be appreciated and should underscore the value you know how to accrue from technical meetings. Learning is an essential part of job growth and letting others know that you do it well is important.
• Wireless router that couldn't support wireless (waiting for second replacement)
• IBM ThinkPad power adapter not usable with Lenovo ThinkPad
• Cable service not able to get a signal from street down my 1,000 ft. driveway
• Two cable modem failures and replacements
• ISP spam blocker blocking good stuff but does not retain it as suspect mail for review
• 10 hours of downtime from my web hosting/e-mail service provider
As one who guides and advises companies on enterprise search selection, implementation and deployment, and various aspects of knowledge asset management, it is a little ironic that I have my own challenges finding quality answers and knowledge to support my home office. I have used these tools in my search for answers:
• Phone - vendor customer service
• Chat - vendor website customer service
• Email - vendor customer service, and to some colleagues for advice
• Web searching - vendor site search, Internet general search engines
• Twitter - comments about troubles; search for similar comments by others
So far, phone discussions have been the only pathway to resolutions, and in one case a technician's house call was required. Most of the issues are still open, however emails and automated phone calls solicit feedback about my satisfaction with support services daily.
What does this have to do with search? I am searching to solve very specific problems, not an uncommon reason to search within the enterprise. As an independent consultant, my "enterprise" is my professional network, the support services I pay for and the WWW. When I fail to garner information I need from electronic sources, I reach out directly to experts in my personal network for answers. Even then, I find electronic dialog mechanisms that require typing a back-and-forth Q & A session to be pretty painful. Usually, one of us resorts to the phone or an in-person session to "see" what is really going on.
What have I learned? 1. When a resolution is needed quickly and efficiently, talking to someone who is really an expert is the best path. 2. When I can't find the answer on-line, I need to find an expert. 3. When I can't find an answer or an expert, I flounder and waste huge amounts of time.
Conclusion: Social tools (public platforms, social search, email, and even phone) require substantive work or communication skill by participants to establish a benefit from communication interchanges. Contextual hooks are needed to improve the results of information exchanges. Socializing is critical to expanding our networks of experts in a way that builds relationships in which we can freely reach out and expect a productive dialogue when we have a need to know. This is something to work at and consider when we embrace social technologies. It isn't the technology tool that makes us social, it is the surrounding sharing and communicating (aka socializing) that breeds the trusting and trusted relationships that will improve our search for answers. Social networks and platforms may give us the tools to search for and share content. But it is the socializing that adds rich context to make it more likely that the expert we want and the answers we seek are the most beneficial.
]]>This is a perfect example of where social tools meet a need. A suggestion I also made as a panelist at FastForward 2009 has emerged spontaneously as a direct result of market forces. My observation had been that the FAST user conference was largely attended by IT folks, and the overwhelming number of keynotes and session topics focused on social tools, not especially tied directly to search either. A recommended call to action directed to Microsoft was that they host a platform of social tools to facilitate genuine user community sharing around the FAST product. The people who most need this are search administrators and content managers who presumably have some governance responsibilities for searchable content.
In Tom's suggestion we see the effective use of a social tool to generate interest among members, a large and focused audience who serve as a great test of the viability of his idea. That is neat!
Almost 30 years ago when I ran a software company, we (the company) organized and ran annual user group meetings in tandem with a large professional conference that most of our customers attended. These meetings were very successful, well attended by 40 - 50% of our customers. Over almost 20 years the group spawned a lot of professional and collegial relationships that gave our small user community a sense of collective investment in furthering the improvement and support of the product around which they met. Efforts to turn over total control of the user group to the community were not successful because, in those days, the infrastructure needed for planning, organizing and running meetings across the North American geography did not exist. My company provided that support mechanism out of necessity.
However, three regional user groups began their own programs to share knowledge, and the entire user community collectively published a "cookbook" of source code for reports that many of the users had built for use with the database application and wanted to share with others.
Today the opportunities for building these communities of practice have a vast number of "free" social tools to employ, so that barrier has gone away. More important, the benefits to the user community are limitless. It gets to drive discussion about the product, share hints, workarounds, and tips for successful implementations. The user community gets to decide what is important, what is needed in the knowledge-base of operational information. It can call for product changes, improvements and use social platforms for galvanizing the community around specific issues.
One of the best outcomes we saw with our own user community was around a visitation day at our offices for customers to meet together to "test-drive" an alpha version of a major new release. We purposely stayed out of the meeting for an extended period. Later we learned that when each had developed a "wish list" of changes and tweaks to the release, some rather marginal choices had died a natural death as a result of the "wisdom of the crowd." This was an ideal scenario for us as a development company because we did not have to disappoint any individual users with a unilateral decision to reject their ideas.
Trust me when I recommend to the enterprise search user community, you will empower yourselves in ways you can't imagine when you join forces with other customers to drive the improvements and success of any product you use and value.
]]>Consider this scenario. I was attempting to find the address of the office of a major global platform vendor (one of the largest) that sells an entire suite of enterprise search and content management software products. One can usually find business location information from links on the home page of any corporate Web site or at least from the site representing the division one is visiting. But there was no such link for this corporate site. Then using the "search" box and later the "advanced search" option, trying a dozen variations of the division name, town in which the office is located, and product names I struck out on every query. All paths lead to a page with a single corporate address, or a couple of other remote addresses, and links to web pages that contained no address. Even those pages with addresses had no link to directions. I followed up with queries using Google and these got me back to the same dead-ends. Finally, I found the address through various online non-specific business directories.
This experience lead to a couple of conclusions about why my search failed: 1. The content does not exist; there is no such listing of locations. 2. The search engine is not properly tuned or metadata is not supplied with labels such as "locations," "directions," "business offices," etc. The immediate solution for this case is to ensure that someone with practical business sense and usability competency has ownership of the overall web site experience to make sure that essential company data is available and easy to find. Or, if the company has made a conscious decision not to publish that information, at the least they should have a page stating the alternative for potential visitors as to how they can find their destination or to what office they can direct postal mail.
I had to two reasons for needing this information; one was a visit to an individual who was not available to give me the address in time to reach the office, and the second was a personal follow-up letter after someone from the company had been a speaker at an event I chaired. As things stand, I have been left with personal skepticism about the commitment of this company to build, produce and actually use content management or search products that will be truly responsive to needs of their potential buyers. When you don't or can't showcase your products, I question "why." This is not a technology problem; it is a human factors and human resource allocation problem.
This brings me to some search fundamentals:
• No content - If content that customers or employees expect to find is not included in explicit directives to the search engine for the repositories to be crawled and indexed, it will never be found.
• No metadata - Any content lacking explicit language likely to be used by a searcher will probably not be found if it also lacks sufficient metadata.
• Poor indexing or search rule base - If the content being searched is business documents without many unique contextual "hooks," such as product names, technical terminology or topics of narrow interest, the search engine being used must be "smart" enough to glean the intent of the searcher from the context of query. In my case, I supplied a half a dozen terms to layer the context, tried them in different combinations, with and without quotations around phrases, but nothing worked.
Conclusion, if you really don't want searchers to find what they want to find, it is not hard at all to compromise findability. I will not arrive at my destination and you won't get any first class letters from me.
]]>Travis, Paul. March 2, 2009 Digital Reef Comes Out of Stealth Mode. 03/02/2009. Byteandswitch.com.
Startup offers content management platform to index unstructured data for use in e-discovery, risk mitigation, and storage optimization. Here is the first evidence that entrepreneurs see opportunity for filling a niche vacuum. In the legal market the options have been limited and pretty costly, especially for small firms. This will be an interesting one to watch. http://www.digitalreefinc.com/
Banking, Finance, and Investment Taxonomy Now Available from the the Taxonomy Experts at WAND. 03/02/2009, PR Web (press release), Ferndale,WA,USA
The taxonomy experts at WAND have made this financial taxonomy available now for integration into any enterprise search software. I have been talking with Ross Lehr, CEO at Wand, for over a year about his suite of vertical market taxonomies and how best to leverage them. I am delighted that Wand is now actively engaged with a number of enterprise search and content management firms, enabling them to better support their customers' need for navigation. The Wand taxonomies offer a launching point from which organizations can customize and enhance the vocabulary to match their internal or customer interests. http://www.wandinc.com/main/default.aspx
Miller, Mark. Lucid Imagination » Add our Lucene Ecosystem Search Engine to Firefox. 03/02/2009
I predicted back in January that open source search and search appliances were going to spawn a whole new industry of services providers and expert integrators because there are just not enough search experts to staff in-house experts in all the companies that are adopting these two types of search products. Well, it is happening and these guys at Lucid are some of the smartest search technologists around. Here is an announcement that introduces you to a taste of what they can do. Check it out and check them out at http://www.lucidimagination.com/
To see the full article with commentary about: social search at NASA, QueSearch, MaxxCat, Aardvark on social search, Attivio, ConceptSearching, Google user-group, Simplexo, Endeca, Linguamatics, Coveo, dtSearch and ISYS ...
]]> Microsharing has benefits for NASA. 03/04/2009.It has been about 18 months since I wrote on social search and this report reveals a program that takes the concept to a new level, integrating content management, expertise locators and search in a nifty model. To learn more about NASAsphere, read this report written by Celeste Merryman. Findings from the NASAsphere Pilot. Jet Propulsion Laboratory, California Institute of Technology Knowledge Arciteture (sic) and Technology Task [Force]. 08/20/2008. The success of the pilot project is underscored in this report recommendation: the NASAsphere pilot team recommends that NASAsphere be implemented as an "official" employee social networking and communication tool. This project is not about enterprise search per se, it just reflects how leveraging content and human expertise using social networks requires a "findability" component to have a successful outcome. Conversely, social tools play a huge role in improving findability.
March 16, 2009. QueSearch: Unlocking the Value of Structured Data with Universal Search really caught my eye with their claim to "universal search" (yes, another) for large and mid-size organizations.
This offering with a starting price of $19,500, is available immediately, with software and appliance deployment options. I tried to find out more about their founders and origins on their Web site without luck but did track down a Wikipedia article and a neat YouTube interview with the two founders, Steven Yaskin and Paul Tenberg. It explains how they are leveraging Google tools and open source to deliver solutions.
Stronger, Better, Faster -- MaxxCat's New Search Appliance Aspires to Be Google Search Appliance Killer, by Marketwire. 03/11/2009.
This statement explains why the announcement caught my attention: MaxxCat product developers cite "poor performance and intrinsic limitations of Google Mini and Google Search Appliance" as the impetus to develop the device. The enterprise search appliance, EX-5000, is over seven times faster than Google Search Appliance (GSA) and the small business search appliance, the XB-250, is 16 times faster than Google Mini. There is nothing like challenging the leading search appliance company with a statement like that to throw down the gauntlet. OK I'm watching and will be delighted to read or hear from early users.
Just one more take on "social search" as we learn about Aardvark: Answering the Tough Questions, David Hornik on VentureBlog. 03/12/2009
This week the Aardvark team is launching the fruits of that labor at South By Southwest (SXSW). They have built a "social search engine" that lives inside your IM and email. It
allows you to ask questions of Aardvark, which then goes about determining who among your friends and friends of friends is most qualified to answer those questions. As the Aardvark team point out in their blog, Social Search is particularly well suited to answer subjective questions where "context" is important. I am not going to quibble now but I think I would have but this under my category of "semantic search" and natural language processing. Until we see it in action, who knows?
A new position at Attivio was announced on March 16th, Attivio Promotes John O'Neil to Chief Scientist, which tells me that they are still expanding at the end of their first official year in business.
Getting to the point, 03/18/2009, KMWorld. http://www.kmworld.com/Articles/ReadArticle.aspx?ArticleID=53070
Several announcements about Concept Searching's release v. 4 of its flagship product, conceptClassifier for SharePoint highlight the fact that Microsoft's acquisition of FAST has not slowed the number of enterprise search solution companies that continue to partner with or offer independent solutions for SharePoint. In this case the company offers its own standalone concept search solution applications for other content domains but is continuing to bank on lots of business from the SharePoint user community. This relationship is reflected in these statements: The company says features include a new installer that enables installation in a SharePoint environment in less than 20 minutes, requires no programmatic support and all functionality can be turned on or off using standard Microsoft SharePoint controls. Full integration with Microsoft Content Types and greater support for multiple taxonomies are also included in this release. Once the FAST search server becomes a staple for Microsoft SharePoint shops, there will undoubtedly be fallout for some of these partners.
Being invited to the Google Enterprise Search Summit in Cambridge, MA on March 19, 2009 was an opportunity for me to visit Google's local offices and meet a bunch of customers.
They were a pretty enthusiastic crowd and are enjoying a lot of attention as this division of Google works to join the ranks of other enterprise application software companies. I suspect that it is a whole new venture for them to be entertaining customers in their offices in a "user-group like" forum but the Google speakers were energetic and clearly love the entrepreneurial aspects of being a newish run-away success within a run-away successful company. New customer announcements continue to flow from Google with SITA (The State Information Technology Agency in South Africa) acquiring GSA to drive an enterprise-wide research project. The solution will also be deployed and implemented by JSE-listed IT solutions and services company Faritec, and RR Donnelly. Several EMC users were represented at the meeting, which made me ask why they aren't using the search tools being rolled out by the Documentum division...well, don't ask.
Evans, Steve. Simplexo boosts public sector search options. Computer Business Review - UK. 03/18/2009.
This is interesting as an alternative to the Lucene/solr scene, UK-based open source enterprise search vendor Simplexo has launched a new search platform aimed at the public sector, which aims to enable central and local government departments to simultaneously search multiple disparate data sources across the organisation on demand. I have wondered when we would see some other open source offerings.
And all of the preceding is about just the startups (plus EMC at Google) and lesser known company activity. This was not a slow month. I don't want all my contacts in the "established" search market to think that I am not paying attention because I am. I've exchanged communications with or been briefed by these known companies with news about new releases, advancing market share, or new executive teams. In no particular order these were the highlights of the month:
Endeca announced three new platforms on Mar 23, 2009: Endeca Announces the Endeca Publishing Suite, Giving Editors Unprecedented Control Over the Online Experience; Endeca Announces the Endeca Commerce Suite, Giving Retailers Continuous Targeted Merchandizing; and Endeca Unveils McKinley Release of the Information Access Platform, Allowing for Faster and Easier Deployment of Search Applications
Linguamatics Agile Text Mining Platform to Be Used by Novo Nordisk. 03/26/2009
I had a fine briefing by Coveo's CEO Laurent Simoneau and Michel Besmer new VP of Global Marketing and see them making great strides capturing market share across numerous verticals where rapid deployment and implementation are a big selling point. They also just announced: Bell Mobility and Coveo Partner to Create Enterprise Search from Bell, an Exclusive Enterprise-Grade Mobile Search Solution.
A new Version 7.6 of a mainstay, plug-and-play search solution for SMBs since 1991, dtSearch, was just released. 3/24/2009
And finally, ISYS is having a great growth path with a new technology release, ISYS File Readers, new executives and a new project ... completed in conjunction with ArnoldIT.com. Steve Arnold, industry expert and author of the Beyond Search blog, compiled more than a decade of Google patent documents. To offer a more powerful method for analyzing and mining this content, we produced the Google Patent Search Demonstration Site, powered by our ISYS: web application.
Weatherwise, March, 2009 is out like a lamb but hot, hot, hot when it comes to search.
]]>Now that we have had a little experience with this type of service, I have decided that it would serve technology "buyers," just as well. The service might prove even more effective for some companies than a lengthy contract for consulting services. Companies devote long lead times to thinking about, budgeting, selecting and procuring software solutions. Most of them don't want a consultant waiting in the wings for the next evolution in a project. What they would like is an expert they can turn to at each project gate where a pivotal decision needs to be made, or for a little guidance on an approach or what the next step should be. As an analyst, talking to technology customers is valuable because I can hear customer thoughts and ideas about products and companies and then present these as anonymous feedback when appropriate.
The Gilbane Group has a long reputation for product independence. Our sponsored research, white papers and webinars focus on timely topical themes, not product briefings or marketing buzz for a particular client. We help vendors get out educational messages about how they view markets, customer needs, tool implementation and strategies for leveraging technologies. We give voice to the values they espouse as companies. When we work for a vendor, we also share advice about how they are perceived in the marketplace, and how to improve their brand because we believe that good and healthy companies make for a vibrant marketplace.
I've been told, "off-the-record," that, while the Gilbane model is laudable, no vendor believes in true analyst or consultant independence. While I am sorry to hear that this might be the prevailing view, it is like saying that no bank can be trusted because of the current financial crisis. Are you hiding your money under the mattress, yet?
Not only are we a trustworthy resource - we have a lot of good people with terrific expertise. Check out the cast of characters at: Gilbane Contacts and consider how great it would be to have them all a phone call or email communication away. Just a thought.
]]>• How will Autonomy be viewed by IT folks, whom I assume are the principal readers of Information Week?
• Is this a shift away from an emphasis on search as "search" by Autonomy?
• Is Autonomy just expanding its range to broader business interests to gain better enterprise penetration?
• Will their deep technical competence in search be as rich in the areas of governance and compliance?
To try to get a handle on all of this, since the second ad had no URL, I went to the electronic version online at Information Week archives but discovered that the ads don't appear in the PDF. No problem; I went to the advertisers' index and clicked on the Autonomy link, thinking that the link would take me to the ControlPoint pages on their Web site. It only took me to the main page for Autonomy where there was nothing referring to ControlPoint, compliance, regulation or governance (all words prominent in the magazine print ads). I tried the drop-down for Products; nothing there either. At least Autonomy uses IDOL as its search engine on its own Web site, so I tried it. Yea! ControlPoint appeared in the results; the first entry got me to a page describing it.
But what else did I learn by following the breadcrumbs? A step back to the "products" level brought me to an Autonomy Electronics Records Management description and I began to notice the logo in the upper right said "Autonomy Meridio." Lots of clicks later, I discovered that Meridio was acquired by Autonomy in 2007, which I probably would have known if I had paid more attention to "non-search" stuff. ControlPoint belongs in that family of products. When I clicked on this sidebar link, Autonomy ControlPoint: Information Governance for SharePoint and this one, Meridio eDRM for Microsoft Office, more questions came to mind:
• Is Autonomy, the search company with its Meridio and Interwoven acquisitions, having a serious run at Microsoft by entering their traditional markets?
• If an office tools software company like Microsoft slides into the search market by acquiring FAST and then leverages its great success with SharePoint by making FAST its default search offering, why shouldn't Autonomy turn the tables?
• By appealing to IT professionals will Autonomy be able to gain mind share that pits them directly against Microsoft with language like "Named Email and Compliance Vendor of the Year by Financial-i" and "Is SharePoint enough?"
Yes, we are going well Beyond Search, aren't we?
Look at three popular concepts about search:
If we accept these propositions, doesn't it say something about the "leaders" in the search industry that we believe and accept so little from search?
Why do most organizations not try to solve at least one of these problems by seeking solutions that will save hundreds of thousands of dollars in wasted labor, litigation costs, R&D expense, or lost customers due to poor service? Why do companies seek to procure search applications from companies that have been around for a decade or more, licensing evolutionary products, not revolutionary ones? Why would a company ignore innovative new products in favor of products that have given "search" a bad reputation? Why do organizations make hundred thousand dollar, or more, procurements without expending a few hundred dollars on documented product comparisons, and instead rely on a few widely published charts with less than a page or two on each product?
Most important, why are organizations not seeking search applications that will give them an edge by uncovering a nugget that will get a product to market faster, help marketing groups position a product better against the competition, or give support services representatives superior tools for getting information back to customers instantly with a proven solution to a query? Where is the will to apply search technology more astutely than your competitors in every area of your business? Why is search not expected to perform flawlessly and be as ubiquitous as any other software tool in your workflow? It does not have to be a poor performing stepchild but it does require its own experts to be well executed. Come to think of it, I have never seen a help wanted posting requiring expertise in search technology implementation. Hmmm...
There are well over a hundred viable search applications and hundreds of other applications that have search embedded for point solutions. You may need to acquire, implement and maintain a number of products across the enterprise to realize all the benefits search can bring but these products can work together, just as other components of a well-run enterprise do. At a time when organizations are cutting employees, appropriate search solutions may just offset the loss of expertise by uncovering at least some of the lost assets left behind.
Some of the history of search federation is described in this rather sketchy article at Wikipedia. However, I want clarify the original call for such a standard. It comes from the days when public access to search technologies was available primarily through library on-line catalogs in pubic and academic institutional libraries. A demand for the ability to search not only one's local library system and network (e.g. a university often standardized on one library system to include all the holdings of a number of its own libraries), but also the holdings of other universities or major public libraries. The problem was that the data structures and protocols from one library system product to the next varied in way that made it difficult for the search engine of the first system to penetrate the database of records in another system. Records might have been meta-tagged similarly, but the way the metadata were indexed and accessible to retrieval algorithms was not possible with a translating layer between systems. Thus, the Z39.50 standard was established, originally to let one library system's user search from that library system into the contents of other libraries with different systems.
Ideally, results were presented to the searcher in a uniform citation format, organized to help the user easily recognize duplicated records, each marked with location and availability. Usually there was a very esoteric results presentation that could only be readily interpreted by librarians and research scholars.
Now we live in a digitized content environment in which the dissimilarities across content management systems, content repositories, publishers' databases, and library catalogs have increased a hundred fold. The need for federating or translation layers to bring order to this metadata or metadata-less chaos has only become stronger. The ANSI standard is largely ignored by content platform vendors, thus leaving the federating solution to non-embedded search products. A buyer of search must do deep testing to determine if the enterprise search engine you have acquired actually stands up well under a load of retrieving across numerous disparate repositories. And you need a very astute and experienced searcher with expert familiarity of content in all the repositories to make an evaluation as to suitability for the circumstance in which the engine will be used.
So, let's just recap what you need to know before you select and license a product claiming to support what you expect from search federation:
o Integration of the results from a number of targeted searchable domains, each with its own search engine
o Disambiguation of content results when similar but non-identical pieces of content might be included
o Normalization of search results so that content from different domains is presented similarly
o Consolidation of the search operation (standardizing a query to each of the target search engines) and standardizing the results so they appear to be coming from a single search operation
Why is this relevant? In short, because it is expected by users that when they search, all the results they are looking at represent all the content from all the repositories they believed they were searching in a format that makes sense to them. It is a very tall order for any search system to do this but when enterprise information managers are trying to meet a business manager's or executive's lofty expectations, anything less is viewed as the failure of enterprise search. Or else, they better set expectations lower.
]]>Before making some observations of my own, I will introduce you to a few of the folks who have had and will continue to have a continuing presence in the content and search arena, particularly as it relates to social tools and knowledge management, two tightly connected areas of interest.
Each of us was interviewed for a kind of video blog session during the meeting. Although you can't view the panel from the final keynote session, I can share these links that will give you an idea of what my cohorts were thinking about the meeting and the state of FastForward in 2009. They are:
Jon Husband, social computing thought leader and architect. He has coined the term "wirearchy," which aptly describes a flow of connectedness over the wires (and wireless) air waves. I really liked his observations about how social technologies encourage self-organizing around issues and make group action so much easier. His interview is a good listen and his blog is fun, too.
Jevon MacDonald, founder of Firestoker and FASTforward blog contributor, had some helpful comments in his interview about the usefulness of social media in aiding companies to be more responsive to their customers.
Euan Semple, independent advisor on social computing, elevated the discussion in favor of social tools improving the flow of knowledge, which is really the point of all this content and search related technology, as far as I am concerned. You'll enjoy the interview with Euan in which he also comments on the ratio of men to women and the IT-centric audience at the meeting, something I observed, as well.
I was also interviewed by Josh-Michéle Ross and my thoughts dovetailed with the others in keeping with the social them of "engage your user," the conference tag line. My mantra throughout the conference and after is that there was just not enough emphasis on how teams work together to build highly functional and easy-flowing search experiences for users. The process of creating a social platform in which search is present in subtle ways that assist connectedness among experts and their content requires human design; this is an art that can't be left to "out-of-the-box" installed technologies. It is a task for those with an aptitude for what users really want, need and will use without being force-fed or artificially manipulated. Here are my comments in the interview.
Other interviews of interest can be found at the FastForward Bloggers page where a lot of thought leaders including Rob Paterson, Bill Ives, Clay Shirky, Charlene Li and Jim McGee among many others put forth some thoughtful comments about the state of technology.
Our panel moderator was, Perry Solomon, VP Business Development and General Manager, Worldwide Media Solutions - FAST. While on the big stage we did not get to speak on all the ideas he asked about in our preparatory session, I can bring them to light in the following. Solomon asked these questions followed by my thoughts after a few days to digest the meeting:
Q: How was the meeting balance in terms of search technology versus use?
LWM: The use cases were compelling and well presented. They were highly evocative of the best applications we can achieve with technology using all the social tools and content management options now available. This is appropriate in keynote/big theatre presentations but what I did not find in the few breakout sessions was more about the "nuts and bolts" of the human design and understanding needed to integrate components.
Among the attendees that I met during meals (system integrator partners from small firms and IT people who were struggling to build applications their internal customers wanted), there was a sense that not enough substantive information was being shared. They had hoped for more "how to" and concrete case studies that described the process of getting from purchasing licenses to deploying solutions. When I suggested to some of these Microsoft customers that it might be helpful to have more of their content managers and search administrators in the audience, they all agreed. None carried an attitude that they were going to design and implement these highly sophisticated content/search solutions with just members of the IT department. Business users were also notably absent from the meeting.
Q: What was the impact of the announcement about product news, FastSearch for Internet Business and FastSearch for SharePoint?
LWM: My own reaction was that it was a logical way to begin to roll out the FAST product with existing and evolving Microsoft products. It was not surprising, revolutionary or exciting. MS is clearly committed to making something of its huge investment in FAST; to align it with the rapidly evolving and highly popular SharePoint is smart business. The sentiment of others I spoke with was pretty much the same, sprinkled with a fair amount of skepticism about schedules for delivery and how well the products will be supported with services and documentation. Cost of ownership is always a big worry; what it will take to get the sizzle and super search results from this technology without a huge amount of human investment and skill on the part of customers or third-party integrators stimulates a deep "wait-and-see" attitude among most.
Q: What was missing or not addressed in the sessions?
LWM: The lack of presentations and involvement of non-IT people. While MS is highly responsive to the IT person's desire for standardizing on a full-function platform and set of tools from a single supplier, this is not the reality in the marketplace. Content is created, manipulated and re-purposed with hundreds of applications that are used by business owners and content managers who bring a deep understanding of what needs to be applied to get the "social" workflow operational and productive in any given culture. My own bias is that the subtleties of organizational culture are often lost on many in IT but are more understood by those deeply immersed in engagement with both experts and their content. A "user-group" meeting must include these "others" and have sessions that support their professional interests so they come away learning substantive stuff from those others in similar situations.
Although "search" was the nominal reason for the meeting, there was no discussion about what it takes to get to the ultimate "user-engagement." Search remains, "smoke and mirrors." Search behind the firewall was still pretty thin as a concept and the emphasis was on e-commerce and monetization. There was a lot of talk about business & customer experiences engaging with search but not much substance as to how to actually create rich search experiences.
Q: What are we going to be talking about a year from now?
LWM: I hope the engagement will provide less visionary "hype," which is not real high-value for the audience in heavy doses. If the meeting becomes more about getting customers to a successful outcome through the engagement of teams with IT, developers, content and business owners coming to a problem using a thoughtful design approach, attendees will leave with a higher commitment to embrace the technology.
Finally, I believe that, as FastSearch solutions are implemented and tested, customers will come to these meetings with higher expectations for helpful case studies that talk about "how the sausage is made," the role of connectors and the actual tuning for higher relevancy. Much reference to search federation will give way to what federation really is and its many tiers of sophistication. Presentation of search results in ways that are compelling and trustworthy for users will need to be explained in more substantive sessions. I hope that we will be talking about social team interaction for implementing compelling search technology experiences for users.
]]>