What buyers need to do first is to stop limiting their procurement selection choices based primarily on license prices. When enterprises begin their selection by considering prices first, many options are eliminated that may be functionally more appropriate and for which the total cost of ownership may be even less.
Product pricing correlates more to the market domain in which a vendor sells or aims to sell than to actual product value per installed user. Therefore, companies in the small to mid-range are particularly vulnerable to unreasonable licensing. I have written about this before but it bears repeating, the strength of the underlying technology has little to do with the price but can influence the total-cost-of-ownership (TCO) dramatically.
Buyers often believe high license price relates to top product value; in general you still need to add another 60-80% for services and support costs to get that value out. But let's look at the business reality and corporate context for sellers of high-priced enterprise search.
Net sales of any company that is large is a significant determinant of its reputation and potential staying power in its industry. However, when actual sales for a search product line are a tiny fraction of total company revenue, potential buyers of enterprise search need to know that and factor it into their decision-making for these reasons:
• The largest software companies are heavily vested in subscribing to analyst services that write about the industry. They are diligent in reporting their sales figures to those companies and publications that do annual surveys on various industry segments. The reporting is usually careful to note when revenues for a particular sector ( like search) are not broken out, but this often escapes the notice of buyers who only see that company X has enormous revenues compared to others. This leaves the impression that they are also a standout in the search sector.
• The fact that a company offers many software products, of which search is only one, has often resulted from acquisition of a lot of products. Search may only be in the mix because it complements other products. The company may or may not have actually retained the technology gurus who originally designed, developed and supported the software. A lot of software quickly becomes stale once acquired by a third-party.
• When a very large company offers many products, it focuses sales, account management, support and development on those with the largest revenue stream or growth potential. Marketing for marginal products may be sustained for a longer period to bring in "easy" business but unfortunately, for too long, search has been treated as a loss leader to attract revenues for other product lines. Where "search" fits into a mix of products, how well it will be serviced and supported over time may be difficult to discern.
• The final situation that happens for very large software companies is that competition is an ever-present cause for shifting agendas. The largest software firms will often abandon technologies whose architecture, unique functions and even their customers do not fit their changing market interests. They will abandon products for which they have paid huge sums once the initial value of the procurement has been realized, when a product's technology has been captured for embedding in other product suites, or if the product is no longer viewed as strategic.
In the next blog posting we'll take a look at some other reasons that vendors make and then abandon their acquisitions. But in the meantime, here is a recommendation to buying decision-makers:
When you see a very long list of customer logos on the web sites of major software vendors there is important context that is not provided. Large corporations can and do buy competing products all the time. Some products get into enterprise-wide use and adoption for the long term while others are used briefly or in smaller applications. You can't know whether a product is even in use in the company whose logo is displayed. Because it is almost impossible for an outsider to find the actual buyer/user of a product in a large enterprise; the posted logos tell you little. Inside an enterprise one may discover endless tales of when, why and how competing products were acquired, many as part of package deals or through a subsidiary acquisition. What is also true is that stories of successful implementations or brand loyalty do not abound.
For you who are new to enterprise search, take control of your own destiny by educating yourself using a lower priced product with a good reputation for a niche application. Invest your budget instead in human resources (internal or 3rd party) to craft the solution you really need. Start with a vision of appropriate scale, tackling a small domain of high value content that is currently hard to find in your organization.
Use the experience of implementing and leveraging this search product and engaging with the vendor to bring a deeper understanding of the technology and applications of search. Working with a vendor dedicated exclusively to search will have another cost benefit because of the focused attention you are more likely to receive. Delving deeply into planning and implementation for a targeted result will have a cost that brings multiple benefits moving forward to larger and more complex implementations - even if you move on to another product.
]]>I don't make forecasts or try to guess whether strategic changes will succeed or fail but a couple of years ago, I blogged on IBM's introduction of Yahoo OmniFind, a free offering and then followed up with these comments just a few months ago. IBM makes their competitors change, try to outsmart, outguess, or copy, just as Microsoft or Google changes cause ripples in the industry.
Meanwhile, OpenText, another large software company with search offerings, is not going to offer search outside of its other product suites. [More is likely to come out after the scheduled analyst meetings today but I'm not there and can't brief you on deeper intent.] We have recently seen an announcement about FAST being delivered with new SharePoint offerings, the first major release of FAST announced since Microsoft acquired them almost two years ago. While FAST is still available as a standalone product from MS, it and other search engines may be steadily moving into being embedded in suites by their acquirers.
Certainly IBM has a lot of search components that they have acquired, so continuing to bind with other content offerings is a probable strategy. Oracle and Autonomy may soon come up with similar suite offerings embedding search once again. Oracle SES (Secure Enterprise Search) does not appear to have a lot of traction and it's possible that supporting pure search offerings may be a burden for Autonomy with its stable of many acquired content products.
All of this leads me to think that, since enterprise search has gotten such a bad reputation as a failed technology, the big software houses are going to bury it in point solutions. Personally, I believe that enterprise search is a failed strategy and SMBs can still find search engines that will serve the majority of their enterprise needs for several years to come. The same holds true for divisions or groups within large corporations.
Guidance: select and adopt one or more search solutions that fit your budget for small scale needs, point solutions and enterprise content that everyone in the organization needs to access on a regular basis. Learn how these products work, what they can and cannot deliver, making incremental adjustments as needs change and evolve. Do not install and think you are done because you will never be done. Cultivate a few search experts to stick with the evolving landscape and give them the means to keep up with changes in the search landscape. It is going to keep morphing for a long time to come.
]]>As part of a panel on semantic search, moderated by Hadley Reynolds of IDC, with Jeff Fried of Microsoft and Chris Lamb of the OpenCalais Initiative at Thomson Reuters, I wanted to give a high level view of semantic technologies currently in the marketplace. I contacted about a dozen vendors and selected six to highlight for the variety of semantic search offerings and business models.
One case study involves three vendors, each with a piece of the ultimate, customer-facing, product. My research took me to one company that I had reviewed a couple of years ago, and they sent me to their "customer" and to the customer's customer. It took me a couple of conversations and emails to sort out the connections; in the end the relationships made perfect sense.
On one hand we have conglomerate software companies offering "solutions" to every imaginable enterprise business need. On the other, we see very unique, specialized point solutions to universal business problems with multiple dimensions and twists. Teaming by vendors, each with a solution to one dimension of a need, create compound product offerings that are adding up to a very large semantic search marketplace.
Consider an example of data gathering by a professional services firm. Let's assume that my company has tens of thousands of documents collected in the course of research for many clients over many years. Researchers may move on to greater responsibility or other firms, leaving content unorganized except around confidential work for individual clients. We now want to exploit this corpus of content to create new products or services for various vertical markets. To understand what we have, we need to mine the content for themes and concepts.
The product of the mining exercise may have multiple uses: help us create a taxonomy of controlled terms, preparing a navigation scheme for a content portal, providing a feed to some business or text analytics tools that will help us create visual objects reflecting various configurations of content. A text mining vendor may be great at the mining aspect while other firms have better tools for analyzing, organizing and re-shaping the output.
Doing business with two or three vendors, experts in their own niches, may help us reach a conclusion about what to do with our information-rich pile of documents much faster. A multi-faceted approach can be a good way to bring a product or service to market more quickly than if we struggle with generic products from just one company.
When partners each have something of value to contribute, together they offer the benefits of the best of all options. This results in a new problem for businesses looking for the best in each area, namely, vendor relationship management. But it also saves organizations from dealing with huge firms offering many acquired products that have to be managed through a single point of contact, a generalist in everything and a specialist in nothing. Either way, you have to manage the players and how the components are going to work for you.
I really like what I see, semantic technology companies partnering with each other to give good-to-great solutions for all kinds of innovative applications. By the way, at the conference I am doing a quick snapshot on each: Cogito, Connotate (with Cormine and WorldTech), Lexalytics, Linguamatics, Sinequa and TEMIS.
Frequently, enterprise stakeholders will reference a commercial web site they like or even search tools within social sites. These are a great starting point for a designer to explore. It makes a lot of sense to visit scores of sites that are publicly accessible or sites where you have an account and navigate around to see how they handle various design elements.
To start, look at:
The point of this list of questions is that you can use it to build a set of criteria for designing what your enterprise will use and adopt, enthusiastically. But this is only a beginning. By actually visiting many sites outside your enterprise, you will find features that you never thought to include or aggravations that you will surely want to avoid. From these experiences on external sites, you can build up a good list of what is important to include or banish from your design.
When you find sites that you think are exemplary, ask key stakeholders to visit them and give you their feedback, preferences and dislikes. Particularly, you want to note what confuses them or enthusiastic comments about what excites them.
This post originated because several press notices in the past month brought to my attention Web applications that have sophisticated and very specialized search applications. I think they can provide terrific ideas for the enterprise search design team and also be used to demonstrate to your internal users just what is possible.
Check out these applications and articles: on KNovel, particularly this KNovel page; ThomasNet; EBSCOHost mentioned in this article about the "deep Web.". All these applications reveal superior search capabilities, have long track records, and are already used by enterprises every day. Because they are already successful in the enterprise, some by subscription, they are worth a second look as examples of how to approach your enterprise's search interface design.
]]>First it explains why none of the articles I write for this blog about enterprise search appear in Google alerts for “enterprise search.” Besides being a personal annoyance, easily resolved if I invested in some Internet search optimization, it may explain why meta tagging is a hard sell behind the firewall.
I do know something about getting relevant content to show up in enterprise search systems and it does depend on a layer of what I call “value-added metadata” by someone who knows the subject matter in target content and the audience. Working with the language of the enterprise audience that relies on finding critical content to do their jobs, a meta tagger will bring out topical language known to be the lingua franca of the dominant searchers as well as the language that will be used by novice employee searchers. The key here is to recognize that in any specific piece of content its “aboutness” may never be explicitly spelled out in terminology by the author.
In one example, let’s consider some fundamental HR information about “holiday pay” or “compensation for holidays” or “compensation for time-off.” The strings in quotes were used throughout documents on the intranet of one organization where I consulted. When some complained about not being able to find this information using the company search system, my review of search logs showed a very large number of searches for “vacation pay” and almost no searches for “compensation” or “holidays” or “time off.” Thus, there was no way that using the search engine employees would stumble upon the useful information they are seeking – unless, meta tags make “vacation pay” a retrievable index pointer to these documents. The tagger would have analyzed the search logs, seen the high number of searches for that phrase and realized that it was needed as a meta tag.
Now, back to Google’s position on ignoring meta tags because writers and marketing managers were “gaming the system.” They were adding tags they thought would be popular to get people to look at content not related but for which they were seeking a huge audience.
I have heard the concern that people within enterprises might also hijack the usefulness of content they were posting in blogs or wikis to get more “eyeballs” in the organization. This is a foolish concern, in my opinion. First I have never seen evidence that this happens and don’t believe that any productive enterprise has people engaging in this obvious foolishness.
More importantly, professional growth and success depends on the perceptions of others, their belief in you and your work, and the value of your ideas. If an employee is so foolish as to misdirect fellow employees to useless or irrelevant content, he is not likely to gain or keep the respect of his peers and superiors. In the long run persistent, misleading or mischievous meta tagging will have just the opposite effect, creating a pathway to the door.
Conversely, the super meta tagger with astute insights into what people are looking for and how they are most likely to look for it, will be the valued expert we all need to care for and spoon feed us our daily content. Trusted resources rise to the top when they are appropriately tagged and become bedrock content when revealed through enterprise search on well-managed intranets.
]]>I wrote about the IBM launch of the OmniFind suite of search products a couple of years ago with positive comments. The reality ended up being quite different as I noted later. Among the negatives were three that stand out in my mind. First, free (as in the IBM OmniFind Yahoo no-charge edition) is rarely attractive to serious enterprises looking for a well-supported product. Second, the substantial computing overhead for the free product was significant enough that some SMBs I know of were turned off; the costs associated with the hardware and support it would require offset "free." Third, my understanding that the search architecture for the free product would provide seamless upgrades to IBM's other OmniFind products was wrong. Each subsequent product adoption would require the same "rip and replace" that Steve Arnold describes in his report, Beyond Search. It is hard to believe that IBM got much traction out of this offering from the enterprise search market at large. Does anyone know if there was really any head-to-head competition between IBM and other search vendors over this product?
On the other hand, does the Microsoft Express Search offering appeal to enterprises other than the traditional Microsoft shop? If Microsoft Express Search went away, it would probably be replaced by some other Microsoft search variation with inconvenience to the customer who needs to rip and replace and left on his own to grumble and gripe. What else is new? The same thing would happen with IBM Yahoo OmniFind users and they would adapt.
I've noticed that free and cheap products may become heavily entrenched in the marketplace but not among organizations likely to upgrade any time soon. Once enterprises get immersed in a complex implementation (and search done well does require that) they won't budge for a long, long time, even if the solution is less than optimal. By the time they are compelled to upgrade they are usually so wedded to their vendor that they will accept any reasonable offer to upgrade that the vendor offers. Seeking competitive options is really difficult for most enterprises to pursue without an overwhelmingly compelling reason.
This additional news item indicates that Microsoft is still trying to get their search strategy straightened out with another new acquisition, Applied Discovery Selects Microsoft FAST for Advanced E-Discovery Document Search. E-discovery is a hot market in legal, life sciences and financial verticals but firms like ISYS, Recommind, Temis, and ZyLab are already doing well in that arena. It will take a lot of effort to displace those leaders, even if Microsoft is the contender. Enterprises are looking for point solutions to business problems, not just large vendors with a boatload of poorly differentiated products. There is plenty of opportunity for specialized vendors without going toe-to-toe with the big folks.
]]>Like other topics in this blog this one requires a readjustment in thinking by technology users. One of the great things about digitizing text was the promise of ways in which it could be parsed, sorted and analyzed. With heavy adoption of databases that specialized in textual, as well as numeric and date data fields for business applications in the 1960s and 70s, it became much easier for non-technical workers to look at all kinds of data in new ways. Early database applications leveraged their data stores using command languages; the better ones featured statistical analysis and publication quality report builders. Three that I was familiar with were DRS from ADM, Inc., BASIS from Battelle Columbus Labs and INQUIRE from IBM.
Tools that accompanied database back-ends had the ability to extract, slice and dice the database content, including very large text fields to report: word counts, phrase counts (breaking on any delimiter), transaction counts, relationships among data elements across associated record types, ability to create relationships on the fly, report expert activity and working documents, and describe distribution of resources. These are just a few examples of how new content assets could be created for export in minutes. In particular, a sort command with DRS had histogram controls that were invaluable to my clients managing corporate document and records collections, news clippings files, photographs, patents, etc. They could evaluate their collections by topic, date ranges, distribution, source, and so on, at any time.
So, there existed years ago the ability to connect data structures and use a command language to formulate new data models that informed and elucidated how information was being used in the organization, or to illustrate where there were holes in topics related to business initiatives. What were the barriers to wide-spread adoption? Upon reflection, I came to realize that extracting meaningful content from database in new and innovative formats requires a level of abstract thinking for which most employees are not well-trained. Putting descriptive data into a database via a screen form, then performing a transaction on the object of that data on another form, and then adding more data about another similar but different object are isolated in the database user's experience and memory. The typical user is not trained to think about how the pieces of data might be connected in the database and therefore is not likely to form new ideas of how it can all be extracted in a report with new information about the content. There is a level of abstraction that eludes most workers whose jobs consist of a lot of compartmentalized tasks.
It was exciting to encounter prospects that really grasped the power of these tools and were excited to push the limits of the command language and reporting applications, but they were scarce. It turned out that our greatest use came in applying text analytics to the extraction of valuable information from our customer support database. A rigorously disciplined staff populated it after every support call with not only demographic information about the nature of the call, linked to a customer record that had been created back at the first contact during the sales process (with appropriate updates along the way in the procurement process) but also a textual description of the entire transaction. Over time this database was linked to a "wish list" database and another "fixes" database and the entire networked structure provided extremely valuable reports that guided both development work and documentation production. We also issued weekly summary reports to the entire staff so everyone was kept informed about product conditions and customer relationships. The reporting tools provided transparency to all staff about company activity and enabled an early version of "social search collaboration."
Current text analytics products have significantly more algorithmic horsepower than the old command languages. But making the most of their potential and transforming them into utilities that any knowledge worker can leverage will remain a challenge for vendors in the face of poor abstract reasoning among much of the work force. The tools have improved but maybe not in all the ways they need to for widespread adoption. Workers should not have to be dependent on IT folks to create that unique analysis report that reveals a pattern or uncovers product flaws described by multiple customers. We expect workers to multitask, have many aptitudes and skills, and be self-servicing in so many aspects of their work, but for them to flourish the tools fall short too often. I'm putting in a big plug for text analytics for the masses, soon, so that enterprise search begins to deliver more than personalized lists of results for one person at a time. Give more reporting power to the user.
]]>It did start me thinking about the realities of how individuals dispose of, store, categorize and manage their work related documents. My own process goes like this. My work content falls into four broad categories: products and vendors, client organizations and business contacts, topics of interest, and local infrastructure related materials. When material is not purposed for a particular project or client but may be useful for a future activity, it gets a metadata record in the database and is hyperlinked to the full-text. The same goes for useful content out on the Web.
When it comes to email, I discipline myself to dispose of all email into its appropriate folder as soon as I can. Sometimes this involves two emails, the original and my response. When the format is important I save it in the *.mht format (it used to be *.htm until I switched to Office 2007 and realized that doing so created a folder for every file saved); otherwise, I save content in *.txt format. I rename every email to include a meaningful description including topic, sender and date so that I can identify the appropriate email when viewing a folder. If there is an attachment it also gets an appropriate title and date, is stored in its native format and the associated email has "cover" in the file name; this helps associate the email and attachment. The only email that is saved in Outlook in personal folders is current activity where lots of back and forth is likely to occur until a project is concluded. Then it gets disposed of by deleting, or with the project file folders as described above. This is personal governance that takes work. Sometimes I hit a wall and fall behind on the filtering and disposing but I keep at it because it pays off in the long term.
So, why not relax and leave it all in Outlook, then let a search engine do the retrieval? Experience had revealed that most emails are labeled so poorly by senders and the content is so cryptic that to expect a search engine to retrieve it in a particular context or with the correct relevance would be impossible. I know this from the experience of having to preview dozens of emails stored in folders for projects that are active. I have decided to give myself the peace of mind that when the crunch is on, and I really need to go to that vendor file and retrieve what they sent me in March of last year, I can get it quickly in a way that no search engine could ever do. Do you realize how much correspondence you receive from business contacts using their "gmail" account with no contact information revealing their organization in the body and signed with a nickname like "Bob" and messages "like we're releasing the new version in four weeks" or that just have a link to an important article on the web with "thought this would interest you?"
I did not have a chance to learn if my new business acquaintance had any sense of the amount of competition he has out there for email search, or what his differentiator is that makes a compelling case for a search product that only searches through email, or what happens to his product when Microsoft finally gets FAST search bundled to work with all Office products. OR, perhaps the rest of the world is storing all content in Outlook. Is this true? If so, he may have a winner.
This topic comes to mind because of two unrelated pieces of content I read in the past 24 hours. The first was an email asking me about personal information management and automated tagging, and the second was an interview I read with Mike Moran, a thought leader in search and speaker at one of our Gilbane Conferences. In the interview, Mike talks about personalized search. Then Information Week referenced search personalization in an article about a patent suit against Google.
Here is my take on the many personalized search themes that have recently emerged. From dashboards to customizing results, options to focus on particular topics or types of content, socialized search to support interacting with and sharing results, to retrieving content we personally created or received (email), content we used or were named in, all might be referred to as search personalization. Getting each to work well will enhance enterprise search but....
Knowing how transient and transformative our thoughts and behaviors really are, we should focus realistically on the complexity of producing software tools and services that satisfy and enhance personal findability. We are ambiguous beings, seeking structured equilibrium in many of our activities to create efficiency and reduce anxiety, while desiring new, better, quicker and smarter devices to excite and engage us. Once we achieve a level of comfort with a method or mechanism, whether quickly or over time, we evolve and seek change. But, when change is imposed on an unprepared mind, our emotions probably override any real benefit that might be gained in productivity. Then we tend to self-sabotage the potential for operational usefulness when an uncomfortable process intrudes. Mental lack of preparedness undermines our work when a new design demands a behavioral shift that lacks connection to our current state or past experiences. How often are we just not in a frame of mind to take on something totally alien, especially with deadlines looming?
Look at the single most successful aspect of Google, minimalism in its interface. One did not need to wade through massively dense graphics scrambled with text in disordered layouts to figure out what to do when Google first appeared. The focus was immediately obvious.
I am presenting this challenge to vendors; there is a need to satisfy a huge array of personal preferences while introducing a minimal amount of change in any one release. Easy adoption requires that new products be simple. Usefulness must be quickly obvious to multiple audiences.
I am presenting this challenge to technology users; focus your appetite. Decide before shopping or adopting new tools what would bring the most immediate productivity gain and personal adoptability for maximum efficiency. Think about how defeated you feel when approaching a new release of an upgraded product that has added so many new "bells and whistles" that you are consumed with trying to rediscover all the old functions and features that gave your workflow a comfortable structure. Think carefully about how much learning and re-adjusting will be needed if you decide on technology that promises to do everything, with unlimited personalization. It may be possible, but does it really feel personally acceptable.
]]>I base this predication on my observation of the on-going struggle for organizations to get a strong framework in place to manage content effectively. By effectively I mean, establishing solid metadata, governance and publishing protocols that ensure that the best information knowledge workers produce is placed in range for indexing and retrieval. Sustained discipline and the people to exercise it just aren't being employed in many enterprises to make this happen in a cohesive and comprehensive fashion. I have been discouraged by the number of well-intentioned projects I have seen flounder because organizations just can't commit long-term or permanent human resources to the activity of content governance. Sometimes it is just on-again-off-again. What enterprises need are people with deep knowledge about the organization and how its content fits together in a logical framework for all types of knowledge workers. Instead, organizations tend to assign this job to external consultants or low-level staffers who are not well-grounded in the work of the particular enterprise. The results are predictably disappointing.
Enter semantic search technologies where there are multiple algorithmic tools available to index and retrieve content for complex and multi-faceted queries. Specialized semantic technologies are often well suited to shorter term projects for which domain specific vocabularies can be built more quickly with good results. Maintaining targeted vocabulary ontologies for a focused topic can be done with fewer human resources and a carefully bounded ontology can become an intelligent feed to a semantic search engine, helping it index with better precision and relevance.
This scenario is proposed with one caveat; enterprises must commit to having very smart people with enterprise expertise to build the ontology. Having a consultant coach the subject matter expert in method, process and maintenance guidelines for doing so is not a bad idea but the consultant has to prepare the enterprise for sustainability after exiting the scene.
The wager here is that enterprises can ramp up semantic search with a series of short, targeted projects, each of which establishes a goal of solving one business problem at a time and committing to efficient and accurate content retrieval as part of the solution. By learning what works well in each situation, intranet web retrieval will improve systematically and thoughtfully. The ramp to a better semantic Web will be paved with these interlocking pieces.
Keep an eye on these companies to provide technologies for point solutions in business critical applications: Basis Technology, Cognition Technology, Connotate, Expert Systems, Lexalytics, Linguamatics, Metatomix, Semantra, Sinequa and Temis.
This plays out in countless variations but the bottom line is that impatience with human processing time and the assumption that a search engine "ought to be able to" solve this problem without human intervention cripple possibilities for success faster than anything else.
It is time for search implementation teams to get realistic about the tasks that must be executed and milestones to be reached. Teams must know how they are going to measure success and reliability, then to stick with it, demanding that everyone agrees on the requirements before throwing the towel in at the first executive anecdote that the "dang thing doesn't work."
There are a lot of steps to getting even an out-of-the-box solution working well. But none is more important than paying attention to these:
• Know your content
• Know your search audience
• Know what needs to be found and how it will be looked for
• Know what is not being found that should be
The operative verb here is to know and to really know anything takes work, brain work, iterative, analytical and thoughtful work. When I see these reactions from IT upon setting off a search query that returns any results: "we're done" OR "no error messages, good" OR "all these returns satisfy the query," my reaction is:
• How do you know the search engine was really looking in all the places it should?
• What would your search audience be likely to look for and how would they look?
• Who is checking to make sure these questions are being answered correctly?
• How do you know if the results are complete and comprehensive?
It is the last question that takes digging and perseverance. It is pretty simple to look at search results and see content that should not have been retrieved and figure out why it was. Then you can tune to make sure it does not happen again.
To make sure you didn't miss something takes systematic "dog work" and you have to know the content. This means starting with a small body of content that it is possible for you to know, thoroughly. Begin with content representative of what your most valued search audience would want to find. Presumably, you have identified these people through establishing a clear business case for enterprise search. (This is not something for the IT department to do but for the business team that is vested in having search work for their goals.) Get these "alpha worker" searchers to show you how they would go about trying to find the stuff they need to get their work done every day, to share with you some of what they consider some of the most valuable documents they have worked with over the past few years. (Yes, years - you need to work with veterans of the organization whose value is well established, as well as with legacy content that is still valuable.)
Confirm that these seminal documents are in the path of the search engine for the index build; see what is retrieved when they are searched for by the seekers. Keep verifying by looking at both content and results to be sure that nothing is coming back that shouldn't and that nothing is being missed. Then double the content with documents on similar topics that were not given to you by the searchers, even material that they likely would never have seen that might be formatted very differently, written by different authors, and more variable in type and size but still relevant. Re-run the exact searches that were done originally and see what is retrieved. Repeat in scaling increments and validate at every point. When you reach points where content is missing from results that should have been found using the searcher's method, analyze, adjust, and repeat.
A recent project revealed to me how willing testers are to accept mediocre results when it became apparent how closely content must be scrutinized and peeled back to determine its relevance. They had no time for that and did not care how bad the results were because they had a pre-defined deadline. Adjustments may call for refinements in the query formulation that might require an API to make it more explicit, or the addition of better category metadata with rich cross-references to cover vocabulary variations. Too often this type of implementation discovery signals a reason to shut down the project because all options require human resources and more time. Before you begin, know that this level of scrutiny will be necessary to deliver good-to-great results; set that expectation for your team and management, so it will be acceptable to them when adjustments are needed for more work to be done to get it right. Just don't blame it on the search engine - get to work, analyze and fix the problem. Only then can you let search loose on your top target audience.
]]>There is one aspect of search output (results) that always makes an impression when I search. Sometimes the display is clear and obvious and other times the first thing that pops into my mind is "what the heck am I looking at" or "why did this stuff appear?" In most cases, no matter how relevant the content may end up being to my query, I usually have to plow through a lot (could be dozens) of content pieces to confirm the validity or usefulness of what is retrieved.
Admittedly, much of my searching is research or helping with a client's intranet implementation, not just looking for a quick answer, a fact or specific document. When I am in the mode for what I call "quick and dirty" search, I can almost always frame the search statement to get the exact result I want very quickly. But when I am trying to learn about a topic new to me, broaden my understanding or collect an exhaustive corpus of material for research, sifting and validating dozens of documents by opening each and then searching within the text for the piece of the content that satisfied the query is both tedious and annoyingly slow.
That is where Documill could enrich my experience considerably for it can be layered on any number of enterprise search engines to present results in the form of precise thumbnails that show where in a document the query criterion/criteria is located. In their own words, "it enhances traditional search engine result list with graphically accurate presentation of the content."
Here are some ideas for its application:
I highly recommend that you ask your search engine solution provider about incorporating Documill into your enterprise search architecture. And, if you have, please share your experiences with me through comments to this post or by reaching out for a conversation.
This week the predictable happened, it was an article by Information Week's Sr. VP focusing on Autonomy's terrific business run in a tough economy. Fair enough - it happens all the time for big spenders.
I just want to remind readers that if you are a small unit in a large organization or a small or medium business, there are dozens of enterprise search solutions that will serve you extremely well, with much lower cost of ownership and startup effort than Autonomy. You do not need the biggest or fastest growing company's products to get good or even excellent solutions. Furthermore, the chances of getting superior customer support and services from a more modest company, which is focused exclusively on search excellence, are much better.
Be sure to check out the offerings at the Gilbane Conference in San Francisco next week. A lot more guidance and good case studies will give you an earful of what else to consider. The search headliners at the conference with Hadley Reynolds moderating are:
E8. Search Survival Guide: Delivering Great Results
Speakers: Randy Woods, Co-founder & Executive VP, non-linear creations, Best Practices for Tuning Enterprise Search and Miles Kehoe, President, New Idea Engineering
E9/I5. The Next Big Thing: Tomorrow's Search Revealed
Speakers: Stephen Arnold, ArnoldIT, What You Need to Know About Google Dataspaces and Jeff Fried, Senior Product Manager, Microsoft
E10/I6. Bringing it All Together: Perils and Pitfalls of Search Federation
Speakers: Helen Mitchell Curtis, Senior Program Director of Enterprise Solutions, MacFadden, Federated Search in a Disparate Environment, Larry Donahue, Chief Operating Officer & Corporate Counsel, Deep Web Technologies, Federated Search: True Enterprise Search and Jeff Fried, Senior Product Manager, Microsoft
E11/I7. The Special Case of Categories - and Where To Find Them
Speakers: Joseph Busch, Founder, Taxonomy Strategies, Taxonomy Validation, and Arje Cahn, CTO, Hippo, Find What You Need in Unstructured Content with the Help of Others (and your CMS): Demo of Wikipedia with Faceted Search
E12/I8. It's Easier with Structure: Leveraging Markup for Better Search
Speakers: Dianne Burley, Industry Specialist, Nstein Technologies, Semantic Search and J. Brooke Aker, CEO, Expert System, A 3-Step Walk Through ECM Using Semantics
E13/I9. Improving SharePoint Search & Navigation with a Taxonomy and Metadata
]]>Here is the April news:
Marketwire. Bell Mobility and Coveo Partner to create Enterprise Search from Bell, an Exclusive Enterprise-Grade Mobile Search Solution, March 31, 2009.
My diminished manual dexterity and old eyes have discouraged me from embracing smart mobile devices but this announcement demonstrates that Coveo does understand the future of search. Coveo is releasing many enhanced functions and new options but the emphasis on search on a handheld is compelling for all road warriors. From what I understand, setup is pretty quick and productivity is immediate. This is a smart move to support Coveo's current customer base and a draw for new ones.
Velten, Carlo. Official Qitera Blog: Qitera Enterprise - Search as a Service, April 2, 2009
There is a 15-day free trial being offered for this new software as a service for aggregating and sharing Internet search results with colleagues within the enterprise. As a consultant in knowledge management, helping people with collaboration and sharing technologies for KM processes, I find this an intriguing option. If any readers have checked it out, please let us know your impressions. http://www.qitera.com/corp/products/overview
Knovel Enhances Engineering Reference Offering, Adding Works from Five New Publishers, 04/06/2009
As a former technical librarian in a Fortune 500 chemical company, I contributed to a massive manually built index to technical information to support our research scientists. We ensured that all property data for any materials we developed, or developed by our competitors was indexed for rapid retrieval (e.g. what is the thermal conductivity of ABC grade of graphite). The overhead for scanning journals, government documents, patents and conference papers to harvest and categorize that information was enormous. Since I learned about Knovel in 2002, I have been a huge fan of the value of the content they codify and make accessible through their proprietary retrieval tools. This content is re-purposed through licenses with publishers who understand the increased value of being able to manipulate tables, charts and graphs as well has being able to compare data from various sources of reference books. Engineers and scientists need to be able to find data expressed in the most relevant form for their purpose. Knovel provides an aggregating and retrieval engine, and researchers can then normalize search results dynamically themselves through simple re-sorting operations. If you want to be kept abreast of the wealth of content that continues to come on-line from Knovel, be sure to visit their site and sign up for announcements. I recommend a subscription to their services to every scientific and engineering library in my client organizations.Endeca Technologies, Inc. ... today announced a formalized partnership to deliver Endeca's Digital Asset Navigator solution on Open Text Digital Media Group's Enterprise Media Management Solution.
April 16, 2009. Endeca's Digital Asset Navigator offers an unprecedented access and discovery experience, combining Endeca's market leading search, Guided Navigation and Content Spotlighting capabilities. It integrates related data from Open Text's enterprise Digital Asset Management solution, as well as databases, file servers, enterprise applications and other source systems... The joint solution also takes advantage of Endeca's advanced security capabilities to ensure that users only have access to data they are approved to see and use...
This is an interesting alliance, to be sure. Digital asset management is an arena ripe for growth and it has not gotten the wide-spread traction I believe it deserves. For publishers and R & D operations the productivity gains can be huge and this combined offering may intensify focus on an under-leveraged technology by ensuing high security and excellent retrieval. Looking at the headline, I would just advise that they pare down the labeling to something pithy and memorable.
X1 is gaining some serious traction and I know it appeals to engineers who like to maintain good order with their piles of data, particularly the flow of large quantities that show up in email and feeds. I spent a few hours four years ago with an engineer, an X1 devotee, who had tagged his email in text files scrupulously and then used X1 to index them every night. He swore by its value and reliability. This looks like they recognize the service they provide to customers who live in email and need to master their desktop domain. Theirs is a niche with a large audience to capture.
ISYS Search Software Announces Release of ISYS:sdk 9. April 28, 2009. Newest Version of Company's Integration Kit Offers Dramatic Performance and Scalability Enhancements, Intelligent Content Analysis and Parametric Search.
ISYS made some significant management changes over the past few months and they are clearly moving along with their marketing efforts as they recognize the value of expanding the re-seller partnership options. Customer comments that come my way continue to be favorable and they have a good story to tell.
Data Search Technology Used by FBI Makes its Way to Enterprises. eWeek New York, NY, April 29, 2009
Add Chiliad to the list of platform search engines, now that they are being highlighted for their value at the FBI. With deep (20+ years) roots in government programs (including early DARPA research and later the SBIR) and a burst of interest and investment by the government after 9/11, Chiliad is taking its venture into more commercial opportunities. We will see how they stack up against the big players in the marketplace. They must have learned something as they worked to "connect the dots" for the FBI.
There have been a lot more stories this past month, but these notices are the ones that kept me engaged in contemplating the enterprise search marketplace that just keeps putting up more options.
]]>One of my own discoveries is that whether I am conducting a meeting, moderating or just attending, seeking out people who might have experiences that could be educational for me is both a way to get into a nice business relationship but it also helps break the ice. It can be awkward going to meetings where we know nobody in advance. Having an agenda that involves meeting people is the ultimate networking model. You might notice that a lot of social networking sites, like LinkedIn, have included a function for asking questions. This has proven popular and I know several people who have leveraged it in beneficial ways.
I have just come from two days at the Infonortics Search Engine meeting and many of you will soon be attending the Enterprise Search Summit in New York, The Gilbane Group conference in San Francisco or SemTech 2009 in San Jose. Here are a few suggestions on how to go shopping for great insight on search tools while establishing a relationship could nurture both you and those you engage for many years to come. Any one of these can start the conversation but think ahead about what you want to ask next once you have your initial answer:
Q: Hi, are you at this conference because you are just beginning to look for a search engine or to find answers about one you are already using? Depending on the answer you will want to find out what they have used, looked at, tested or are researching and what they have learned in the process.
Q: Hi, I see you are from ABC Corporation. How are you involved with search technology there? The answer will give you an idea what line of questioning you might pursue based on the person's presumed experience and knowledge. IT people, developers, content managers or expert searchers will each have a different view of the technologies they have or would like to use. Any role offers a unique perspective for you to draw out and understand for your own institution. Knowing how different professionals view search in other organizations can give you insight into the people you may have to team with in your own organization.
Q: Have you heard any talks at this meeting that have been particularly helpful for you? What have you learned that you didn't know about before? Follow up, and if you sense that some expertise you have might be interesting, sharing it can begin to build a trusted exchange that might prove helpful to you both.
Q: What are a couple of mandatory requirements for a search engine in your organization? Have you been using anything recently that you feel is serving you well or are you having problems? Any time you get a response from another attendee that indicates they are experienced and engaged with specific products, learn everything you can about their: selection process, implementation, deployment and user experiences. Talk to them about what their objectives were and whether and how those were met.
Going to meetings, chatting up attendees, asking questions, and sharing what you know are great ways to build a community of practice outside your internal communities. This brings fresh insights and gives you a valuable networking resource. Don't leave without contact information so you can continue the dialogue. Continue it with online exchanges based on their preference for communication.
Finally, the expense of going to meetings is increasingly hard to justify. But the benefit of finding key vendors and others with a common purpose in one place where you can quickly coalesce around the topic of search (or any other topic) gives you an easy sociability that can then be sustained. To solidify what you have learned and from whom, write a trip report; broadly disseminate it to all those in your enterprise network or team, as well as your boss. This sharing will be appreciated and should underscore the value you know how to accrue from technical meetings. Learning is an essential part of job growth and letting others know that you do it well is important.