Using Social Search to Drive Innovation through Collaboration

Lynda Moulton, Lead Analyst, Enterprise Search, Gilbane Group, October 2007

Executive Summary

Organizations need innovation not only to grow market share and to influence positive outcomes but also to establish sustainability. Innovation can be reflected in new products to gain market position, as well as in operations, marketing and services to build for a strong and continued leadership presence.

Whether the organization is a for-profit business enterprise, a learning institution, a not-for-profit society, or governmental agency, positive achievement comes from the collective work of the parts. Star individual performers can affect short term visibility or wins, but only through collaboration and the shared insights of many types of individuals does a repeatable model for success emerge.

Search technologies, when implemented with features that invite routine use, have the potential to bring collaborative clarity to problem-solving, setting business strategies and exposing content for innovative use in all areas of the enterprise.

Planning for any enterprise initiative requires vision. But what if true visionaries are not on the team to focus the project on the right objectives and the right sequence of steps? If you are challenged to get the vision right, this paper offers ideas to inspire a new vision, one that embraces social computing. Specifically, it is about how collaborative search can impact innovation in any vertical industry or segment of an enterprise. It is partly about seeing a clear pathway to success and knowing what you define as success. It is also about seeking and leveraging solutions that are appropriate to a particular outcome and will be enthusiastically adopted in your organization.

Search has been narrowly defined in two categories for decades: structured search commercially available since the early 1970s, and unstructured/full text search deployed in organizations since the mid-1970s and on the Web since the mid-1990s. Now we are beginning to see an explosion of more imaginative ways to leverage search. Whether used as a tool for professionals doing research, or as a primary tool for job execution, it is as critical to your enterprise as capital equipment was to businesses in the past.

In this paper, we try to help you strike a balance between solid business thinking and explosive enthusiasm for any new technology option with 2.0 in the description. Our goal is to give you a new way of looking at old challenges.

We will try to answer:

Why are transformational technologies worth adopting?
What are some key concepts that help contribute to social computing?
How does social search contribute to innovation?
Where do BI and search meet in the enterprise?
What are the business outcomes we seek from social search experiences?
How is social search being leveraged in different industries?

Making a Business Case for Transformational Technology

Strategic thinking and innovation appear to be partners in tension when it comes to enterprise planning and operational execution. Strategic thinking implies structure or formality while innovation is often aligned with thinking outside the box. You might wonder what this has to do with search technologies – the answer is a lot, if you don’t have search aligned with your enterprise. Considering search options and justifying a system is really about creating a strategy based on projected business outcomes. Developing a solid set of outcomes that your organization is challenged to achieve makes it easier to define the precise technology assets your organization needs.

Technology tools that move the enterprise toward operational efficiencies in collaborating, finding expertise and stimulating knowledge-seeking behaviors can be crucial components to meshing strategy with outcomes.

If your enterprise’s most targeted outcomes include: better alignment with customer needs, gains in operational efficiency, new or improved products, these are reason enough to strive for innovation. Better, gains, new and improved are all words descriptive of positive change and an explicit call to innovation. Improvement has always been a driver for innovation but it usually takes game-changing thinking to achieve. Often it is a person: an inspirational speaker, new way of thinking, or an influencer in the culture of the enterprise that drives innovation. But we also know that technologies that are easily and enthusiastically leveraged can have an impact, as well.

In the next few pages you will be challenged to imagine the ways in which social search technology can be the transformational jolt in your enterprise that inspires new ways of thinking and doing. It provides a connector between people and technology that is just beginning to gain traction. We will expose you to case examples that reveal what vertical industry leaders are doing to transform strategic goals into positive innovative outcomes.

Advanced Search Concepts Inspired by Social Computing

Before describing how some new social search functions are being leveraged it may help to understand some concepts shared with Web 2.0, Collaboration, and Business Intelligence tools. It is useful to establish some clarity around these complementary topics to focus our thinking on terminology that is heavily used in conjunction with enhanced versions of search. In addition to giving you working definitions, we’re adding some commentary on how they relate to newer collaborative search. They are all noun forms of verbs that describe a search system operation or user action associated with it.

Aggregating – usually applied to the activity of forming distinct sets of content. While the process of crawling or spidering a number of repositories in an enterprise to create a single index has been commonly understood by search aficionados for some years, there is another, more recent, development. It relates to the computational assembly, on the fly, of content into an order by broad topics or relevancy by search engines on the Web as part of the results display.

Analyzing and Analytics – the process of enhancing retrieved content using graphical visualization or reformatting content into reports for the purpose of clarifying aggregated results for others. Graphs, charts, and other layouts for reassembling data to create new understanding about information have morphed from humans generating reporting to software assembly of content to options for supporting human interaction with machine analyzed results. An example would be for humans to add commentary to relevance ranked results with links to other results that might illustrate a bias.

Annotating – the act of contributing additional content in the form of comments, clarifications, and expansion of substance to content. This is a new search engine feature that is beginning to show up for the purpose of enhancing a set of results. It is a companion of human enabled aggregation. When paired, they form a strong element of the collaborative foundation of social search. Of particular note is the elimination of the need to route and email discovered content because the commentary is embedded in the results for others to find in their own time and space, or as part of their preferred workflow. However, the function of automatic Alerts can also be enabled, to push out via email or RSS feeds to announce the existence of enhanced content.

Clustering – is an algorithmic or human process for gathering unstructured content into a common space for the purpose of assigning it to a pre-determined category. Clustering may be by topic, type of content format, source, author, or other attribute. Clustering is an easy way to build navigation into a search environment intended for discovery because it will enable other workers to narrow down their search more quickly. Seeking information on “cells” for biological research is aided by having content on the subject organized by tabs or folders named: Biological, Telecommunications, Criminal Justice, Social Systems, and Electrochemistry. Humans will typically lend a more granular categorizing schema to content that reflects the special interests of their domain. Enabling researchers to create and share clusters or folders is an even more powerful feature of some systems.

Collaborating – is a human activity now widely support by technology to aid communication, sharing, facilitating, and teaching. Social search has as its principal function, supporting collaborative activities in enterprises. Saving, sharing, and exporting search results has been a clumsy operation until now, and slow to be widely adopted. The new user interfaces for doing this quickly and updating the results easily later is a great addition to search.

Connectivity – is an essential part of a technology infrastructure in which we will be coping with a heterogeneous environment of applications from many vendors for many years to come. It is the tool set from most major search vendors that will ensure extraction and indexing from all the candidate data repositories and file structures an enterprise is likely to have in place. Having a rich suite of pre-built connectors is a must to get maximum coverage of your enterprise from a search product.

Federating – is an expansion of the concept of aggregation. It has play in a multi-domain environment (internal sites or a mix of internal and external). Across domains it supports at least four distinct functions:

Integration of the results from a number of targeted searchable domains, each with its own search engine
Disambiguation of content results when similar but non-identical pieces of content might be included
Normalization of search results so that content from different domains is presented similarly
Consolidation of the search operation (standardizing a query to each of the target search engines) and standardizing the results so they appear to be coming from a single search operation

Navigating – is a presentation environment based on linking that simplifies selection of a path for viewing the next content set. It usually supports presenting a narrowing or expanding view of options. It has the advantage of presenting limited views, one at a time, so as not to overwhelm the user. If the content domain is unstructured, or is a combination of structured and unstructured, the navigation scheme is usually categories of content. When the content has been structured with rich metadata, the search engine will make use of assigned metadata to form a simplified taxonomic layout of the terminology for simple navigation. In essence, navigation presentations are a textual form of visualizing a domain of content by topics.

Personalization through Dashboards – is a visualization mechanism that organizes and presents information in easy-to-read formats. Search engines are using these features to present a richer and more interactive view of search results in formats users are comfortable navigating and sharing with others. The fact that search results can be customized for understanding and organized to suit a specific purpose is one aspect of social computing that will transform the search experience significantly. In a recent CIO Insight survey of executives on business intelligence (BI) [1.], the use of dashboards for personalizing content was rated as very important because retrieved results are so often misinterpreted.

Tagging – is an activity for labeling content. It had been equated to indexing or cataloging by professionals using controlled lists of terms to categorize content. However, it is increasingly used in a collaborative mode to label content dynamically as it is discovered without regard for controlled terminology or conventions. Large lists of tags that may scale quickly in a collaborative space can, through social tagging in project teams be brought into focus more easily. This is a great asset for marking up clusters of search results on an ad hoc basis.

Visualizing – applies to a broad range of graphical interface displays of content. It has been a staple of BI and text analytics systems because graphical representations of extracted content help users “see” relationships more quickly. However, problems arise when the visualization is not easy to interpret or conveys the wrong idea about analyzed data. Enabling users who search to determine how best to represent their findings for their own community is a terrific improvement on a good search tool.

How Collaboration Drives Innovation

Self-service, self-sufficiency, and simplicity underscore the S in “social search” technologies. These are outcomes in the enterprise that create the same type of enthusiasm as mashups on the Web. At recent symposium of the Boston KM Forum featuring Dave Snowden and members of the original IBM Institute for Knowledge Management, several speakers conveyed the extent to which human connections form the basis for most of our successful technology experiments. We seek content that is attributable to trusted human sources, validation of retrieved content by others of interests similar to ours, and references to content “out there” that is contributed by experts we hold in high professional esteem. Content vetted and recommended by our trusted colleagues sits high on our list of what we read and use in our work.

Another outcome of the KM Forum was the business user’s perspective that workers have little patience for being forced to use inefficient and cumbersome systems. Their impatience rapidly morphs into outright rejection. They want to be able to access and use content they need when and where they need it. Usually, this means information in small chunks, not too much and appropriate to the activity embedded in their workflow. This is an ideal that comes from younger workers but it is also driving newer technology offerings. When workers become frustrated with tools their employers offer, they take their technical expertise and insights into how tools should work elsewhere. They often leave to build and deploy new products, which emerge in the marketplace with astonishing speed.

However, this is exactly the type of innovation that organizations want to tap and preserve for their own benefit.

Smart enterprises are recognizing the power of collaborative tools that help strengthen the ties among experts. They recognize the power of sharing insights, discoveries and teaching opportunities among experts.

We offer these observations to encourage readers to embrace the possibilities for better search tools and understand that they can make a big difference in the quality of work output and quality of the work experience for the work force in any enterprise. Departing frustrated employees are not a good business outcome but high retention, enthusiasm and optimal collaboration will make a difference in the bottom line when focused on purposeful outcomes.

Does Social Search Empower the Enterprise in Ways that BI Has Not?

We have long thought of search as being an “end-game” in all types of business applications, from Web content management to customer service solutions and all manner of operational systems supporting enterprise processes. Search was simply about being able to find a specific piece or set of content to fulfill an inquiry request. Let’s briefly look at another popular business concept, business intelligence or BI. BI has received major attention among analysts and executives because the technology applications delivered from this space appealed to the way they like to see data presented.

Defining BI as technologies that gather, store, analyze and make accessible data to help enterprise users make better business decisions, let us consider the ways in which it has failed to deliver expected results. While BI systems include decision support, query and reporting, online analytical processing, statistical analysis, forecasting and data mining, it can fall down in the ways it actually conveys data. If executives expect to discover new trends, gaps in organizational research, and insights into where they should be building new products, they may be disappointed. What they are seeking is really a distillation of data into information and then into knowledge. Because BI technology traditionally focuses on extracting and manipulating data from structured databases including numeric data, it often misinterprets unstructured content. It is not designed to uncover subtleties like intent, context and meaning.

However, social search that exploits linking of all types of content from diverse sources reveals just how much more valuable search results can be when analyzed, evaluated, visualized and shared by humans. This has taken us to the next generation of search enhancement, paying attention to search results for what they can teach us about our operations, products, science research, services and competition.

In October 2007, CIO Insight reported on what 200 executive respondents said they want from BI technology. These are needs that illustrate where the analysis of search results will be considered seriously whether it comes from one tool or another. Business needs highlighted are:

“Gaining insight into internal operations and processes” and customer facing activities.
Developing new products including patent discovery and improving operations
“Improving our ability to capture, analyze and provide real-time information”
Embed analysis into business processes and workflows
Eliminate errors when interpreting information and make it easier to understand
Find innovative ways of managing

We see a huge opportunity here to turn social search toward meeting these business needs with its ability to index both structured and unstructured data with deeper contextual relevance, and supplement it with human collaboration. Excellent search results coupled with strong analytical features, better visualization and superb data manipulation and reformatting options should be a winning combination to overcome misinterpreted or poorly analyzed results. We believe that the option of introducing a “social” or human collaborative component into enterprise search has great potential for enhancing and clarifying what users see in the information they use every day.

How We Define Positive Business Outcomes from Search

Like many technologies that support the retrieval, sharing and exchange of knowledge assets, the procurement and implementation of enterprise searchis usually justified based on its potential for:

Saving time
Uncovering lost assets
Discovering the “needle in the haystack”
Creating an opportunity for learning

These outcomes all have the potential for creating a net positive impact. Although we are not conditioned to think that way, they can also have a downside. We certainly do not want to dwell on negative outcomes. But strategic thinking requires that we at least consider that technologies can and do often take enterprises down unexpected paths that actually encumber progress, giving us efficiency in one area but suck up time in another so that the net gain is, in fact, a negative.

Let’s explore how these stated objectives can work for and against the best business strategy.

Saving Time

Saving knowledge worker’s time is viewed as the premier benefit of most technologies. The faster we discover, learn and utilize information to create new knowledge, the sooner improvements in our product can be rolled out. Conversely, we know that a rush to conclusions without vetting our source information or validating the authority of content can rob us of mindful usage of search results. Therefore, search technology coupled with features and techniques for rating content by peers or experts within a system can provide the benefit of saving time and improving the validity of content being retrieved. This might be thought of as human enhanced machine intelligence.

Uncovering Lost Assets

Retiring baby-boomers or experts being hired away by the competition are just two pressures that make organizations seek technologies that will aid the discovery of re-useable information in content left by those departed workers. We also know that the bulk of this content is unstructured and poorly documented as to purpose and original significance. A search engine that leads unsuspecting researchers to vast amounts of disorganized and random content is another potential time waster. The same content could be retrieved and sifted through many times, then left in the shared searchable space for others to encounter it repeatedly.

Think, however, of the great potential for a search system that enables tagging and annotating discovered content by those digging into it. Think of this as an editorial function, a sharing of insights or new perspectives, an early-warning system about content found and known to be faulty, or any number of other helpful knowledge worker contributions. Most organizations never have the luxury of a massive effort for retrospectively reviewing, cataloging and metatagging content left behind in retired worker’s personal repositories. Even if that content is moved to shared drives or central systems, it usually languishes. But if a simple mechanism exists for commentary at the point of every search to annotate and classify results, imagine what a body of collaborative new knowledge can accrue through routine day-to-day investigations. The contributed human intelligence described here is a positive addition to machine generated search results.

Discovery

An anecdote from a recent engagement brought to the author’s attention that just because a research project is terminated, does not mean there is no value in the content left behind. It is very common for the learnings of a research pursuit to be ignored by others because an entire program is abandoned. However, experience shows throughout history that important facts, established within the context of an incomplete project, could have eliminated errors in judgment, moved another project along faster, or stimulated a path to new thinking – if only those facts were found sooner. Search engines can turn up faulty information just as quickly as they can retrieve the golden “needle in the haystack.” A search engine that promotes simple flagging of unique nuggets or “voting” on content as candidates for another opinion may add up to sizable chunks of seemly random information that can be put to important use in the future.

Learning Opportunities

Finally, think about how many times in a week you send someone else a link or attachment with a comment about why you think it is important or why they might be interested. We all love to point to things we discover that are relevant to work we share and interests we have in common. We are especially proud to bring content to others’ attention when we can point to its value in a particular context. The opportunities for learning and expanding our own knowledge abound with linking and search technologies. But, we can find ourselves in lengthy distractions as links lead us from one related topic to another until we are far from the original topics.

What we want to do is create a virtual system of boundaries and guideposts that information discoverers can navigate without too many digressions. We know that the process of structuring clear and easily navigated results is going to be chaotic and messy. But social interaction within the search framework is a great metaphor for how teams work anyway. Only now, the interactions can be quicker and results more immediate because they are embedded in the knowledge worker’s own workflow. Meetings do not need to be scheduled, and the team can react when it is convenient. This type of process has its roots in mind mapping technologies with a collaborative bent.

Now you are armed with awareness of how implementation of desirable functions, embedded within a search engine, can help to avoid unwanted consequences. It is now possible to think of search as the platform that will truly give your content a refreshed face with potential for releasing far more value.

Social Search has Different Play Depending on the Industry

Vertical markets or industries each have their own content challenges and differences. As a final measure of beneficial search tools, we are going to revisit the new tools and search concepts described in the previous sections applied to different industries.

We present a major media company with a large news division. It needs to discover developing stories, build archives of background information, research developing events, and archive the content it creates. It survives on the speed, accuracy, authority, and innovativeness of presentation that it brings to its product, news. Behind the scenes of every broadcast summary or publication are hundreds of people retrieving, annotating, vetting and validating content on a 24X7 basis. The organization would die without search access to the worldwide web of open content, news feeds from other organizations, access to hundreds of deep-Web resources (paid search), as well as internally developed content from active production and archives. KEY POINT: New search tools that enable clustering by researchers to collaborate from local and remote sites, annotating and tagging content to coordinate content usage by a team producing a piece, is just the type of time savings we seek for best strategic outcomes.
We present a large pharmaceutical company whose principal asset is expertise in the science, art and business of bringing new candidate drugs to market. Its principal work product is information that comes from internal experimentation, intellectual property licensing and acquisitions, literature searching across the deep Web for primary source information, patents, and market analysis, and very deep BI data routinely created from its own databases and a reservoir of competitive intelligence harvested from field research. It takes thousands of content resources to contribute enough knowledge to produce a new drug compound. The process to decide which compounds to pursue, develop, test and submit for approval is made more complex by the diversity of the people involved in the total activity. They include market analysts, scientists, business managers, manufacturing chemical engineers, market researchers, sales and account representatives and trial overseers (physicians).This is an environment in which expertise access is vital. KEY POINT: Workers learn and innovate based on where they find experts who provide them with the fuel for their innovation. Search systems that enable a complex interweaving of resources from experts at every point in the process of research and development are crucial. These are the systems that support content federation and content re-organization that is tailored to each member of the audience. Finally, integration of all the content assembled for a multi-year, multi-million dollar effort places experts and content in the same contextual framework. The potential for maintaining these contextual and human relationships for easy navigation and finding the right expert within large team is invaluable.KEY POINT: Navigated search is a path for discovering content you might have missed but for exposure to other colleagues’ paths of discovery revealed through social tagging. This takes research support to a new level and opens the door to serendipitous experiences with potential for innovation greatly increased.
We present a multi-national electronics firm with a huge call center operation. Teams supporting a major client create and contribute content in random bursts of activity. Content may be contributed routinely about a new client just installing systems but months may pass between calls once the installation has stabilized. Changes in personnel at the vendor’s call center or client site create risk of discontinuity in understanding needs and history. Similarities in customers may result in very similar call histories contributed to the call knowledge-base by totally different service engineers. KEY POINT: The opportunity for reassembling dispersed chunks of information about similar projects worked on by geographically dispersed teams with different customers will benefit from automatic clustering that puts similar call information in the same categories. Tags and commentary contributed by field engineers will add a conversational exchange of tips and insights into how to handle unique situations. With the number of variables involved in any large, high-tech installation, field support and call centers profit enormously from the consolidation and normalization of remote experiences similar to theirs.
We present a collection of governmental bodies which operate independently but need access routinely to each other’s content. They also serve a constituency with variable needs for public content – the searchers and content are completely heterogeneous in every respect. KEY POINT: Search with dashboard and personalization features for teams deploying content to their constituents are the most efficient mechanisms for keeping up with perpetually changing content and content intent. Environments with little constituent continuity and variable needs driven by external factors need to react quickly to demands for ease of use, to create reusable display templates and constant repurposing of content. They are in need of dynamically tunable search tools. KEY POINT: On the government worker side, the potential for analyzing search results, visualized in new arrangements, can exploit content to expose operational deficiencies and expose trends in the use of the tools by constituents. For constituent users, this search environment, with its rich federating functions, eliminates the need for bouncing from one agency’s Web site to another.
We present a professional services firm in which only a small percentage of deliverable content to clients is totally new and unique. Proposals contain substantial boilerplate, and client reports are consistent in format with a substantive amount of re-usable content. However, high-priced expertise goes into developing strategies for solving client problems and bringing innovative solutions to new situations. KEY POINT: Building new and innovative strategies is supported by collaborative search in which archived content may form the basis of new assemblies of content that can be produced and quickly shared for immediate feedback and further investigation. Again, shared access and expert contributors become accessible to partners for bringing needed expertise quickly into the collaboration effort.

Summary

When new models for search and discovery are rolled out and adopted, every enhancement and improvement usually stimulates the imagination and whets the appetites of users for more and better features. Matching experts and their content is the ultimate experience for enriching content in context. Technology is not the expert, nor is it the collaborator, but it is an enabler. NET RESULT: Collaborative Search is about leveraging experience and knowledge by exploiting the full value of content. We need to view search as an enabling tool and a business asset to achieve a set of desirable process behaviors. Organizations need innovation not only to gain market share and to influence positive outcomes but also to establish sustainability.

Collaborative search is squarely in the thick of Web 2.0 social conventions and that includes its relationship to Andrew McAfee’s Enterprise 2.0. He defines it as having six key attributes which he shortens to SLATES: Search, links, authoring, tags, extensions, and signals. If you consider the attributes of collaborative search, its relationship to BI and the scenarios we have presented, it becomes clearer that collaborative or social search, as it is being offered today, more than meets McAfee’s criteria. Whether it leads to true innovation is a people problem but the tools are there to leverage them and the expertise they embody.

Just look at the variety of search needs that have been described in the previous examples to build a vision of how each need can lead to a positive outcome for an organization with a diverse community of users. By envisioning a holistic outcome, achieving your real business objectives becomes more attainable.

Social search allows you to create value on the fly from assets you already have – content and people. You are using the wisdom of multiple contributors to explain, support, transform, improve, engage, stimulate, challenge, expose, demonstrate, and illustrate.

The power of linking simulates how we learn and think. It also acts as an agent for innovation and change by helping to transform our thinking in subtle and non-obtrusive ways, which is what true collaboration should be.

[1.] CIO|Insight Research, Business Intelligence (fielded Aug 2007), Ziff Davis, Oct 2007.

Sponsor Information

Gilbane Group thanks Vivisimo for contributing the content on this page.

Vivísimo, Inc. Headquarters
1710 Murray Avenue
Pittsburgh, PA 15217
Telephone: +1-412-422-2499
Fax: +1-412-422-2495
http://www.vivisimo.com
Email: licensing@vivisimo.com

Vivísimo’s mission is to help organizations find, organize and use the massive amount of information available in today’s world. Leveraging its core competency in developing enterprise search software, as well as its experience in consumer search, Vivisimo delivers search solutions that improve workforce productivity, streamline business processes, raise customer satisfaction and increase sales.

For enterprises, Vivísimo’s Velocity Search Platform provides them with innovative search solutions that allow users to access, extract and manipulate all available content, regardless of location. The latest version of Velocity is the first major enterprise search platform to bring business-ready Web 2.0 functionalities, such as social tagging, social bookmarking, social networking and mash-ups to the market. By combining the simplicity of consumer search with the flexibility and control of enterprise software, organizations can reap the benefits of increased user adoption and satisfaction without compromising on performance, scalability, security or the ability to configure a solution specific to its business environment. Based on a service-oriented architecture that enables rapid deployment, Vivisimo Velocity can be up and running in just weeks.

Fortune 500 companies, top websites, prestigious publishers, government agencies and millions of users employ Vivísimo’s solutions. Vivísimo’s technology has received a tremendous amount of industry recognition. Search industry experts at eWeek, KMWorld, EContent, PC Week, and Search Engine Watch proclaimed Vivísimo as a provider of solutions that are changing the way people search.

brought to you by
Vivisimo

Using Social Search to Drive Innovation through Collaboration

Using Social Search to Drive Innovation through Collaboration

Table of Contents

Executive Summary

Making a Business Case for Transformational Technology

Advanced Search Concepts Inspired by Social Computing

How Collaboration Drives Innovation

Does Social Search Empower the Enterprise in Ways that BI Has Not?

How We Define Positive Business Outcomes from Search

Saving Time

Uncovering Lost Assets

Discovery

Learning Opportunities

Social Search has Different Play Depending on the Industry

Summary

Sponsor Information

Subscribe to the Gilbane Advisor

Choose Language

Topics we cover

Policies

Contact