Curated for content, computing, and digital experience professionals

Month: October 2010 (Page 1 of 3)

Lucene Open Source Community Commits to a Future in Search

It has been nearly two years since I commented on an article in Information Week, Open Source, Its Time has Come, Nov. 2008. My main point was the need for deep expertise to execute enterprise search really well. I predicted the growth of service companies with that expertise, particularly for open source search. Not long after I announced that, Lucid Imagination was launched, with its focus on building and supporting solutions based on Lucene and, its more turnkey version, Solr.

It has not taken long for Lucid Imagination (LI) to take charge of the Lucene/Solr community of practice (CoP), and to launch its own platform built on Solr, Lucidworks Enterprise. Open source depends on deep and sustained collaboration; LI stepped into the breach to ensure that the hundreds of contributors, users and committers have a forum. I am pretty committed to CoPs myself and know that nurturing a community for the long haul takes dedicated leadership. In this case it is undoubtedly enlightened self-interest that is driving LI. They are poised to become the strongest presence for driving continuous improvements to open source search, with Apache Lucene as the foundation.

Two weeks ago LI hosted Lucene Revolution, the first such conference in the US. It was attended by over 300 in Boston, October 7-8 and I can report that this CoP is vibrant, enthusiastic. Moderated by Steve Arnold, the program ran smoothly and with excellent sessions. Those I attended reflected a respectful exchange of opinions and ideas about tools, methods, practices and priorities. While there were allusions to vigorous debate among committers about priorities for code changes and upgrades, the mood was collaborative in spirit and tinged with humor, always a good way to operate when emotions and convictions are on stage.

From my 12 pages of notes come observations about the three principal categories of sessions:

  1. Discussions, debates and show-cases for significant changes or calls for changes to the code
  2. Case studies based on enterprise search applications and experiences
  3. Case studies based on the use of Lucene and Solr embedded in commercial applications

Since the first category was more technical in nature, I leave the reader with my simplistic conclusions: core Apache Lucene and Solr will continue to evolve in a robust and aggressive progression. There are sufficient committers to make a serious contribution. Many who have decades of search experience are driving the charge and they have cut their teeth on the more difficult problems of implementing enterprise solutions. In announcing Lucidworks Enterprise, LI is clearly bidding to become a new force in the enterprise search market.

New and sustained build-outs of Lucene/Solr will be challenged by developers with ideas for diverging architectures, or “forking” code, on which Eric Gries, LI CEO, commented in the final panel. He predicted that forking will probably be driven by the need to solve specific search problems that current code does not accommodate. This will probably be more of a challenge for the spinoffs than the core Lucene developers, and the difficulty of sustaining separate versions will ultimately fail.

Enterprise search cases reflected those for whom commercial turnkey applications will not or cannot easily be selected; for them open source will make sense. Coming from LI’s counterpart in the Linux world, Red Hat, are these earlier observations about why enterprises should seek to embrace open source solutions, in short the sorry state of quality assurance and code control in commercial products. Add to that the cost of services to install, implement and customize commercial search products. The argument would be to go with open source for many institutions when there is an imperative or call for major customization.

This appears to be the case for two types of enterprises that were featured on the program: educational institutions and government agencies. Both have procurement issues when it comes to making large capital expenditures. For them it is easier to begin with something free, like open source software, then make incremental improvements and customize over time. Labor and services are cost variables that can be distributed more creatively using multiple funding options. Featured on the program were the Smithsonian, Adhere Solutions doing systems integration work for a number of government agencies, MITRE (a federally funded research laboratory), U. of Michigan, and Yale. CISCO also presented, a noteworthy commercial enterprise putting Lucene/Solr to work.

The third category of presenters was, by far, the largest contingent of open source search adopters, producers of applications that leverage Lucene and Solr (and other open source software) into their offerings. They are solidly entrenched because they are diligent committers, and share in this community of like-minded practitioners who serve as an extended enterprise of technical resources that keeps their overhead low. I can imagine the attractiveness of a lean business that can run with an open source foundation, and operates in a highly agile mode. This must be enticing and exciting for developers who wilt at the idea of working in a constrained environment with layers of management and political maneuvering.

Among the companies building applications on Lucene that presented were: Access Innovations, Twitter, LinkedIn, Acquia, RivetLogic and Salesforce.com. These stand out as relatively mature adopters with traction in the marketplace. There were also companies present that contribute their value through Lucene/Solr partnerships in which their products or tools are complementary including: Basis Technology, Documill, and Loggly.

Links to presentations by organizations mentioned above will take you to conference highlights. Some will appeal to the technical reader for there was a lot of code sharing and technical tips in the slides. The diversity and scale of applications that are being supported by Lucene and Solr was impressive. Lucid Imagination and the speakers did a great job of illustrating why and how open source has a serious future in enterprise search. This was a confidence building exercise for the community.

Two sentiments at the end summed it up for me. On the technical front Eric Gries observed that it is usually clear what needs to be core (to the code) and what does not belong. Then there is a lot of gray area, and that will contribute to constant debate in the community. For the user community, Charlie Hull, of flax opined that customers don’t care whether (the code) is in the open source core or in the special “secret sauce” application, as long as the product does what they want.

Adobe Releases Digital Publishing Suite

Adobe has released a series of tools designed to expedite digital publishing across various platforms. Digital Publishing Suite allows publishers to upload articles directly from InDesign CS5, and supports PDF and HTML5. The software can run on RIM’s PlayBook, Samsung’s Galaxy Tab, Apple’s iPad and forthcoming Android platforms. The software allows dynamic user control of online publications, and readers are able to resize pages and move content around to suit their needs. Graphics, video and audio content can be built into the publications. The full version of Digital Publishing Suite will be available next year for a $699 licence fee, but beta code is now available from Adobe. http://www.adobe.com/

You Are Your Organization’s Chief Collaboration Officer

There have been a couple of interesting blog posts about organizational collaboration leadership penned recently by respected, influential thinkers. Last week, Morten Hansen and Scott Tapp published Who Should Be Your Chief Collaboration Officer? on the Harvard Business Review site. Yesterday, Dion Hinchcliffe posted Who should be in charge of Enterprise 2.0? on Enterprise Irregulars.

It is logical that the question of the proper seat of ownership for enterprise collaboration efforts is being raised frequently at this moment. Many organizations are starting the process of rationalizing numerous, small collaboration projects supported by enterprise social software. Those social pilots not only need to be reconciled with each other, but with legacy collaboration efforts as well. That effort requires leadership and accountability.

Both of the posts cited above – as well as the comments made on them – add valuable ideas to the debate about who should be responsible for stimulating and guiding collaboration efforts within organizations. However, both discussions miss a critical conclusion, which I will make below. First, allow me to share my thoughts on the leadership models suggested in the posts and comments.

While it is critical to have collaboration leadership articulated and demonstrated at the senior executive level, the responsibility for enterprise collaboration cannot rest on one person, especially one who is already extremely busy and most likely does not have the nurturing and coaching skills needed for the job. Besides, any function that is so widely distributed as collaboration cannot be owned by one individual; organizations proved that long ago when they unsuccessfully appointed Chief Knowledge Officers.

Governance of enterprise collaboration can (and should) be provided by a Collaboration Board. That body can offer and prescribe tools, and establish and communicate policy, as well as good practices. However, they cannot compel others in the organization to collaborate more or better. Yes, Human Resources can measure and reward collaboration efforts of individuals, but they can only dangle the carrot; I have never seen an organization punish an employee for not collaborating when they are meeting other goals and objectives that are given higher value by the organization.

There is only one person (or many, depending on your perspective) for the job of actively collaborating – YOU! Ultimately, each individual in the organization is responsible for collaboration. He can be encouraged and incented to collaborate, but the will to work with others must come from the individual.

Collaboration in the enterprise is similar in this regard to knowledge management, where the notion of Personal Knowledge Management (PKM) has been gaining acceptance. PKM advocates believe that having each member of the organization capture, share, and reuse knowledge, in ways that benefit them personally, is far more effective than corporate mandated knowledge management efforts, which generally produce benefits for the enterprise, but not the individuals of which it is comprised.

So it is with collaboration. If an individual does not see any direct benefit from working with others, they will not do so. Conversely, if every employee is empowered to collaborate and rewarded in ways that make their job easier, they will.

The Enterprise 2.0 movement has correctly emphasized the emergent nature of collaboration. Individuals must be given collaboration tools and guidance by the organization, but then must be trusted to work together to meet personal goals that roll-up into measures of organizational success. The only individual that can “own” collaboration is each of us.

« Older posts

© 2020 The Gilbane Advisor

Theme by Anders NorenUp ↑