The Gilbane Report: Volume 11, Number 2The Classification & Evaluation of Content Management Systems
May 2003
Download a PDF version of this article Read the news for this issue.
The Classification & Evaluation of Content Management Systems
The continued dramatic
growth in content management systems (CMSs) and technologies - there are hundreds
of CMSs, including dozens of open source tools - has defied the usual rules
of business software markets. The number of new product launches by old and
new companies somehow still manages to keep ahead of the ongoing consolidation.
This is very healthy. However, even a full-time market analyst paid to be a
content management expert is not going to be able to keep up with all the products
and features, especially since managing content involves technologies that go
well beyond a CMS (See last month's report on integration, infrastructure, and
content services technology.) IT strategists and project managers who have other
responsibilities don't stand a chance of keeping up-to-date.
Fortunately, there is an
industry effort gathering steam to provide an open and free list of CMS products
and features. This public domain classification will be based on an XML schema
(CMSML) so that anyone can use the information. Although the project is in its
early stages, there is already one public domain tool for mining this CMS data,
which we will make available at gilbane.com in early June. This is a huge and
complex task and promises to be ongoing. Even given a predictable perpetual
state of flux, this project should make information collection about CMSs and
their features a lot easier for everyone. This month Bob Doyle, editor of the
CMSML project, describes the project and how to participate, and shares the
draft feature list.
The Classification &
Evaluation of Content Management Systems
Content Management Systems
(CMSs) are a most unusual category of computer software. Most software categories
as well known as content management are dominated by a few products that have
captured market share and users' mind share. But the CMS space is different,
very different, especially the most familiar Web Content Management System segment.
There are several reasons for this difference.
First, since a
CMS has been notoriously hard to define (we address this below), there are
many very different products that think they are providing a "complete"
CMS. This includes fast-growing personal web publishing weblogs, group wikis,
and organizational news portals, which have received a lot of press lately.
As the Gilbane Report
suggested three years ago, "content management" has become an all-encompassing
buzzword for document management, information management, and knowledge management.
"It is largely because web publishing is so different from other types
of publishing that content management has come to encompass such a wide variety
of functions." (Volume 8, Number 8, October 2000).
Second, the huge
potential market has attracted many independent software developers. Although
we have seen a shakeout among the major early players, market entrants continue
to appear, with several hundred companies listed on the DMOZ Open Directory
Project under Content Management.
Third is the very
low cost of market entry. With online marketing and delivery over the web,
many companies dream of being the next Vignette. They don't realize that software
is a small fraction of a content management solution.
Perhaps most important is
that almost everyone who is managing a web site is fighting, some desperately,
to get control over their site management. As a result, tens, perhaps hundreds,
of thousands of organizations and individuals have produced something like their
own CMS. A significant fraction of these have decided to market their CMS to
others, selling them or renting them as Application Service Providers (ASP),
and thus created a software marketplace like no other.
Since websites vary from
individual user weblogs (personal web publishing), through organizational news-oriented
portals and e-commerce sites, to massive enterprise portals that are the public
front end for a complex of intranet corporate portal sites (Microsoft has 8000
of these), we expect a wide range in the power, complexity, and cost of the
right CMS.
Any attempt to evaluate
a CMS must begin with understanding what it does. Even more important though,
is to understand what it is the website needs done. Hundreds of millions of
dollars have been spent in recent years on early versions of a CMS that did
not have the functionality needed. In-house developers have added their own
functions, and smart CMS developers have adapted many of these solutions back
into their general offerings. So we are seeing some convergence on the core
functionality of a CMS, whether for an enterprise or a small website.
The number of features associated
with a CMS continues to grow. For example, most of the major CMS vendors have
now made a significant commitment to the tools of, what some would call, "knowledge
management". Some of these tools are new, but most are venerable techniques
and terminology from print publishing, library science, linguistics, semiotics,
and even philosophy.
The time has come for a
serious effort to identify these core features and functions. If we are lucky,
we might even get some agreement from vendors on what to call them.
A number of industry analysts,
consultants, vendors, and users have joined us in a collaborative effort to
provide a public domain taxonomy and descriptive tool to aid in the understanding
of what a CMS is. "We", in this article, refers to this collaborative
group.
A CMS is What a CMS Does
Regular readers know we
prefer definitions that reflect reality and are practical, so a definition of
a CMS should not be based on what some think it should be. Rather, we
prefer to attempt a definition of a CMS by describing all the things people
actually are doing with one. And we can use the words they use to describe
what they are doing.
Nearly three years ago we
asked "What Is Content Management?" in the single most popular issue
of the Gilbane Report. Now we want to continue our attempt at an answer by starting
an industry-wide collaborative online effort to seek agreement on the core features
and functions of a CMS. We will look for agreement from analysts, from industry
experts in their books and articles, but especially from vendors and ultimately
from the users.
Investigating CMSs with
CMS Tools
We will present our research
in controlled vocabularies, faceted classification systems, topical searches,
etc., as a demonstration that uses Content Management tools to study
Content Management. We will do this online, we will do it collaboratively, and
we will do it with content management tools and techniques themselves. How?
First, we are building an
online database of features and functions, plus a list of evaluatory questions,
that vendors can access themselves with edit permission to enter quantitative
values and qualitative answers.
The database is designed
to be very flexible and extensible, allowing feature names and functions to
be changed, definitions to be revised, and room for every vendor to enter their
own qualified description when the generic feature or function does not fit
their product. The inherent ability for CMS version control will allow the work
to evolve with the moving target both vendors and users are aiming at.
Second, we think only the
widest possible collaboration is likely to get the necessary support from the
vendors. If the resulting research is publicly available, not just on our website
but on any collaborating CMS-related website that provides advice on CMSs, we
expect the vendors will keep their data accurate and up to date. They will know
that they enter data once and it comes out in many places on the web.
Distributing the knowledge
base will take advantage of syndication technology, similar to news feeds, but
designed to allow complete repository replication if desired, so multiple institutions
can have full copies of the data. We will publish syndicated "knowledge
feeds," which allow collaborating websites to provide straight lists of
CMSs, sophisticated searches for CMSs with a particular set of features and
functions, a faceted classification directory of CMSs, and a unique feature
comparator that lines up the features and functions of two CMSs side by side
to facilitate user decisions.
These knowledge feeds are
a special web service of the CMS industry, by the industry, and for the industry.
We will encourage the major
vendor-neutral CMS consultants, and even smaller vendor-specific consultants,
to publish these interactive knowledge feeds on their websites to educate their
clients. The question that usually comes right after "What is a CMS?"
is "Which CMS is the right one?" Knowledge retrieval through a faceted
classification scheme will allow users to put their most important CMS facets
(characteristics or properties) at the top of their custom-built hierarchies.
This should help bring the "right one" to the top of their list.
Syndicating CMS Knowledge
Services in a Wide Variety of Formats
We will export our CMS data
in unstructured HTML pages, as well as structured XML, RDF, OPML, XFML, and
possibly XTM Topic Maps. We will do this to demonstrate the kinds of publishing
possible with a CMS today, and to feed structures that will be maximally useful
to those who want to post-process them into a more useful display format for
their own needs. If web services are to succeed, we should demonstrate their
success as an industry analytic tool.
The XML will be our
own special markup language for describing Content Management Systems called
CMSML. CMSML is a joint development of the OSCOM softwareML project, the University
of Washington iSchool CMS Evaluation Lab, and CMS Review. It is described at
www.cmsml.org. The mailing list is softwareML@oscom.org.
The RDF structure
will enable inference engines to crawl the available CMS and draw conclusions
about them. People may develop semantic web inferencing programs to help choose
the right CMS from the many available. This is a major commitment to permanent
URIs that describe CMS Resources.
OPML is Dave Winer's
Outline Processor Markup Language. An organization can prepare a specific set
of facets ordered by their priorities, and then create an OPML document that
captures their choices, for further study and critical annotation in OPML readers
and browsers. See www.opml.org.
XFML is Peter van
Dijck's eXtendable Faceted Metadata Language, and it will transport the entire
knowledge base in a file that can be browsed in tools like those from Travis
Wilson's www.FacetMap.com. See
www.xfml.org.
Although in many respects
Topic Maps are isomorphic functionally to RDF Resources and both can implement
the model of a semantic web, there are probably advantages to the CMS industry
in a set of published Topic Maps that establish the core concepts and terms
as well as the functions and features of content management systems. Others
can then build their own Topic Maps that associate with these CMS Topics.
The Draft Feature and Function
List
To develop our initial list
we have looked at the best-known books and reports on the CMS business. Here
is a bibliography:
- Content Critical,
by Gerry McGovern
- Content Management
Bible, by Bob
Boiko
- Content Management
for Dynamic Web Delivery, by JoAnn Hackos
- Content Management
Systems - Tools of the Trade, by Dave Addey, James Ellis, Phil Suh, David
Thiemecke
- Managing Enterprise
Content, by Anne Rockley
- Web Content Management,
by Russell Nakano
- CMS Report, by
Tony Byrne, CMS Watch
- CM Requirements Toolkit,
by James Robertson, Step Two Designs
- The Gilbane Report
Our list is divided into
five major areas.
- A Product Overview
where we collect general information about the product (Description, Status,
Marketing, Installation, and Support).
Then we look at four main
phases of Content Management:
- Content
Creation (Acquisition, Aggregation, Authoring)
- Content Management
Proper (Workflow, Editing, Approvals, Staging, Repository, etc.)
- Content Delivery
(Live Server, Publishing, Syndication)
- Content Lifecycle
Enhancements (Security, Business rules, Integration, Metadata management,
Transformations, Associations, Analysis, Search and Locate)
The current proposed feature
and function list is attached in Appendix A.
Notes on the Comparator
Tool
How Some Features Become
Facets
When a feature or function
can be described only with a text entry, it has limited use in a faceted classification
scheme. However, when the possible values of a feature can be enumerated from
a list, this feature becomes a facet that can be used to sort the CMS in different
ways.
One such facet is the computer
operating system. Another is the programming language. When deciding on a CMS,
arranging systems in a hierarchy that puts operating system and programming
language in the top layers aggregate systems that will suit different IT strategies.
Figure 1 shows an example
of choosing a platform (OS), a web server, and a programming language in the
Faceted CMS Directory. The list at the right now shows the CMS Frameworks
(our fourth facet choice) available.
Figure 1.
Figure 2 is an example of choosing a platform (OS), a web server, and a programming
language. The list at the right now shows the CMS products available.
Figure 2.
Figure 3 is the result of choosing two CMS for a side-by-side comparison in
the Feature Comparator
Figure 3.
Cautionary Notes on
CMS Evaluation
Some clever programmer may
take a knowledge service feed and try to generate a rating for various CMSs
based on our data. We encourage serious users to regard these data as a starting
point for their own CMS evaluation and selection process. Work with an established
CMS consultant or your own in-house IT staff who knows how to use the data to
create the right fit for your CMS needs. Tools such as these cannot automatically
choose "the right" CMS.
Vendor marketing teams will
naturally try to use any classification scheme to make their products look better.
Our approach to problem is to support vendor-qualified descriptions for each
feature. This will encourage a clear statement of what the vendor means in the
particular case. Also, any particular feature may be open to (mis)interpretation,
either deliberate or accidental. Different vendors may genuinely mean different
things by the same term. These are both further reasons to work with a qualified
internal or external consultant.
Note that we are making
no effort whatever to create new terminology. We firmly believe that "meaning
is use," and we are trying to discover that use at the same time we try
to discover what CMSs have in common to help understand and define them. We
think we can help by publishing a glossary of terms - a controlled vocabulary
for the CMS industry - that is used to describe the core features. We want to
get everyone at least familiar with industry jargon. This will increase the
chances that they will converge on shared concepts. Formally we describe this
as building a thesaurus of terms. As in ordinary language, a thesaurus can have
rich semantic relationships, with many terms substituting for one another, and
single words having different meanings in different contexts. We have the technology
to identify preferred terms and common variants.
How to access and use CMS
Knowledge Services
For Users of the Services
Visit one of the collaborating
CMS-related sites who are supporting the development version of the knowledgebase
and tools:
For CMS-related sites
that would like to add CMS Knowledge Services for their site visitors
For CMS vendors that
would like to edit their feature data
- Go to www.CMSReview.com
- Login without a user
name and password. You will be invited to join as an individual or organization
and you will be sent a password. You will be given edit privileges by the
administrator.
- Click on the Edit links
next to your product name, log in, and edit your data. Further help is available
at http://www.cmsreview.com/HowTo/ForVendors.html
For all interested who
would like to criticize the feature list
Please Join Us!
The knowledge services tools
described here are in development. We need the participation of the CMS vendor
community to fill in their data. We need them and we need CMS users to help
us refine the most meaningful features and functions to be included in the master
list.
A progress report on this
project will be delivered at the Gilbane Conference on Content Management
@ Seybold in San Francisco, September 9-10, 2003.
Bob Doyle
bobdoyle@skybuilders.com, with
Gregor Rothfuss gregor@apache.org, and
Frank Gilbane frank@gilbane.com
Appendix A:
Proposed CMS Taxonomy
This is a work
in progress. Please comment, suggest, critique, or congratulate. editor@cmsreview.com.
1. Product
Overview
- Description
- Product Name
- Company Name
- Company/Organization
website
- Product web page
- Company's description
- Our Description
- Technology
- License - Open-source,
Proprietary, which
- Type - General CMS,
Framework, Front end (UI), News Portal, Blog, Wiki
- Platform - Windows,
Linux, Mac, etc.
- Web Server - IIS,
Apache, etc.
- Application Framework
- Perl, Python, .NET, J2EE, PHP, Cold Fusion, etc.
- CMS Framework - AxKit,
Cocoon, Midgard, Zope, etc.
- Languages - Perl,
VB, Java, PHP, Python, etc.
- Databases - Oracle,
SQL Server, MySQL, PostgreSQL, any ODBC, etc.
- API - public to allow
extensibility
- Status
- Release - 2.0, etc.
- Year introduced
- Number of Installs,
Downloads
- Developer Community
(website?, mail list?)
- Marketing
- Price
- License (per CPU,
per user, etc.)
- Market Position (Revenues,
Competitors)
- Sales Methods (Sales
Force, Online)
- Support Contracts,
Consultants
- Online Demos, Sandbox,
Prototype, Trial
- Installation
- Online How To
- Hours/Days for Typical
Install
- Documentation online/printed
- Download site/CD-ROMs
- Code Commented
- Support
- Online Help
- Tutorials
- Training Classes
- Cost
- Commercial Contracts
- Help Desks
- Independent Consultants
2. Content Creation (Acquisition,
Aggregation, Authoring)
- Acquisition
- Native support for
filetypes
- Multiple file transfers
(FTP, site import)
- Conversion tools
(e.g.,Word to XML "chunks")
- Rights management
- Mandatory tagging
(force structure and semantics)
- Supported RDF ontologies
(e.g., Dublin Core)
- Aggregation
- Incoming syndicated
feeds
- Metadata management
(read incoming metadata)
- Integrated Web services
(e.g., currency conversion)
- UDDI tools
- Authoring (Editing Tools,
Templating, Tagging)
- Content Element Editors
(Naive and Power Users)
- WYSIWYG Through-The-Web
- Text-only Forms
- XML Editor
- Spell checker
- Content objects
use templates
- Asset repository
(images, sounds, Flash, video, etc.)
- Template Editor
- WYSIWYG Through-The-Web
- Template Gallery
- XML Editor
- Tag Editor (semantics
and style)
- Drop-down menus
of all tags
- Metadata Thesaurus
- Taxonomies/Ontologies
online
- Help online
- Context-sensitive
help
- Documentation
- Example
3. Content Management Proper
(Workflow, Editing, Approvals, Staging, Repository, etc.)
- Workflow
- Access Permission
Levels (Privilege granularity)
- Number of levels
- Per User, Per
Folder, Per Role, Per Item
- Flexible assignments
to workflow
- Creator automatic
owner
- User subscription
to workflow
- Check In/Check Out
- Open page on
web (Edit this page)
- Automatic file
lock on open
- Conflict Resolution
(who has it?)
- Instant Messaging
(email, phones)
- Merge Tools,
Diffs
- Workflow Messaging
- Email notifications
(links to work)
- Status (stage
in workflow)
- Comments at each
stage
- Audit trail (workflow
log)
- Arbitrary Roles (Writers,
Editors, Graphic Artists, Rights Managers, Publishers, etc.)
- Versioning
- Scheduling, Expiration
- All elements, templates
date/time stamped
- Archive with rollback
(per file or site?)
- Personalization
- Relationship Management
(History)
- Actions tracking
- Session/Click/Behavior
analysis
- Individual visitor
ID (cookies)
- Localization
- Multilingual server
- Respond to browser
language requests
- Gist translation
option
- Workflow
- Automatic notifications
- Quality checkers
- UI multilingual
- Reporting
- Chrono workflow and
by worker
- WebTrends-style for
whole site
- Specific monitors
- Performance (page
delivery times)
- Storage
- Format (text, HTML,
XML)
- Database only
- Files
- Files and database
- Backup
- Onsite and offsite
- Files and database
- To nonvolatile media
- Disaster recovery
plan
- Security
- Firewall rules
- Encrypted sessions
- Staging Server for QA
- Testing methodology
- Replicates publishing
environment
4. Content Delivery (Live
Server, Publishing, Syndication)
- Publishing (Delivery)
- Separate Delivery
from Creation/Staging/Testing
- Use different
server platform?
- Replication
- Synchronization
of mirror sites
- Multi-Publishing
to different clients
- HTML, XHTML,
XML
- PDF
- PDAs
- Cell phones
- Handicap accessibility
- Syndication
- RDF Syndicated News
Feeds
- Web services
- Information feeds
5. Lifecycle Enhancements
(Apply to all three stages above)
- Security
- Audit trails
- Users
- System
- Network
- Business rules
- Records policies
- Privacy policies
- Integration
- Single authentication
- Enterprise portal
- Legacy database reuse
- Data warehousing
- Metadata management
- Digital rights management
- Digital rules management
- Transformations
- Associations
- Hierarchy, taxonomy
- Index
- Cross reference
- Analysis
- Analytic Tools
- Pattern recognition
- Search and Locate
|
