"Count what is countable. Measure what is measureable. What is not measureable, make measureable." -- Galileo

Wednesday, April 1, 2009

Idealware CMS Report

No matter what fault you may find with Idealware's recent "Comparing Open Source Content Management Systems," we have a rare jewel in the world of web comparative analysis: a report with a completely documented methodology. Of course, you are welcome to disagree with their ratings, their categories, and their thresholds, but at least they are 100% up front about where the numbers come from. Tip o' the hat to Idealware.

So... down to the nitty gritty. What does the report have to say? The easiest thing to do is take their verbal scores (None, Fair, Solid, Excellent) and convert them to numeric values (0, 1, 2, 3 respectively). The result is a modified version of their comparison chart on page 16 (numbered page 14 due to front matter).
















Category
WordPress Joomla Drupal Plone
Hosting/Installation
3331
Simple Site Config
3222
Complex Site Config
3321
Content Admin
3223
Graphical Flexibility
3333
Structural Flexibility
1233
Roles and Workflow
1123
Web 2.0
2232
Extending and Integrating
2233
Scalability/Security
2223
Maintenance
3222
Support/Community
3333


29273029
The long and the short of it is that there is only a 10% difference between the bottom (Joomla) and the top (Drupal). WordPress and Plone are only 3% behind the front runner. I'd say that the differences are "in the noise."

Perhaps a better way to approach this is to weigh those comparison categories to match your needs and requirements, run the math, and see what comes out. For example, if security is a big driver (as it is for me), giving it a weighing factor of 2 might be appropriate. Then the final scores are 31, 29, 32, and 33.

But as is so often the case, we're looking at apples and oranges here from the get-go. WordPress is a blogging/collaboration tool, Joomla is a web CMS, Drupal is a complex mixture of both. Plone by virtue of its powerful combination of features is a collaboration system and a web content management system, yet strong enough to do the heavy lifting for an enterprise portal.

Its always nice to see good scores and the fact that Idealware chose Plone along with only 3 other serious open source systems is high praise in and of itself. Idealware should be complimented for having a transparent methodology, a relatively neutral approach, and giving good press to four worthwhile systems.

Still, I can't stress enough that your specific, unique requirements must drive your choice for a CMS. Don't let someone else's numbers blind you to what you and your community of users need to accomplish.

Coming next: a stroll through Idealware's Consultants listing in the appendix.

3 comments:

Laura S. Quinn said...

Great to hear you found the report useful! I'm Laura Quinn, one of the authors of the report.

And yeah, you've actually just summed up the reason that we don't provide the ratings in numerical form. There's no "winner" of a report like this - what would that mean? If you need to create a 15 page campaign site by tomorrow, and you don't have experience with any of these tools, Drupal would be a silly choice, even if it has the highest score. So as you say, your needs always need to be the key driver.

Schlepp said...

I appreciate your comment and also your remarks over at Four Kitchens.

Lennart Regebro said...

What Plone needs to be on top is a simple theming story. Plone doesn't have that, because Plone is typically used by companies and organisations that want their own custom design anyway. So it's probably a waste of effort to make it. But comparing the Easy of setup for a simple site shows that Plone gets only a one, because it's hard to install prepackaged themes.

That and hosting is the only thing Plone get's a 1 on. And the hosting we can't do anything about, because Plone isn't PHP.