"Count what is countable. Measure what is measureable. What is not measureable, make measureable." -- Galileo

Friday, July 1, 2011

CMS Matrix Revisited

Every now and then I wander over to CMSmatrix.org and check things out.  Systems come, systems go, and for a handful, there's a steady maturation and growth.  Tonight I took the top ten CMSs as rated by "Comparisions."  Must mean something if people are usually looking at the same set of systems.

I then counted the CMS Matrix features that they lacked, as defined by a feature having a "No" or "Limited" or "Costs Extra" in a particular cell.  One minus that value divided by the total number of features is given in the Pct Features column below.  The "System Requirements" section was not tallied due to it's largely text content.

CMS Pct Features
TYPO3 4.5.2 97%
Plone 4.0 96%
WebGUI 7.9 89%
eZ Publish 4.2 83%
Joomla 1.6.0 82%
Drupal 6.10 77%
Xoops 2.0.18 71%
Mambo 4.6.1 71%
WordPress 3.0.4 63%
PHP Nuke 6 15%

If you prefer the graphic, here's the same data set.

It's interesting to see that among top-compared CMSs in CMS Matrix, there's a surprising amount of variability.  TYPO3 and Plone top the chart above 90% and the rest trail away down to 63%.  Then PHP Nuke languishes at the bottom.

Perhaps tomorrow I'll dig out my copy of 'R' and see if these cluster in any meaningful way.  I have the information summarized by each CMS Matrix category (Security, Support, Ease of Use, Performance, Management, Interoperability, Flexibility, Built-in Applications, and Commerce), so I'll be able to look at each section by itself in the days ahead.  But one word of warning before you base a decision on these data, check the specific features you require against any candidate system.  No need to throw out a possibility, if the reason is the lack of a feature you'll never use. 


Dylan Jay said...

Just had a look at the feature comparisons between wordpress and plone. It doesn't look like either is very very accurate in the way those features lists are filled out. .e.g. no on multisite of WP?
So I think this particular statistic isn't worth much and may do Plone more harm than good.
I guess thats a cmsmatrix failing however.

Schlepp said...

It's true that since CMS Matrix info is supplied by volunteers almost certainly involved in a particular CMS, there can be biases and wildly uneven quality in the data. Not all terms and features are clearly defined.

That said, there must be some self-policing -- saying you have a feature that's not present could easily backfire in a viral way. Last I heard, Geir was taking care of Plone updates to CMS Matrix.