CMS Matrix, Part Three

Every now and then I've poked around at the CMS Matrix numbers and found them interesting. I first did this last January and then again in June. Seems like I'm on a bi-annual cycle, so I'll open this year's metrics with another look at the stats there. There are over 1000 CMSs listed with CMS Matrix, so obviously content management is still a hugely important application arena. Equally important, thousands of people turn to CMS Matrix to get a quick take on the relative strengths and weaknesses of the many products listed there.

Just glancing at the Plone page shows things to be very well off. Plone answers almost all desired characteristic with either an affirmative or with a free add-on available. The weak spots where Plone sports a "No" are:

• SSL pages (Security)
  
• Certification Program (Support)
  
• Site Setup Wizard (Ease of Use)
  
• Matrix Application (Built-in Applications)
  
• Web Service Frontend (Built-in Applications)
• Affiliate Tracking (Commerce)
  
• Wish List Capability (Commerce)
  

CMS Matrix groups each of its characteristic features into one of nine major functional areas and those are shown in parentheses above. Of these, then, here are the number of features in each functional area, the number of features Plone is "lacking," and that lack as a percent of total features in each functional area.

Functional Area	Number of Features	Features "Lacking" in Plone	Percent "Lacking"
Security	19	1	5%
Support	15	1	7%
Ease of Use	17	1	6%
Performance	15	0	0%
Management	7	0	0%
Interoperability	9	0	0%
Flexibility	5	0	0%
Built-in Applications	40	2	5%
Commerce	9	2	22%

Of course, Plone would probably like to be able to answer affirmatively to all of CMS Matrix's features (and I know many are the subject of current development), its clear that the Commerce category takes the biggest percentage hit. Then again, its one of the smallest categories with only 9 features.

On the other hand, support and usability issues probably have a bigger impact on all users, whereas commerce is perforce a smaller subset of users. I personally am looking forward to improvements in site setup and the possibilities for rapid and easy installation in cloud buildouts. I also think that a certification program may be a useful thing for the Foundation to look into.

Now back to features per functional area, let's compare Plone with some other top-of-the-line CMS. I've chosen the ten CMS who have Packt titles that are also listed in CMS Matrix. Then I've run the "compare" feature to generate a huge feature comparison table. Finally, I've counted the number of feature per functional area that are listed as not available, of limited availability, or only as an extra cost item. (The italics mean nothing; they are there only to help visually separate the columns. I've split the table in half due to width limitations in my CSS.)

Number of features "lacking" ('No', 'Limited', or 'Costs Extra')
	Alfresco	Drupal	e107	eZ Publish	Joomla!	Mambo
Security	6	7	7	3	8	9
Support	1	1	7	0	3	3
Ease of Use	4	7	7	4	7	8
Performance	1	2	2	0	2	3
Management	3	4	6	1	4	3
Interoperability	1	3	4	2	4	3
Flexibility	1	1	2	0	0	2
Built-in Applications	28	8	12	10	6	9
Commerce	4	1	8	3	0	1
Total	49	34	55	23	34	41

Number of features "lacking" ('No', 'Limited', or 'Costs Extra')
	OpenEdit Cms	Plone	TYPO3	WordPress
Security	4	1	0	8
Support	3	1	0	6
Ease of Use	2	1	0	3
Performance	0	0	0	2
Management	2	0	0	9
Interoperability	1	0	0	2
Flexibility	1	0	0	4
Built-in Applications	11	2	1	14
Commerce	3	2	2	9
Total	27	7	3	57

Only TYPO3 surpasses Plone and both of these are far ahead of the others with eZ Publish running a distant third.

Continuing with our plumbing of CMS Matrix depths, there's a handy little table on their home page showing which CMS are best rated by users.

Best Rated By Users
System Requirements	7.02/10	Jaws
Security	6.72/10	Jaws
Support	6.54/10	MySource Matrix
Ease of Use	7.18/10	Jaws
Performance	6.60/10	e107
Management	6.62/10	Jaws
Interoperability	6.44/10	eZ Publish
Flexibility	7.00/10	Plone
Built-in Applications	6.54/10	Xaraya
Commerce	6.88/10	OpenEdit CMS, eCommerce, Digital Asset Management

Of these CMS, Jaws appears to be a newcomer to the CMS scene (first beta only in May 2008) yet it has over 800 evaluation responses. MySource Matrix has been around since 2004 and is making points down under in Austalia. e107 seems reasonably mature and stable. Both these have more than 100 evaluations making up their scores. eZ Publish has about 80 evals while Xaraya has less than 40. Similarly, OpenEdit etc. has about 40 reviews with the disadvantage that many important add-ons are at cost. Plone BTW has 113 evaluations at CMS Matrix--feel free to hop over and add to the score.

Plone appears to be in the senior bracket of these "Best Rated" systems in terms of years of service and current version. That not withstanding, it has captured top honors for Flexibility. I'll have to do some digging to find out what's going on with Jaws and its highly elevated position in so many slots. Given its relative immaturity compared to most of the others, I'm surprised is has many hundreds more reviews. The suspicious mind might think that someone is stuffing the ballot box.

That aside, what can we conclude? If we want to "game" CMS Matrix, a high priority should be given to filling in the blanks in their feature lists. However, letting someone else's list of CMS features drive development seems like a bad idea in principle.

My personal favorite is to setup a certification program for Plone consultants, trainers, and companies. This seems like a natural area where the Plone Foundation could set standards and provide objective, professional certification. For someone to sport a "Certified Plone Professional" badge or a company to have a "Certified Plone Provider" rating would assure prospective clients that consultants or companies have guaranteed minimum qualifications.

I realize that the topic of certification is a touchy one, especially within an open source project, where individual initiative is highly valued, so I welcome comment here or over on the boards.