First, the methodology. Candidate CMS's are the top ten listed in CMS Matrix when sorted by compares (BTW, Plone comes in sixth). The source for my security data is the National CVE and CCE Vulnerability Database, searching each candidate CMS for vulnerabilities listed in the past 3 years and past 3 months. The highest level of vulnerability in the past 3 months were counted and totaled as a "Seriousness" score. Here they are in alphabetical order.
| CMS | 3 yr. vuln. | 3 mo. vuln. | Seriousness |
| DotNetNuke | 13 | 0 | 0 |
| Drupal | 266 | 16 | 6 medium |
| Joomla | 426 | 87 | 36 high |
| Mambo | 85 | 0 | 0 |
| PHP Nuke | 6 | 0 | 0 |
| Plone | 8 | 0 | 0 |
| TYPO3 | 224 | 44 | 26 high |
| WebGUI | 9 | 0 | 0 |
| WordPress | 141 | 6 | 4 high |
| Xoops | 65 | 3 | 1 high |
Second, the results. I'm pleasantly surprised that 5 out of 10 turned up with no vulnerabilities in the past 3 months. However, of those only PHP Nuke has a better 3-year number of vulnerabilities. TYPO3 and Joomla have a much larger number of vulnerabilities rated "high" by the National Vulnerability Database than all the rest combined.
Typically one hears defenses like "CMS x has more vulnerabilities reported because it is more popular/has a larger install base and therefore has more eyes looking for problems." That's the same argument Microsoft put forward for all its security flaws when compared with Linux. You don't want to go there.
But even putting aside the logical fallacies in such arguments, both pro and con, it's clear that systems with more security flaws require more effort to patch and lock down. The bottom line is that more flaws mean more chances for the sys admin to slip up and more opportunities for the bad guys.
By way of an anecdote, at my day-job our computer security people were much relieved to learn that we were using a non-PHP-based CMS. On that note, Happy World Plone Day 2010.
7 comments:
Excellent post! I always make a point to bring up Plone's security record (not typically bringing up the poor records of the other tools, though.)
I think that the inherent NoSQL model that Plone uses out-of-the-box plays a big part in its superior security - not SQL-injection techniques allowed, and that's one of the most common vulnerabilities of all web apps and especially PHP ones it seems.
It also helps that Plone and Zope do security checks on ever item on the page (links to other pages that you shouldn't be able to access are hidden, such that, for example, your search results differ from another user with different permissions/roles.)
This security argument for Plone and the comparison to how other tools are doing does need to be kept up each year, though, so a sincere thanks for going through the effort to update the current state of affairs.
For Drupal's vulnerabilities: you must have looked at *all* vulnerabilities, that is: Drupal core + its >5000 contributed modules.
Because Drupal core has only had one security advisory (with a corresponding release that fixes the bug) in 2010 so far …
Get your facts straight first ;)
Source: http://drupal.org/security
Thanks for your comment, Wim. Like I said, this is without a doubt the most contentious subject I touch on in this blog.
The methodology I documented gives the results stated in the table as of 27 April 2010. Enter "Drupal" in the NVD search box and that's what you get. The facts are straight, in so far as the NVD reports them.
The NVD is including contributed modules for all the other systems as well. Since Plone sports 3639 products, it should be facing the same magnitude of handicap in the NVD. The other CMS's are in the same boat.
Other authors who have discussed this subject in the past have pointed out that core products are rarely used alone, but what constitutes a fair selection of add-ons for any particular system? I have relied on a specified, explicit, and repeatable methodology to arrive at comparable numbers.
I may in some future post extract the numbers for just the core CMS's, but given the length of some listings, this will have to wait until I have more time.
One final note, you correctly point out that a Drupal bug fix is in place. For many, many of the NVD vulnerabilities, the CMS developers involved have responded aggressively to deal with them. I do not want to impute that any CMS community mentioned here is taking security lightly or being cavalier about this matter.
Ironically, the *vast* majority of those security reports for Drupal were on contributed modules that did not use the core Drupal API.
Plone runs on top of an extraordinary application platform: Zope 2.x. Zope provides a security safety net for Plone add-ons as well as Plone.
Plone doesn't deserve credit for Zope's great security as an app server. And, Drupal (per se) doesn't deserve the blame for PHP's poor record.
But, it is reasonable when evaluating a CMS to consider the degree to which you will need add-ons, and whether or not those add-ons will be operating on an app-server platform with very little in the way of a safety net.
It's also worth noting that Plone is a very feature complete CMS out-of-the-box. Drupal has, by comparison, followed a micro-core strategy. In some ways it's more of a CMS-base platform than a complete CMS. My point is that evaluations need to be done on the basis of the code quality of what you'll actually use: including add-ons required to meet your need.
By the way, I think that the Drupal core folks deserve huge props for communicating good coding standards to add-on developers in their most current release. In some ways, the story here isn't how good Zope/Plone is on security. It's always been excellent. What's news is how much better the Drupal ecosystem is doing than it used to, and how much better they've gotten than the rest of the PHP world. If you absolutely must use PHP for a web CMS, and if (despite that requirement) security remains a concern, Drupal is a much better bet than other PHP solutions.
@Schlepp: My apologies, that wasn't clear from the blog post. Maybe you should clarify that?
In that case: impressive! :)
Just one more note: while you need to be approved to be allowed to contribute modules to Drupal.org, not every contributor is very proficient at writing code, let alone security. I'm trying to say: there are lots and lots of crappy modules maintained by crappy developers on drupal.org. Does the same apply to Plone? Just asking out of curiosity.
And FWIW, IMO all CMSes and web frameworks suck. I haven't seen a single one that is 100% awesome. Drupal sucks. Joomla sucks more. Wordpress sucks. Django sucks. And so on. The only API I've seen so far that is 100% awesome, is Qt. And that's for desktop app development only.
P.S.: get rid of blogger.com. It stinks. Commenting is a pain and is ridiculous: I don't *want* a blogger profile, I want to link to my own website: http://wimleers.com/.
I agree, commenting is a pain. I had to switch to moderated commenting after I started getting routine "Nice post" comments with hidden links to spam. Now after almost 3 yrs and 200 postings, I'm too invested (and lazy) to move.
Re: Quality of 3rd-party modules. It's definitely a case of some being better than others. Occasionally I hear talk about a product QA effort or at least a "thumbs up/thumbs down" system. Don't know what the status of any of that might be right now.
That's one of the fundamental questions for the Web these days: how can you distinguish between and trust different information sources?
Post a Comment