"Count what is countable. Measure what is measureable. What is not measureable, make measureable." -- Galileo

Showing posts with label CMS comparisons. Show all posts
Showing posts with label CMS comparisons. Show all posts

Wednesday, July 4, 2012

Confirmation Bias in Software Comparisons

I'm definitely guilty of confirmation bias... frankly, it's part of my job to justify the software tools that my team uses.  But I'm not a paid Plone consultant, so I don't have a conflict of fiduciary interest.  This blog is a voluntary effort completely independent of the Plone Foundation. Full disclosure:  my web team at my day job implements and manages Plone sites. 

What Paul Kaiser is doing, since he makes a living off of the WordPress ecosystem, is criticizing a system with an eye towards bolstering his own line of work.  Since he does make a living with WP, he has a genuine conflict of interest.  To his credit, he makes full disclosure of this.

His recent piece about Plone vs WP has garnered thoughtful and measured responses to clarify some of the statements.  Plonistas David Glick, Alex Clark, Eric Brehault, David Bain, and Matt Hamilton have quickly stepped in to provide the Plone point of view.  Paul's responses have been equally civil, so here's a tip of the hat to polite, open discourse, sharing of ideas, and no flame wars. 

Here are my thoughts, taking Paul's points one by one. 

Straight away Kaiser gives a nod to Blender, PyGame and Trac.  Django, Pyramid, TurboGears, and other top-flight Python frameworks are left off the table.  In the comparison of underlying languages, take a look at http://time2hack.blogspot.com/2012/01/php-vs-ruby-vs-python.html#axzz1zfOHeTNY for a recent set of stats re:  Python, Perl, PHP, Ruby, etc.

But then this is a "WP instead of Plone" article.  Move along, nothing here....

Plone != Harder;

Matt Hamilton's reply covers the misconception that Plone is harder to install, maintain, and develop.  When I was teaching at the College of Santa Fe, we'd install Plone, build a UML model, run it through ArchGenXML, and have a custom archetype all in a single evening.  It probably comes down to the background and experience of the implementer.  More about that down below under human factors. 

Regarding the other points in this section:

1.  "Fewer commercial web hosts support..."  Physical proximity is not the be-all and end-all that it once was.  With cloud-based deployments, this will be less so in the future.  If you control your own servers, it's not nearly such an issue.  We recently lost our long-time sys admin, but a new fellow with no Plone/Zope experience picked it right up.  With his CENTOS background and the clear Plone 4.1 documentation, he was able to build our new cluster without supervision by me. 

2.  "...PHP, MySQL, and WP [programmers] are far easier to locate..."  This opens the entire out-sourcing can of worms.  Take a look at the discussion at http://forums.whirlpool.net.au/archive/1772689, for example, or this interesting anecdote (http://www.electoral-vote.com/evp2012/Pres/Maps/May31.html#item-2). 

All those Plone advantages

Plone _is_ more secure.  David Glick's comment is absolutely on target -- Plone has an order of magnitude fewer vulnerabilities.  This doesn't mean that WP is less secure, it just means you must spend lots more time patching your installation.  It also means that there's more of an opportunity to miss something. 

Plone is faster.  Again David's comment is a good one.  Both WP and Plone can handle loads if configured correctly.  That said, these kinds of stories seem to be a staple of Plone:
Faster and more scalable.  No, using the ZODB isn't cheating, it's good design.  Systems running in front of MySQL just have to live with all that comes with it:  slower out-of-the-box speed, less security. 

Document Management Functions

I manage websites for my neighborhood association and the Albuquerque Bonsai Club, both WP sites.  I shudder to think about scaling these up to handle thousands of documents.  I have Plone sites with tens of thousands of pdf's, LDAP integration, complex workflows, formal review and approval processes, versioning, and multiple languages to boot.  Despite what Kaiser says about WP ease of use, I find Plone far easier to deal with than the two WP blogs I manage.

Customer experience and background count for a lot when you must satisfy a client.  In the end it comes down to listening to your customer and matching requirements to systems.  WP was designed to be a blogging system and that architectural history burdens it's ability to easily and efficiently handle document management use cases. 

Search.  David Bain's comment pretty much covers this.  Nuff said. 

Dealmaker:  Human Factors

Yes, people use the tools they are familiar with.  Typically that means an organization has evolved to have staff skillsets with flexibility and extensibility to handle their current and plausibly foreseen use cases. 

Existing skills and installed systems are themselves aspects of requirements that need to be assessed as constraints.  I won't hire a Ruby developer for a one-of-a-kind project even if using in-house Python talent is not most cost effective; it is an optimum use of HR resources.  The overhead of hiring is an indirect cost that would need to be considered and I'd be left with a under-utilized Ruby developer afterwards.  If you have outsourcing and consulting options available to your client, then that's another situation. The entire system within which software is to be recommended has to be considered by the analyst. 

Weakness in complexity.  From my background in ecology, complexity actually generates stability.  I'm seeing that in software systems as well, depending on how you define "complexity."  And if a four-layer technology stack is too much to handle, you'd better stay away from the likes of these listed here (http://www.tutkiun.com/2011/07/web-technology-stack-analysis.html).  Complex technology stacks, especially in the open-source arena, are generating co-dependencies that actually strengthen them in what mirrors co-evolutionary adaptations between biological species.

If the final analysis hinges on the human side of the equation, one only has to participate in a Plone sprint or conference (not to mention other communities involved with Zope and Python) to see that Plone wins this hands down.  In researching Plone and WP conferences this evening, I was astonished to learn that WP's first international conference was only held in 2011.  Plone Conferences have been alternating between Europe and North America since 2003 (New Orleans, followed by Vienna 2004). 

In conclusion...

I'd like to go back to Paul's initial paragraph:
"Plone is a web-based content management system built on Python, sharing many similarities with WordPress. As a WordPress developer, you may from time-to-time find clients leaning towards Plone. Learn the important differences between WordPress and Plone, and you’ll be better prepared to help such clients."
I'd take issue with the "sharing many similarities" part.  Although the Real Story Group's CMS Subway Map considers Plone and WP to be on the "Web Content Management" and "Collaboration & Social Software" lines, Plone is one of the few CMS's on those two lines that includes the "Portals & Content Integration" line.  RSG places WP far, far from Plone, and also far from Drupal and Hannon Hill, WP's colleagues on the red and blue lines. Alex Clark's comment on feature-for-feature comparisons are very relevant here, as well. 

CMS Matrix shows Plone to rank higher than WP when ordered by clicks (10:33) and views (6:13).  That's indicative of WP being the choice for many people who select it without due diligence because of its low- or no-cost hosting and ability to meet simple use cases.  Plone, by contrast, is considered by those making thoughtful, feature-by-feature assessments to meet stated requirements. 

I'd also take issue with Mr. Kaiser's insinuation that "leaning towards Plone" is a bad thing and that "to help such clients" implies choosing WP instead of Plone.  While the words appear neutral, the general tone of the article is Plone- and Python- negative.  Python is only described derogatorily as "sexy."  Statements like "hard to imagine why anyone would even consider using Plone" slide in while listing only Drupal, Joomla!, and WP as "professional tools."  Disparaging Plone as a "little content management system" is not being cute, it's a euphemism that intends to give positive appearances to negative, misleading statements. 

Paul's not-so-hidden agenda is clear in the paragraph:
"A few issues I discovered almost held water for Plone. Here’s the information you need to knock them down."
 It's obvious that Mr. Kaiser has a case of confirmation bias and is trying very hard to spin everything toward his business of WP.  To their credit, members of the Plone community have called him on his misrepresentations, misstatements, factual errors, and logical fallacies.  Alas, the comments (as of 7/4/12) don't have anyone from the WP community besides the author jumping in to the discussion.  To everyone's credit, the debate has been mutually respectful and polite.

In closing, 

I'll stick to my mantra that all information system decisions need to be well thought-out based on requirements, requirements, requirements.  Those requirements need to be considered at the broadest level taking into account existing resources, human factors, future needs, and many other aspects of the problem to be solved and its environment.

That WP fits a set of very common use cases, is widely available for no- or low-cost hosting, and in its simplest forms can be managed by those with little or no CMS experience goes without saying.  But to say that WP is the "surprise winner" in a Plone vs WP throw-down is just not true.  Especially with regard to meeting requirements of security, workflow, indexing and search, usability, and document management in general, Plone excels. 

Thursday, December 9, 2010

IdealWare CMS Report

Once again IdealWare has come out with their 2010 annual review of Plone, WordPress, Drupal, and Joomla. As last year, they are one of the few software comparisons of any type to have a completely transparent methodology. Not only is the body of the report well balanced and thought out, the detailed system-by-system accounting of the subtopics that make up their category scores makes for very worthwhile reading. Tip o' the hat again.

Also like last year, I'm taking the liberty of translating IdealWare's scoring of  None, Fair, Solid, Excellent into 0, 1, 2, 3.  The result is the following table.


WordPress Joomla 1.6 Drupal Plone
Ease of Hosting and Installation 3 3 3 1
Ease of Setup: Simple Site 3 2 2 1
Ease of Setup: Complex Site 3 3 2 2
Ease of Use: Content Editors 3 2 2 3
Ease of Use: Site Administrator 3 2 2 2
Graphical Flexibility 3 3 3 3
Accessibility and Search Engine Optimization 2 3 1 3
Structural Flexiblity 2 2 3 3
User Roles and Workflow 1 3 2 3
Community/Web 2.0 Functionality 3 2 3 2
Extending and Integrating 3 3 3 3
Security 1 2 2 3
Support/Community Strength 3 3 3 3
2010 33 33 31 32
2009* 29 27 30 29
* In 2009 only 12 categories (instead of 13) were used.

Once again there is only a small spread in the scores, about 6%, well below the 10% of 2009.  All CMS's here are improving quickly.  Reviewers still hold Plone's non-LAMP hosting model against it, giving it a Fair for hosting and installation.  I beg to differ with them about setting up a simple site.  Once the software is installed, Plone's out-of-the-box feature set solves a wide range of CMS use-cases with a minimum of fuss. 

What I stated last year still stands:

"Its always nice to see good scores and the fact that Idealware chose Plone along with only 3 other serious open source systems is high praise in and of itself. Idealware should be complimented for having a transparent methodology, a relatively neutral approach, and giving good press to four worthwhile systems.

"Still, I can't stress enough that your specific, unique requirements must drive your choice for a CMS. Don't let someone else's numbers blind you to what you and your community of users need to accomplish."
I urge those shopping for a CMS to weigh the scores according to your needs and requirements.  If ease of use for content contributors, SEO excellence, and security are key features, Plone clearly comes out ahead.  Go ahead and cut-and-paste my table into a spreadsheet, change the values, play with different weights, and then be sure to take a test drive** before you make a deep and lasting commitment.  Changing CMS in mid-stream is not a simple task--take the long view and get it right the first time. 
_______________
** Try a Plone demo site of your own at Six Feet Up or request a free Plone site at Objectis. 

Wednesday, April 1, 2009

Idealware CMS Report

No matter what fault you may find with Idealware's recent "Comparing Open Source Content Management Systems," we have a rare jewel in the world of web comparative analysis: a report with a completely documented methodology. Of course, you are welcome to disagree with their ratings, their categories, and their thresholds, but at least they are 100% up front about where the numbers come from. Tip o' the hat to Idealware.

So... down to the nitty gritty. What does the report have to say? The easiest thing to do is take their verbal scores (None, Fair, Solid, Excellent) and convert them to numeric values (0, 1, 2, 3 respectively). The result is a modified version of their comparison chart on page 16 (numbered page 14 due to front matter).
















Category
WordPress Joomla Drupal Plone
Hosting/Installation
3331
Simple Site Config
3222
Complex Site Config
3321
Content Admin
3223
Graphical Flexibility
3333
Structural Flexibility
1233
Roles and Workflow
1123
Web 2.0
2232
Extending and Integrating
2233
Scalability/Security
2223
Maintenance
3222
Support/Community
3333


29273029
The long and the short of it is that there is only a 10% difference between the bottom (Joomla) and the top (Drupal). WordPress and Plone are only 3% behind the front runner. I'd say that the differences are "in the noise."

Perhaps a better way to approach this is to weigh those comparison categories to match your needs and requirements, run the math, and see what comes out. For example, if security is a big driver (as it is for me), giving it a weighing factor of 2 might be appropriate. Then the final scores are 31, 29, 32, and 33.

But as is so often the case, we're looking at apples and oranges here from the get-go. WordPress is a blogging/collaboration tool, Joomla is a web CMS, Drupal is a complex mixture of both. Plone by virtue of its powerful combination of features is a collaboration system and a web content management system, yet strong enough to do the heavy lifting for an enterprise portal.

Its always nice to see good scores and the fact that Idealware chose Plone along with only 3 other serious open source systems is high praise in and of itself. Idealware should be complimented for having a transparent methodology, a relatively neutral approach, and giving good press to four worthwhile systems.

Still, I can't stress enough that your specific, unique requirements must drive your choice for a CMS. Don't let someone else's numbers blind you to what you and your community of users need to accomplish.

Coming next: a stroll through Idealware's Consultants listing in the appendix.