"Count what is countable. Measure what is measureable. What is not measureable, make measureable." -- Galileo

Wednesday, July 4, 2012

Confirmation Bias in Software Comparisons

I'm definitely guilty of confirmation bias... frankly, it's part of my job to justify the software tools that my team uses.  But I'm not a paid Plone consultant, so I don't have a conflict of fiduciary interest.  This blog is a voluntary effort completely independent of the Plone Foundation. Full disclosure:  my web team at my day job implements and manages Plone sites. 

What Paul Kaiser is doing, since he makes a living off of the WordPress ecosystem, is criticizing a system with an eye towards bolstering his own line of work.  Since he does make a living with WP, he has a genuine conflict of interest.  To his credit, he makes full disclosure of this.

His recent piece about Plone vs WP has garnered thoughtful and measured responses to clarify some of the statements.  Plonistas David Glick, Alex Clark, Eric Brehault, David Bain, and Matt Hamilton have quickly stepped in to provide the Plone point of view.  Paul's responses have been equally civil, so here's a tip of the hat to polite, open discourse, sharing of ideas, and no flame wars. 

Here are my thoughts, taking Paul's points one by one. 

Straight away Kaiser gives a nod to Blender, PyGame and Trac.  Django, Pyramid, TurboGears, and other top-flight Python frameworks are left off the table.  In the comparison of underlying languages, take a look at http://time2hack.blogspot.com/2012/01/php-vs-ruby-vs-python.html#axzz1zfOHeTNY for a recent set of stats re:  Python, Perl, PHP, Ruby, etc.

But then this is a "WP instead of Plone" article.  Move along, nothing here....

Plone != Harder;

Matt Hamilton's reply covers the misconception that Plone is harder to install, maintain, and develop.  When I was teaching at the College of Santa Fe, we'd install Plone, build a UML model, run it through ArchGenXML, and have a custom archetype all in a single evening.  It probably comes down to the background and experience of the implementer.  More about that down below under human factors. 

Regarding the other points in this section:

1.  "Fewer commercial web hosts support..."  Physical proximity is not the be-all and end-all that it once was.  With cloud-based deployments, this will be less so in the future.  If you control your own servers, it's not nearly such an issue.  We recently lost our long-time sys admin, but a new fellow with no Plone/Zope experience picked it right up.  With his CENTOS background and the clear Plone 4.1 documentation, he was able to build our new cluster without supervision by me. 

2.  "...PHP, MySQL, and WP [programmers] are far easier to locate..."  This opens the entire out-sourcing can of worms.  Take a look at the discussion at http://forums.whirlpool.net.au/archive/1772689, for example, or this interesting anecdote (http://www.electoral-vote.com/evp2012/Pres/Maps/May31.html#item-2). 

All those Plone advantages

Plone _is_ more secure.  David Glick's comment is absolutely on target -- Plone has an order of magnitude fewer vulnerabilities.  This doesn't mean that WP is less secure, it just means you must spend lots more time patching your installation.  It also means that there's more of an opportunity to miss something. 

Plone is faster.  Again David's comment is a good one.  Both WP and Plone can handle loads if configured correctly.  That said, these kinds of stories seem to be a staple of Plone:
Faster and more scalable.  No, using the ZODB isn't cheating, it's good design.  Systems running in front of MySQL just have to live with all that comes with it:  slower out-of-the-box speed, less security. 

Document Management Functions

I manage websites for my neighborhood association and the Albuquerque Bonsai Club, both WP sites.  I shudder to think about scaling these up to handle thousands of documents.  I have Plone sites with tens of thousands of pdf's, LDAP integration, complex workflows, formal review and approval processes, versioning, and multiple languages to boot.  Despite what Kaiser says about WP ease of use, I find Plone far easier to deal with than the two WP blogs I manage.

Customer experience and background count for a lot when you must satisfy a client.  In the end it comes down to listening to your customer and matching requirements to systems.  WP was designed to be a blogging system and that architectural history burdens it's ability to easily and efficiently handle document management use cases. 

Search.  David Bain's comment pretty much covers this.  Nuff said. 

Dealmaker:  Human Factors

Yes, people use the tools they are familiar with.  Typically that means an organization has evolved to have staff skillsets with flexibility and extensibility to handle their current and plausibly foreseen use cases. 

Existing skills and installed systems are themselves aspects of requirements that need to be assessed as constraints.  I won't hire a Ruby developer for a one-of-a-kind project even if using in-house Python talent is not most cost effective; it is an optimum use of HR resources.  The overhead of hiring is an indirect cost that would need to be considered and I'd be left with a under-utilized Ruby developer afterwards.  If you have outsourcing and consulting options available to your client, then that's another situation. The entire system within which software is to be recommended has to be considered by the analyst. 

Weakness in complexity.  From my background in ecology, complexity actually generates stability.  I'm seeing that in software systems as well, depending on how you define "complexity."  And if a four-layer technology stack is too much to handle, you'd better stay away from the likes of these listed here (http://www.tutkiun.com/2011/07/web-technology-stack-analysis.html).  Complex technology stacks, especially in the open-source arena, are generating co-dependencies that actually strengthen them in what mirrors co-evolutionary adaptations between biological species.

If the final analysis hinges on the human side of the equation, one only has to participate in a Plone sprint or conference (not to mention other communities involved with Zope and Python) to see that Plone wins this hands down.  In researching Plone and WP conferences this evening, I was astonished to learn that WP's first international conference was only held in 2011.  Plone Conferences have been alternating between Europe and North America since 2003 (New Orleans, followed by Vienna 2004). 

In conclusion...

I'd like to go back to Paul's initial paragraph:
"Plone is a web-based content management system built on Python, sharing many similarities with WordPress. As a WordPress developer, you may from time-to-time find clients leaning towards Plone. Learn the important differences between WordPress and Plone, and you’ll be better prepared to help such clients."
I'd take issue with the "sharing many similarities" part.  Although the Real Story Group's CMS Subway Map considers Plone and WP to be on the "Web Content Management" and "Collaboration & Social Software" lines, Plone is one of the few CMS's on those two lines that includes the "Portals & Content Integration" line.  RSG places WP far, far from Plone, and also far from Drupal and Hannon Hill, WP's colleagues on the red and blue lines. Alex Clark's comment on feature-for-feature comparisons are very relevant here, as well. 

CMS Matrix shows Plone to rank higher than WP when ordered by clicks (10:33) and views (6:13).  That's indicative of WP being the choice for many people who select it without due diligence because of its low- or no-cost hosting and ability to meet simple use cases.  Plone, by contrast, is considered by those making thoughtful, feature-by-feature assessments to meet stated requirements. 

I'd also take issue with Mr. Kaiser's insinuation that "leaning towards Plone" is a bad thing and that "to help such clients" implies choosing WP instead of Plone.  While the words appear neutral, the general tone of the article is Plone- and Python- negative.  Python is only described derogatorily as "sexy."  Statements like "hard to imagine why anyone would even consider using Plone" slide in while listing only Drupal, Joomla!, and WP as "professional tools."  Disparaging Plone as a "little content management system" is not being cute, it's a euphemism that intends to give positive appearances to negative, misleading statements. 

Paul's not-so-hidden agenda is clear in the paragraph:
"A few issues I discovered almost held water for Plone. Here’s the information you need to knock them down."
 It's obvious that Mr. Kaiser has a case of confirmation bias and is trying very hard to spin everything toward his business of WP.  To their credit, members of the Plone community have called him on his misrepresentations, misstatements, factual errors, and logical fallacies.  Alas, the comments (as of 7/4/12) don't have anyone from the WP community besides the author jumping in to the discussion.  To everyone's credit, the debate has been mutually respectful and polite.

In closing, 

I'll stick to my mantra that all information system decisions need to be well thought-out based on requirements, requirements, requirements.  Those requirements need to be considered at the broadest level taking into account existing resources, human factors, future needs, and many other aspects of the problem to be solved and its environment.

That WP fits a set of very common use cases, is widely available for no- or low-cost hosting, and in its simplest forms can be managed by those with little or no CMS experience goes without saying.  But to say that WP is the "surprise winner" in a Plone vs WP throw-down is just not true.  Especially with regard to meeting requirements of security, workflow, indexing and search, usability, and document management in general, Plone excels.