"Count what is countable. Measure what is measureable. What is not measureable, make measureable." -- Galileo

Tuesday, May 10, 2011

Plone Books and Plone Impact


A bit past due, I'm back to Plone Metrics with an update on my almost quarterly Amazon stats. 

The data set continues to grow unpredictably with 13 out of 15 texts that I track reporting sales rank numbers.  The spread between minimum (good) and maximum (not so much) is the largest ever, due to slow sales of Gross's specialized multimedia tome.  Some values are nearly constant, while others show the inevitable decline (increase in sales rank) as Plone 2 textbooks become obsolete.

For the first time ever, there are no books below (better than) 500,000.  Part of that is slow roll out of Plone 4 texts.  Aspeli's latest is included, due out in June.  Below is a chart showing publication numbers for each year since 2004. 


I had the good fortune today to attend a workshop in Santa Fe on Measuring & Mapping Science.  Many of the concepts have applicability to measures of CMS effectiveness and I'll certainly bringing these new ideas to this blog in the coming months.  While many of the presentations today concentrated on tracking the network of citations in scientific publications, others brought up novel ways of looking at usage statistics.

One important distinction was that between prestige and popularity.  Simply counting clicks, views, downloads, and so forth is a surrogate measure for something's popularity.  But the impact of something must be measured differently to get a handle on its importance or prestige. 

As a first whack at a quick-and-easy impact metric, I turned to Google Scholar to see what sort of journal citations the various CMS's are picking up.  I used the top 10 CMS's from CMS Matrix with the largest numbers of compares. 

The result has almost zero correlation, meaning that importance in terms of who's writing articles about Plone doesn't have any relationship to the number of comparisons CMS Matrix reports. Here Plone (red) has 1300 citations and almost 150,000 compares. 

Google Scholar is just a first cut at a measure of impact--citation counts are only the first step.  The context of those citations now has to be understood to learn what is a positive vs neutral or negative cite.  Because Plone is a very mature CMS, in coming weeks I'll be looking for better metrics of how it has had a significant impact over the last decade in terms of driving the CMS environment by being a leader in innovation, emergent features, and usability.