Measures of Effectiveness

Back in September I introduced Bullock's dissertation on measures of effectiveness. In it he argues that for an entity to reach a desired end-state, one must identify objectives that are described by values. These values will have attributes, which are in turn measurable.

I did not have to identify the exact Plone end-state, choosing instead to focus on two objectives: widely adopted and stable yet evolving. I believe these two objectives form a reasonable desired state for Plone, although the term "end-state" seems a little too final for something that we all hope is on-going.

The problem that Bullock recognized is that objectives and values are quite likely something that can not be directly observed. We only have hard measurements for attributes of values. That's where his use of a Kalman filter is novel. A Kalman filter essentially uses observed attributes to predict unobservable values. Its a useful way of pooling apples and oranges.

Here are my slightly revised attributes, measurements as of 24 Nov. 2007, and the units.

Attribute	Observed	Actual Measure
Release Frequency	0.94	Years/major release
Bugs	704	Active tickets
Core mailing lists	1.875	Msgs/day
Support mailing lists	17.5	Msgs/day
Security vulnerabilities	4	Recent but resolved
Size of CDT	48	Oct & Nov 2007 from Core Dev forum
Involvement of CDT		__________
New features	95%	Percentage of CMS Matrix features
Downloads		__________
Installations		__________
Defectors		__________
Economic health of third-party companies		__________
Technical reviews	8.7	InfoWorld
High-profile installations	950	Plone.net/sites
Users of Plone portals		__________
Visitors to Plone portals		__________

As you can see, I still have some attributes to collect. As a proxy for development, use, and usage, I may look at measures of activity on Openia and Objectis. More to come.