Here are some interesting tidbits and links:

• Nucleus Research 2003 whitepaper "ROI Comparison Report: Portals"
• "Recommendations for Benchmarking Web Site Usage among Academic Libraries" by Christy Hightower, Julie Sih, and Adam Tilghman (1998 but still interesting)
• "Content Management Problems and Open Source Solutions" by Seth Gottlieb (2005)
  
• "Defining Open Source Software Project Success" by Kevin Crowston, Hala Annabi, and James Howison (2003)

Also, here is the link to an Excel version of my matrix of effectiveness measures.

I'll be looking for more up-to-date material in the days and weeks ahead, but its clear that people have been mulling over the problem since the late 90s. I'd like to hear from those attending the Naples Convention on what they learn about the issue.