why 5-a-day makes me cringe

After reading Mackenzie’s post about 5-a-day on the Ubuntu UK podcast I have to say that:

Dave: “What I’m getting at is, the Ubuntu community is vast, why in the last 7 days is there only 35 people working on it?”

is exactly what I don’t like about 5-a-day.

An issue with any metric is that you need to make very sure that you’re actually measuring/reporting what people think you are. 5-a-day stats are exactly that, stats on the 5-a-day participants, not Ubuntu as a whole. A problem with these types of community metrics is that they’re most often based on voluntary participation and the stats actually become more about who participated in the statistical method than the actual “thing” you were trying to measure.

Now, I think a problem with 5-a-day is not that it’s a bad initiative. I think it’s done a lot of great things around the community and I want to see it continue. However, I do feel like there are some issues with it that I think people need to think about some:

  1. 5-a-day promotes quantity rather than quality. The problem is clearly that quality is very difficult to measure, but the fact remainst that you can create a lot of bug “churn” and not get much of any actual useful work done. We can create a highly participatory community that contributes pretty much nothing to its users, other distros, or the FLOSS landscape as a whole.
  2. 5-a-day is heavily promoted as the way to track bug triaging work in Ubuntu. While it certainly measures some, I believe 5-a-day only measure a minority of the actual work being done in Ubuntu
  3. The 5-a-day stats page gives no indication of what it’s actually measuring and could be easily misconstrued, IMO
  4. 5-a-day not only doesn’t track all contributors, it doesn’t necessarily track all contributions by a by a particular contributor. It relies on a person actively putting bug numbers in their 5-a-day applet/CLI tool.

Lastly, I think a lot of this comes down to 5-a-day being designed to be a fun “scoreboard” for the community to have a little competition. The problem is when people see it rather as serious business, and especially in these days where Ubuntu is being challenged about its contributions I think we need to be perhaps more careful with how these types of metrics are seen by “outsiders”.