Free / Libre Open Source Software Community Metrics meeting recap

After the Community Leadership Summit, our next big event in Portland has been the FLOSS Community Metrics meeting, organized by us together with Puppet Labs, that hosted the meeting in their offices. Special thanks to Dawn Foster and Kara Sowles for all their help and support.

The room was crowded, with people from organizations like Eclipse Foundation, Red Hat, Google, Twitter, PayPal, Open Source Initiative, LibreOffice, Kaltura, Cloudera, etc. There has been a lot of interesting topics and talks, and almost everything is already available in the 2014’s edition website

Let’s try to brief how it was…

James Falker, from Liferay, talking about bullshit metrics like downloads, etc.

There were some interesting remarks during the meeting:

First level metrics vs. Second and Third level metrics
James Falkner from Liferay, and almost everybody, described downloads, number of commits, etc. as first level metrics. They provide numbers, but they mean almost nothing because they are easy to cheat. They become more meaningful when they are combined, providing the second level of metrics. Things like activity by timezone, contributors by subproject evolution, etc. would be in that category. One step beyond, the community could be described as buckets of people, having activity in different areas, and the evolution and changes on those buckets would be the third level of metrics and even more valuable, with data about people moving from casual developers to core contributors, people moving from developing (coding) to managing (more active in mailing lists, etc.). This can only be achieved with a whole view, using tools that gather data from all the areas of development in the community, not just from code management systems or even issue tracking systems, as Jesús showed in his presentation about MetricsGrimoire. We already have those second level metrics and we have started to provide third level of metrics to some of our customers on their dashboards, and we will post about it later.
Executive reports
There were a lot of critics about executives and the kind of information they demand, more related with the first level metrics (those big numbers) than with the real evolution of the community. At least, we had one executive in the meeting that could defend by himself. Mike, as Executive Director of the Eclipse Foundation presented how metrics are important for him/them, not just numbers. We would like to thank his positive feedback on the work we are currently doing for them:

Take care of what you measure
Quoting Simon Phipps: “What you measure is what you become”. Each project is different, each community too, so we should take care about what we want to achieve with the metrics we collect. There is no “one metric to rule them all”, and each project/community needs to find their targets, and their metrics to measure how they go for it. Bad metrics could be very dangerous as Dave Neary pointed out in his presentation. That’s the reason we work customer by customer with our tools, instead of massive approach, because some metrics/KPI could ruin projects while help others a lot.
Last but not least important, we were in a metrics meeting, talking about tools to measure Free / Libre Open Source Software projects, so almost everybody agrees that we should be consistent with our ideas about it. There are several tools outside to measure a lot of things in software development, and even providing rankings about communities, but if the project wants to ensure transparency, their metrics should rely on Free / Libre Open Source tools, to make anyone free to repeat, verify, and even improve them.

After a great meeting, is time to start thinking on the next edition. It might be in Europe, or it might be not. If you are interesting in participating or supporting it, just let us know.

Everyhting ended as any good meeting should end, with friends taking beers in a good bar close to Puppet Labs offices.
FLOSS Community Metrcis meeting attendees taking some beers in a bar

And that was our third day in Portland… Next stop: OSCON 2014. I will write about it soon. Stay tuned!

One thought on “Free / Libre Open Source Software Community Metrics meeting recap

  1. Pingback: OSCON recap | Bitergia's blog

Leave a Reply

Fill in your details below or click an icon to log in: Logo

You are commenting using your account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s