After the Community Leadership Summit, our next big event in Portland has been the FLOSS Community Metrics meeting, organized by us together with Puppet Labs, that hosted the meeting in their offices. Special thanks to Dawn Foster and Kara Sowles for all their help and support.
The room was crowded, with people from organizations like Eclipse Foundation, Red Hat, Google, Twitter, PayPal, Open Source Initiative, LibreOffice, Kaltura, Cloudera, etc. There has been a lot of interesting topics and talks, and almost everything is already available in the 2014’s edition website
Let’s try to brief how it was…
There were some interesting remarks during the meeting:
- First level metrics vs. Second and Third level metrics
- James Falkner from Liferay, and almost everybody, described downloads, number of commits, etc. as first level metrics. They provide numbers, but they mean almost nothing because they are easy to cheat. They become more meaningful when they are combined, providing the second level of metrics. Things like activity by timezone, contributors by subproject evolution, etc. would be in that category. One step beyond, the community could be described as buckets of people, having activity in different areas, and the evolution and changes on those buckets would be the third level of metrics and even more valuable, with data about people moving from casual developers to core contributors, people moving from developing (coding) to managing (more active in mailing lists, etc.). This can only be achieved with a whole view, using tools that gather data from all the areas of development in the community, not just from code management systems or even issue tracking systems, as Jesús showed in his presentation about MetricsGrimoire. We already have those second level metrics and we have started to provide third level of metrics to some of our customers on their dashboards, and we will post about it later.
- Executive reports
- There were a lot of critics about executives and the kind of information they demand, more related with the first level metrics (those big numbers) than with the real evolution of the community. At least, we had one executive in the meeting that could defend by himself. Mike, as Executive Director of the Eclipse Foundation presented how metrics are important for him/them, not just numbers. We would like to thank his positive feedback on the work we are currently doing for them:
My pleasure! @Bitergia: Thanks @mmilinkov for the positive feedback about the dashboard http://t.co/qcMOhi25KZ #fcm2 pic.twitter.com/Oudxulb71u
— Mike Milinkovich (@mmilinkov) July 20, 2014
- Take care of what you measure
- Quoting Simon Phipps:
“What you measure is what you become”
. Each project is different, each community too, so we should take care about what we want to achieve with the metrics we collect. There is no “one metric to rule them all”, and each project/community needs to find their targets, and their metrics to measure how they go for it. Bad metrics could be very dangerous as Dave Neary pointed out in his presentation. That’s the reason we work customer by customer with our tools, instead of massive approach, because some metrics/KPI could ruin projects while help others a lot. - Transparency
- Last but not least important, we were in a metrics meeting, talking about tools to measure Free / Libre Open Source Software projects, so almost everybody agrees that we should be consistent with our ideas about it. There are several tools outside to measure a lot of things in software development, and even providing rankings about communities, but if the project wants to ensure transparency, their metrics should rely on Free / Libre Open Source tools, to make anyone free to repeat, verify, and even improve them.
.@jgbarah is right on: open development analytics should be baked into the open development process. Too little transparency today. #FCM2
— Justin Kestelyn (@kestelyn) July 20, 2014
After a great meeting, is time to start thinking on the next edition. It might be in Europe, or it might be not. If you are interesting in participating or supporting it, just let us know.
Everyhting ended as any good meeting should end, with friends taking beers in a good bar close to Puppet Labs offices.
And that was our third day in Portland… Next stop: OSCON 2014. I will write about it soon. Stay tuned!