Mid Liberty Release Cycle: OpenStack Quarterly Report

Finally we released a new version of the OpenStack Quarterly report. This is intended to provide insights about the software development process of the OpenStack projects. This covers information from several data sources such as Git, Launchpad, Gerrit, Mailing lists, ask.openstack.org and IRC channels. And it aims at providing quantitative and qualitative information about the activity, community and software process.
The executive summary of the document is as follows:
  • Users community keeps growing. With an increase of more than a 300% in the total number of questions posted in ask.openstack.org, this is the unique communnication channels with such measured increase. Other channels such as IRCor mailing lists also grow but slower. Although this is the activity measured for the last year, the last quarter shows a significant activity drop.
  • Active Core Reviewers are increasing. Although mid-release cycle analysis usually show drops of activity in most of the areas, the total number of active core reviewers has reached a new peak. Up to 339 core reviewers participated in the review process. This shows an increase of around 9% if compared to the previous quarter analysis.
  • Process keeps stable. The time to merge patches for the main projects show similar numbers than in previous quarter. However, Nova seems to be the project out of the common time to review with up to 10 days. On the other hand, Glance is more in line with the rest of the projects if compared to previous quarter. In any case, Glance shows the second highest meadian time to review with up to 7 days.
  • IRC activity recovers old activity. Due to unknown issues, the total logs analyzed in the last two previous quarters indicated a huge drop of activity. However, last analysis on this data source shows activity in line with previous quarters. Although this is still an unknown issue, the IRC activity has recovered expected activity.

Project teams are also covered in this quarterly report. This quarterly report follows the Governance file for projects, but ignores specs files. Previous versions of the quarterly report such as 2015-Q1 or 2014-Q4 divided projects into the integrated project, specs, clients and others, with specific sections.

This study can be replicated using the databases available for the OpenStack project and the tag 15.02 of the GrimoireLib library that allows to query those databases.

Kilo: the new OpenStack release

[Updated results based on methodological changes]

Kilo, the new OpenStack release, shows a continuous increase of activity if compared to Juno. From Icehouse to Juno, there was an increase of 6.22% in the number of commits and 17,07% in the number of unique authors. From Juno to Kilo, there’s a higher jump in terms of commits (11,23%) and a lower increase in terms of authors (11,16%). However, with this increase, there is a new peak in the number of unique authors contributing to the OpenStack Foundation projects with close to 1,600 different people participating in its development.

After the continuous increase of activity from release to release that we observed in the past, Kilo, the latest release of OpenStack is showing some stabilization. The differences  between Juno (the previous release) and Kilo are the lowest in the history of the analysis we’ve performed for the OpenStack Foundation. Although this release has reached a new peak in contributors, close to 1,500 different persons, the increase from Juno to Kilo was of around 900 commits and 200 authors while from Icehouse to Juno it was of  700 commits and 70 developers.

The list of organizations participating in the development of OpenStack keeps growing as well: close to 170 different organizations have contributed with at least one commit to the development of Kilo.

As the top ten contributors, we find the following organizations:


Regarding to the community itself, the timezones analysis shows a widespread activity around the world. OpenStack is a truly 24 hours-a-day continuous development community. There are three main groups of activity: America, on the left side of the chart, Europa/Africa in the center and Asia, on the right.

Total commits by timezone as detected in Git repositories

Total commits by timezone as detected in Git repositories

Ignoring the UTC 0 activity, that may be biased by developers using UTC 0 as their timezone with independence of their point of residence, the rest of the activity shows North America East and West coasts as the main contributors in number of commits. Europe/Africa is quite close to this activity (most of it due to Europe), although biased by the UTC peak of activity. India could be represented by the the small peak in UTC+5, and finally the rest of Asia, with China and Japan in first place, which is consistent with the localization of some contributing companies.

Methodological notes:

  • Some of the repositories under the OpenStack project have been removed of the analysis. As an example, specification projects are not counted for this analysis. The full list of repositories is available at the last quarterly report sponsorized by the OpenStack Foundation.
  • Developers are counted as the actual authors of the piece of code merged into upstream.
  • The time of commit takes into account the time when that piece of code is merged into upstream.
  • Each release, new repositories are added to the list of analyzed projects. This partially explains the continuous increasing activity in the OpenStack Foundation projects.

Behind the big numbers on the Wikimedia code review process

Having a dashboard usually opens new paths to understand software development communities. This may be seen as the entry point that helps to understand the basics of a community. And on top of this, there may appear new questions related to those basics or to more advanced issues. This is the case of the new work we are working on with the Wikimedia community metrics analytics team: Core Reviewer and Participants.

  • Core reviewers are defined as those developers that can exercise a +2/-2 review in Gerrit. In addition to this, it is of interest for the community to remove auto merges. Although this is an undesired behaviour, that takes place, and those should be removed.
  • On the other hand, Participants in Gerrit are defined as any member leaving any type of trace in the system. In this set we can find reviews (-2,-1,+1,+2), uploads, comments and others.

It is interesting to notice that depending on the community, requirements are slightly different. In the case of the OpenStack community, there are extra requirements for the Core Reviewer definition. And this is that reviews should be found in master branch. This specific measure can be found in the OpenStack quarterly reports for each of the projects of the Foundation.

Continue reading

Data, data and data about your favourite community: GrimoireLib

[This post is part of the lightning talk presented at FOSDEM 2015. The talk was titled as “Data, data and data about your favourite community” whose slides are available in the Bitergia’s Speakerdeck place. The ipython notebook used for visualization purposes is accesible through nbviewer and can be downloaded in GitHub. This is a basic introduction to GrimoireLib.]

GrimoireLib aims at providing a transparency layer between the database and the user. This helps to avoid the direct access to the databases while providing a list of available metrics.

This is a Python-based library and expects an already generated database coming from some of the Metrics Grimoire tools. CVSAnalY, MailingListStats, Bicho and most of the tools are already supported by this library.

Continue reading

FLOSS Community Metrics meeting and FOSDEM 2015: Metrics matter!

As expected, it has been an intense weekend in Brussels, participating in FLOSS Community Metrics meeting, and the stressful FOSDEM.

FLOSS Community Metrics meeting

On Friday, we participated in the FLOSS Community Metrics meeting, that has doubled the number of attendees from last year edition in Portland!!!

It has been great learning from Dawn Foster’s and Jesús González-Barahona’s talks about strategies and tools. We have some good insights about Debian from Stefano Zachiroli, and a very enjoyable presentation form Prof. Merelo about contributors on GitHub by city in Spain, with very funny moments…

During ligthning talks there were a bunch of very interesting presentations from developers, community managers, etc. On the Bitergia side, we presented the work that is being done in MARKOS Project about analyzing project licenses.

The meeting ended with a very interesting open discussion about the topic. The conclusions were clear, everyone is interested to know better what happens in their project, and metrics can help, but you must know how to use them. On the other hand, if we want transparency and work openly, the tools we use to produce these metrics should be free, open and transparent.

Last, it was announced that we are already working on a new meeting for July, close to OSCON dates… Stay tuned!


On Saturday we participated, along with community managers from Puppet and Chef and more than 150 people, in an impromptu AMA session of questions and answers about the importance of metrics to determine the health of the community. It was rewarding to see that many of the metrics and services already available, were interesting for the people. In addition, services that we are working on, as analysis of activity in Meetup, were most requested, along with sentiment analysis.

On this last point, we have made some approximation, but for now we focus on highlighting where the most significant discussions take place, saving community managers from monitor all traffic from mailing lists or forums (like Askbot, StackOverflow, Discourse, etc.).

On Sunday, our colleague, Daniel Izquierdo took part in the lightning talks presenting GrimoireLib, Python library that represents the core Bitergia tools to provide metrics and reports.

And that has been almost all, from a very intense weekend, meeting with old friends, new colleagues, customers and many interesting people and projects… Let’s meet again in 2016!

Akademy 2014

Pretty late writing about the Akademy, but I really wanted to share my experience with this community.

First of all and as a disclaimer, I already knew some members of the community, so this was not my first time with them. I have always enjoyed my time with them and get pretty useful feedback.

This visit was related to the KDE devleopment dashboard that Bitergia prepared in collaboration with some KDE developers.


Continue reading

The OpenStack Juno release: activity and organizations

Within a few hours the OpenStack Juno release will be delivered. At the moment of writing this analysis the OpenStack Activity Board shows 91,317 commits spread across 108 repositories. All of this activity was performed by close to 2,600  developers, affiliated to about 230 different organizations. In addition, around 75,000 changesets have gone through code review, submitted by 3,082 developers.

With respect to community communication channels, there were more than 3 million messages exchanged in the IRC channels, close to 10,000 questions asked on the Askbot instance of the OpenStack Foundation and about 3,600 people posting to the mailing lists.

Focusing on the Juno six-month release cycle, activity was intense:

  • 18,704 commits, 8.65% increase if compared to the previous release cycle, Icehouse.
  • More than 130 organizations contributing.
  • Close to 1,420 developers, an increment of about 16%.
  • IRC channels grew from 926K to 1,024K messages, 10% more than for Icehouse.
  • Mailing lists reduced activity (8.8% less messages). However, there is an increase of 18% in the number of new questions in Askbot, with a total of about 3,000 new questions during the Juno release cycle.

The top 10 organizations contributing to the Juno release shows a list similar to the one for the Icehouse release.

Top 10 organizations contributing to OpenStack Juno

Top 10 organizations that contributed to the development of the OpenStack Juno release

Continue reading