When interested in understanding how the activity of a software project is evolving over time, there are too many parameters to follow, even considering only those that can be directly extracted from development repositories. In our previous post on plotting the activity of a project, we proposed a miriad of plots to show the many aspects of this activity. However complete this perspective may be, it is still somewhat confusing for the untrained eye, and it is difficult to relate how different parameters were behaving on the same time span. Therefore, we have been exploring using Envision to show a more compact, yet complete, view of what is happening in the project over time.
GNOME Shell activity over time
If you browse the real webpage, you’ll notice how you can move the pointer over the plots, to see how they are related, and find out numerical information at any point in time. In this case, information from the GNOME Shell git repository is presented in the left, while the right shows data about the activity in its Bugzilla repository (it could be top and bottom instead of left and right, if you have a narrow browser). You can drag the selectors on both ends of the lower plots to select the time frame which is of interest, to study it in more detail. The idea is to visualize in just one shot how the project is performing, but with enough information to perceive the evolution of the different parameters at the same time.
These days, we’re very busy preparing some new technologies to show how software projects are performing. One of them will provide a combination of automatic extraction of data from software development repositories, semi-automatic filtering and analysis of that data, and visualizations in ways that make it clear the progress and performance of the project as a whole.
Preview of a part of the dashboard for a project
Above you can see the idea, which you can also explore in more detail in the preview page we have set up with real data from the GNOME Shell project (we already talked about it some days ago). In short, it is just a bunch of interactive plots, which in this case present the evolution of some interesting parameters related to the source code management system (git in this case) and to the issue tracking (aka bug reporting) system (Bugzilla).
GNOME was one of the first projects adopting time-based release management in 2002. So, in the last ten years GNOME has published a new release each March and September with only one exception: GNOME 3.0 and the introduction of GNOME Shell. Given this history, how has the time-based release plan affected GNOME projects? In our report on GNOME Shell we have analyzed and visualized GNOME Shell activity, and the results clearly show how GNOME projects are orchestrated.
[The image above is just a static preview, click on it to see the live interactive chart, and to learn about the meaning of each element in the figure]
The information displayed shows the activity (commits and tickets) of the project since 2009. A clear, large peak can be seen when GNOME 3 was released, but for each major release (labeled in the chart) clear increments of activity can also be shown. In fact, the graph shows how GNOME is working with a rhythm: every six months there is a rush to report, fix and commit, with periods of less activity in between. As expected, commit and ticketing cycles are not perfectly synchronized: commits happen mostly just before release-time (to have new functionality and fixes ready for the release); the increase in closing tickets happens before or around it (to have a release as bug-free as possible, and to fix the most urgent issues raised by the distribution of the release); and the increase in opening tickets comes mostly after it (as new bugs introduced by the release are reported).
Last May we were invited to deliver a talk at Akademy-es, about the development history of KDevelop. Unfortunately, we had some issues downloading information from their Bugzilla (basically, we were hammering it, asking for all changes to all tickets, and it seems we were banned access). After some tinkering with delays and with different ways of getting information out of Bugzilla, we finally could get the complete status of all the tickets up to June 17th (7,255 tickets). And after some hectic weeks starting up the company, finally we had some time to complete the analysis on those tickets.
With this new information, we’ve updated the slides we used for the talk. It now includes the analysis of the git repository, which deals with more than 12 years of history (from January 1999 to April 2012), and the complete analysis of Bugzilla tickets, for a simialar period. The slides includes some interesting plots on the evolution of the source code of the project, on different kinds of activity (commits, ticket handling, etc.), among which we show here a couple of them.
The next graph shows nicely when the project had more activity in the past (or when it was shaken up for some reason), and is a good complement to the plot showing the number of commits per month.
Lines added / removed (master branch)