4 Problems you might have doing open source in your company and how to solve them

North Bridge and Black Duck published last January their 2016 Future of Open Source Survey Results with a lot of interesting conclusions. Maybe the biggest one it’s that Open Source continue gaining force inside the IT business, but its management is chaotic because the lack of  process.

Most common problems related on the survey were:

  • Nearly 50% of companies have not formal policy and process for selecting and approving open source code.
  • One of the major problems of that is security. 47% don’t have a formal process in place to track the code and only 19% of vulnerabilities are detected and fixed automatically.
  • Nearly 1/3 has no process for identifying tracking or solving known open source vulnerabilities.
  • Over 1/2 companies has no responsible to identify and tracking remediation.

Software development as processes you can track

Software development is a process that you can track to have entire visibility of what is being done in a project or the contribution of each team player.

When measuring software development in your company, there are three main purposes to use metrics: check on going work, lead process improvement and motivational aspects.

  • Check on going work: this helps to understand where the development is right now. To be aware of the status helps to understand how fast things are changing when a new process is in the pipeline.
  • Lead process improvement: From a hierarchical way to a flatter way of working, software development needs indicators to help to determine if that process improvement is properly working. And if this is not working, then using another approach or apply other policies.
  • Motivational aspects: metrics can help you to improve meritocracy and transparency in your organization. Metrics will let developers know where they are and how their process is working, but also to have some fun within the work and competitions through challenges, hackathons and other measurable activities that lead to a more community-oriented organizations.

How do we track inner source software projects?

We’ve been developing metrics dashboards and consultancy for 5 years for open source projects. We’ve worked for companies such Red Hat or the Linux Foundation and our work with them is always focus on target and measure their goals.

We use Goal-Question-Metric Approach to establish with our clients which metrics are relevant to track the goals they have set up over the process, the code and the motivational aspects.

When analyzing software development projects, there are several areas that can be analyzed. Traditional analysis are focused on code metrics (code quality analysis), while the activity, community and processes are also key components to take into account. We use tools such Grimoire Lab to track different sources of activity around a project.

We are able to track:

  • Activity: this component focuses on the basics. Any developer when interacting with the infrastructure of the organization leaves traces of activity. From commits to emails, from tickets activity to code review actions. Measuring activity means quantifying the basic traces left by the developers. And when this information is aggregated and calculated over time, it is possible to obtain trends about this type of information.
  • Community: people are key when talking about open source and contribution so we need to measure how people interact with other contributors. Although this is again measured through the traces left by the developers, the analysis proposed are pretty focused on the people and how they build their own social networks. As each developer has her own interests, it is necessary to understand how developers are attracted and retained in a project. What policies are more successful than others when attracting those and who are leaders in the software development process.
  • Process: Process can be defined as the bureaucracy followed by the developers to write a piece of code. When you´re inner sourcing, that is, doing open source styled development at your company, process is quite similar to open source communities where code style, documentation, code review process, continuous integration, design meetings are part of this process. The main difference here is how developers directly interact with other developers, even being from different business units, even been far away geographically. That interaction is the one we need to polish and improve if we want to increase the quality and velocity of the development process.
  • Code: this aspect of the analysis is probably a well known one and there are several tools to do it. Grimoire Lab is focus on the 3 previous aspects. There are usual metrics that determine the complexity of the code, modularity, test coverage, documentation coverage, and others. What we have when measuring this type of metrics is that we can control where our inner source method is leading those metrics. Increases in the code complexity or less test coverage may indicate unexpected behaviors that should be fixed and controlled.

Online dashboards for monitoring

We offer our clients online dashboards with the metrics we’ve established on a consultancy previous process.

With those, we will be able to have almost real time metrics of the code contributions by channel, time lapse to fix an issue, main contributors on code and on mail lists with the time dimension you chose.

Here you have an example from OPNFV project from Linux Foundation. Please, visit it and grab the metrics you want by yourself.

Open Source Software dashboard for tracking

Open source software dashboard for tracking

 

Our book to learn and to start managing inner source projects

We are writing an open book about how to manage inner source at your company.

We can help you with our metric and dashboards. But, In the meanwhile, please, take a sit and enjoy the lecture and if you are going to OSCON, don’t miss our talk there: Are you succeeding when InnerSourcing? Defining a metrics strategy!!!

Dashboard celebrating 25 years of Linux development

To celebrate 25 years of Linux kernel development, we at Bitergia have produced the Linux development history dashboard. This dashboard visualizes the current Linux git repository from two points of view: the history of all commits (changes to the source code) up to now, and the history of all lines in the current version. The dashboard visualizes the main parameters about the development (the who, when and what) are visualized, and allows for drilling down in the data, for example finding the specific commits that lead to a specific part of the code.

linux-dashboard-blame

Do you want to learn about when the lines in the current kernel were authored? Who has participated in specific areas of the kernel? How many files have remain untouched for more than 10 years? Play with the dashboard and find your own interesting details!

The dashboard was produced using only free, open source software tools (among them, GrimoireLab, our tookit for software development analytics). If you want to learn more details, check the slides I intended to use for my presentation at LinuxCon, which unfortunately I couldn’t attend. Those provide some more insight about how it was produced, some examples about how it can be used, and some curiosities found by exploring it.

Continue reading

5 reasons to have a Software Development Community Dashboard

Community managers spend their time in numerous community activities related with his/her main role: to get people to talk and contribute, react to the community managed, keep people engaged, etc. Key Performance Indicators (KPIs) should be set for each community based on its goals. It’s part of the job to elaborate reports with multiple metrics on community health for example. But, measuring should be an effective task.

Keeping this in mind, I’d like to share with you 5 reasons about why community managers or any other professional related with software development should have a dashboard that provides all the data about the community or project that she/he manages:

Continue reading

Understanding the code review process in OpenStack

As a part of our tests with Kibana and Elasticserch as frontends for our MetricsGrimoire databases, we’ve set up a dashboard for understanding the code review process in OpenStack (be sure of visiting it with a large screen and a reasonable CPU, otherwise your experience may be a bit frustrating).

Screenshot from 2015-10-22 00-24-53This dashboard includes information about all review processes (changesets) in OpenStack, using information obtained from their Gerrit instance. For each review, we have information such as the submitter (owner), the time it was first uploaded and accepted or abandoned, the number of patchsets (iterations) needed until it was accepted, and the time until it was merged or abandoned. With all of them we have prepared an active visualization that allows both to understand the big picture and to drill down looking for the details. Follow on reading to learn about some of these details.

[Note: this is our second post about our dashboards based on Kibana. If you’re interested, have a look at the first one, about OpenStack code contributions.]

Continue reading

Testing Kibana: OpenStack code contributions dashboard

We at Bitergia are busy testing new stacks for analyzing and visualizing the software development data we collect. Some our latest tests involve using Kibana for visualization. In this case, we have prepared a dashboard showing the latest contribution data for OpenStack.

Screenshot from 2015-10-19 01-28-46Screenshot from 2015-10-19 01-29-27One of the nice things that these new dashboards allow is the level of filtering and drill down which is possible. For example, in the above dashboard, it is possible to click on any sector on a pie chart, on any entry of a table, on any bar in a bar chart, and the corresponding filter will act. This allows for obtaining specialized dashboards very easily, such as this one with the contributions by RedHat (produced by clicking on RedHat in the list of of top organizations, or the contributions to Liberty, the latest release cycle of OpenStack, by selecting the corresponding period (last bar) in the “OpenStack ten top organizations by release” chart.

If you’re interested in learning about some tips and tricks about what can be done with these dashboards, follow on reading…

Continue reading

Akademy 2014

Pretty late writing about the Akademy, but I really wanted to share my experience with this community.

First of all and as a disclaimer, I already knew some members of the community, so this was not my first time with them. I have always enjoyed my time with them and get pretty useful feedback.

This visit was related to the KDE devleopment dashboard that Bitergia prepared in collaboration with some KDE developers.

kde-main-page

Continue reading

How the automotive industry is participating in the development of GENIVI: BMW and Itestra leaders

After our visit to the Open Automotive’14 hosted by the Genivi Alliance, we have prepared a new version of the activity dashboard based on the preliminary Genivi Alliance report.

BMW (> 1,500 commits) keeps growing in the community, being the leader of the open source development process in GENIVI ecosystem, or at least to the publicly available repositories that Bitergia has had access. However, we do not find another automobile and engine manufacturing company until Ford (10 commits), with a discrete development participation. Although raising to the 5th position if counting commits from its owned subsidiary Livio Connect.

Top 10 affiliations contribution to the development of the GENIVI open source ecosystem

Top 10 affiliations contribution to the development of the GENIVI open source ecosystem

Continue reading