4 Problems you might have doing open source in your company and how to solve them

North Bridge and Black Duck published last January their 2016 Future of Open Source Survey Results with a lot of interesting conclusions. Maybe the biggest one it’s that Open Source continue gaining force inside the IT business, but its management is chaotic because the lack of  process.

Most common problems related on the survey were:

  • Nearly 50% of companies have not formal policy and process for selecting and approving open source code.
  • One of the major problems of that is security. 47% don’t have a formal process in place to track the code and only 19% of vulnerabilities are detected and fixed automatically.
  • Nearly 1/3 has no process for identifying tracking or solving known open source vulnerabilities.
  • Over 1/2 companies has no responsible to identify and tracking remediation.

Software development as processes you can track

Software development is a process that you can track to have entire visibility of what is being done in a project or the contribution of each team player.

When measuring software development in your company, there are three main purposes to use metrics: check on going work, lead process improvement and motivational aspects.

  • Check on going work: this helps to understand where the development is right now. To be aware of the status helps to understand how fast things are changing when a new process is in the pipeline.
  • Lead process improvement: From a hierarchical way to a flatter way of working, software development needs indicators to help to determine if that process improvement is properly working. And if this is not working, then using another approach or apply other policies.
  • Motivational aspects: metrics can help you to improve meritocracy and transparency in your organization. Metrics will let developers know where they are and how their process is working, but also to have some fun within the work and competitions through challenges, hackathons and other measurable activities that lead to a more community-oriented organizations.

How do we track inner source software projects?

We’ve been developing metrics dashboards and consultancy for 5 years for open source projects. We’ve worked for companies such Red Hat or the Linux Foundation and our work with them is always focus on target and measure their goals.

We use Goal-Question-Metric Approach to establish with our clients which metrics are relevant to track the goals they have set up over the process, the code and the motivational aspects.

When analyzing software development projects, there are several areas that can be analyzed. Traditional analysis are focused on code metrics (code quality analysis), while the activity, community and processes are also key components to take into account. We use tools such Grimoire Lab to track different sources of activity around a project.

We are able to track:

  • Activity: this component focuses on the basics. Any developer when interacting with the infrastructure of the organization leaves traces of activity. From commits to emails, from tickets activity to code review actions. Measuring activity means quantifying the basic traces left by the developers. And when this information is aggregated and calculated over time, it is possible to obtain trends about this type of information.
  • Community: people are key when talking about open source and contribution so we need to measure how people interact with other contributors. Although this is again measured through the traces left by the developers, the analysis proposed are pretty focused on the people and how they build their own social networks. As each developer has her own interests, it is necessary to understand how developers are attracted and retained in a project. What policies are more successful than others when attracting those and who are leaders in the software development process.
  • Process: Process can be defined as the bureaucracy followed by the developers to write a piece of code. When you´re inner sourcing, that is, doing open source styled development at your company, process is quite similar to open source communities where code style, documentation, code review process, continuous integration, design meetings are part of this process. The main difference here is how developers directly interact with other developers, even being from different business units, even been far away geographically. That interaction is the one we need to polish and improve if we want to increase the quality and velocity of the development process.
  • Code: this aspect of the analysis is probably a well known one and there are several tools to do it. Grimoire Lab is focus on the 3 previous aspects. There are usual metrics that determine the complexity of the code, modularity, test coverage, documentation coverage, and others. What we have when measuring this type of metrics is that we can control where our inner source method is leading those metrics. Increases in the code complexity or less test coverage may indicate unexpected behaviors that should be fixed and controlled.

Online dashboards for monitoring

We offer our clients online dashboards with the metrics we’ve established on a consultancy previous process.

With those, we will be able to have almost real time metrics of the code contributions by channel, time lapse to fix an issue, main contributors on code and on mail lists with the time dimension you chose.

Here you have an example from OPNFV project from Linux Foundation. Please, visit it and grab the metrics you want by yourself.

Open Source Software dashboard for tracking

Open source software dashboard for tracking

 

Our book to learn and to start managing inner source projects

We are writing an open book about how to manage inner source at your company.

We can help you with our metric and dashboards. But, In the meanwhile, please, take a sit and enjoy the lecture and if you are going to OSCON, don’t miss our talk there: Are you succeeding when InnerSourcing? Defining a metrics strategy!!!

Grimoirecon North America 2017 next May in Austin

grimoirecon North America. Tickets available!

Grimoirecon North America

Using the context of OSCON next May in Austin, we will celebrate our GrimoireCon North America.

We want to take the advantage of having lot of our customers there to show them our newest updates for our metrics and dashboard, such as having Slack integration. We will show also use cases and practical workshops for developers, community managers, project managers and for anyone interested on monitoring open source!

Full agenda will be published during next days, but we can announce today that Ben Lloyd will join us as speaker.

His talk will show how he uses quantitative metrics generated with GrimoireLab to evaluate the strategic value of open source projects and their communities, and it will cover how Samsung is planning to use this information to inform its overall strategy. How effective are internal open source engineering teams at getting code upstream? Does the community incorporate new features and fix bugs and security flaws adequately to meet the needs of the company? What companies and organizations exert the most influence over the community? Is the overall health of the community adequate to meet the needs of the company? How diverse is the developer ecosystem and are there ideal candidates for hire? We’re seeking to answer these questions and more.!

Tickets are available now! Get your early bird before the 20th of April.

 

New release of Perceval: Slack support now available!

Perceval is the Grimoire Lab tool that gives the first step for allowing Grimoire to gather automatic and incremental data for almost any tool related with contributing to Open Source.

Grimoire Lab architecture draft, showing its different pieces

Grimoire Lab Architecture (draft). Some pieces still under heavy development

Perceval goes on the quest to retrieve and gather data from git, GitHub, Bugzilla, JIRA, Gerrit, mbox, pipermail, StackExchange, Discourse, etc.   for producing valuable indices, with GrimoireELK.

New release: Perceval 0.7.0

The new release increase the brunch of sources supported, gaining accuracy for the tracking of the project.

New features and improvements:

  • New set of backends added (some of them sponsored by our clients):
  • RateLimitError exception added for handling rate limit errors.
  • Code was cleaned to follow most of the PEP8 style guidelines.

Backend improvements:

  • git
    • retry calls on SSH commands were added to avoid temporal server errors
  • github
    • HTTP 404 errors are managed when user’s organizations are fetched
    • generic RateLimitError exception is used

Bugs fixed:

  • In Mediawiki backend, the log messages written when a revision is not found were set to ERROR when the real level should have been WARNING.
  • The URL used to fetch jobs in Jenkins was not common to all servers.
  • When UUIDs are generated with some input data, some errors may be raised due to problems encoding invalid characters on the input. To avoid these problems, a surrogate escape control error has been set when data is encoded to UTF8. (#123)
  • Handle Meetup requests rate limit. (#126)

We want you to test Perceval and Grimoire following the under development tutorial. If you are a GitHub user you might try  The Cauldron for free to analyze some GitHub organizations. And of course, feedback, issues and pull requests are always welcome!!!

Key aspects when inner-sourcing your infrastructure

Not everything in inner source is about community building [but a big portion!] and the infrastructure is basic to foster collaboration, transparency and community building.

This post is about aspects to have in mind when deploying the tooling needed to provide support to developers within the organization and across business units.

There is a more extended version of this infrastructure topic in the work-in-progress book about inner source: Managing Inner Source Projects that we, in Bitergia, are writing. Specifically in the infrastructure chapter. This is available under a CC-BY-SA 4.0 license and anyone is more than welcome to propose new sections, improve the current ones and collaborate in any way. Please feel free to redistribute!

Continue reading

We are hiring! Looking for a Python developer

We are hiring a Python developer to help us developing GrimoireLab, our stack for software development analytics. We are looking for a smart, high-energy person, with a passion for learning, contributing and collaborating in free, open source projects to work with us in Madrid (Spain) area.

We offer a 6 month job, with possibilities for a later extension, and a salary range of 25K – 35K € / year.

Restrictions

  • Commuting at least three days a week is required
  • No Agencies Please

Requirements

  • Effective communication and collaboration in English
  • Experience developing with Python (+2 years)
  • Free, open source software knowledge and background
  • Experience with collaborative tools, including Git and GitHub

We would like to do a background check: GitHub profiles or other ways for checking past developments are welcome.

Other skills and experience that will be valued

  • Participation in free, open source software projects (please include references)
  • ELK stack (Logstash, ElasticSearch, Kibana)
  • SQL (MySQL or MariaDB)
  • Python ORM libraries (SQLAlchemy, elasticsearch-dsl)
  • Django
  • Test Driven Development (TDD) and Continuous Integration

About the Company

Bitergia analyzes community software development processes to help companies like Red Hat or Intel, and organizations like Linux Foundation, Mozilla or Wikimedia Foundation to understand and manage the free, open source software, and inner source projects in which they are involved. Bitergia provides training and consultancy for managers and teams willing to take advantage of software development metrics to improve their effectiveness.

Submit your Résumé / CV / references..

If you are interested, please, send us an email!

Gender Diversity Studies and Talks Summary

When looking back nowadays to the work done on diversity, I’ve realized that it has been quite a trip! My first approach to the topic was in an informal meeting with Nithya Ruff, currently at Comcast. She mentioned that the OpenStack Summit in Tokyo reached (as far as I remember!) 13% of women attending the Summit. And this was a great number if compared to previous summits as the percentage kept growing. But she also mentioned that they received a tweet asking about the current number of technical contributions. Then this is where we decided to have a look at that issue: have numbers, and try to produce some of them from a quantitative point of view.

Continue reading