InnerSource: Efficiency and Quality of Open Source in the Corporate World

Software development has always been a process with many challenges. As an organization grows, it is essential to work collaboratively to have more efficiency, productivity, reuse, and fewer bugs and, as a result, accelerate the innovation process. In today's post, we'll talk about how to achieve these results with InnerSource.

What is InnerSource? In simple words, InnerSource is a growing trend in high-performing software development teams that adopt some principles and practices of open-source teams within an organization.

Today we can see a significant change in the behavior of companies like Microsoft, which went from the biggest enemy to one of the biggest allies in the open-source world in the last two decades. One possible reason for this shift in thinking is survival.

It is estimated that 90% of the entire internet runs on Linux, and between 65% and 95% of an application's code is third-party code, and a good part of these libraries are likely to be open source.

In other words, it is practically impossible to think about software engineering or application development without thinking about or going through some tool, language, or database that is not open source.

Open Source? But Isn’t That Philosophy?

Unlike many people realize, open source goes far beyond philosophy. Currently, the most complex software across the industry has some open licenses. That generated billionaire opportunities, such as in the case of IBM buying the largest open source company in the world, Red Hat, for thirty-four billion dollars (at the time).

Entering the Developer Experience market, or DevEx, these numbers are hegemonic. You can see that the most popular languages in the world are open-source. The same goes for the most popular databases.

Today we can see a significant change in the behavior of companies like Microsoft, which went from the biggest enemy to one of the biggest allies in the open-source world in the last two decades. One possible reason for this shift in thinking is survival.

It is estimated that 90% of the entire internet runs on Linux, and between 65% and 95% of an application's code is third-party code, and a good part of these libraries are likely to be open source.

In other words, it is practically impossible to think about software engineering or application development without thinking about or going through some tool, language, or database that is not open source.

Challenges and Similarities Between Open Source and Large Organizations

That's right! Both open source and large organizations have a lot in common, including the pains: both have to deal with multiple components, contributors, and tools, in addition to strategies.

Let's look at some of these pains below.

Team Topology

One point is team topology: after all, many companies started with a philosophy of remote-first and distributed teams. This became very evident, especially in the 2020s for the corporate world; however, the open source world already has experience with this for at least twenty years.

Code Reuse

Code reuse is another point both aim for; after all, creating the same solution several times causes the effort and knowledge to be duplicated, in addition to a high maintenance cost. Much is discussed about the problem of reinventing the wheel at the solution level; however, little is said at the organizational level.

In the open source world, we have the case of the Apache Foundation with the Apache Commons project, a Java library whose focus is to centralize reusable components in Java, which among its thousands of users are also the Apache Foundation projects.

The same Apache Commons library is used in Apache Kafka, HBase, and Hive instead of each having its solution. This gives you a component that is tried and tested, from commercial applications to databases and a distributed event platform.

Another point of the reuse practice is the possibility of using tools that facilitate the use of components and libraries in developing your solution. Speaking again of the Apache Foundation and the Java world, there is Maven. That allows, in an effortless way, the use of components, such as libraries and plugins within the Java universe.

Team Management and Decision Making

The traditional model tended to work differently, mainly focusing on a management hierarchy. Which, many times, leaves all the responsibility for the result in the hands of leaders in management and directors.

With time and the project's growth, this general management model has some bottlenecks. In this approach, the increase in complexity and the involvement of a larger number of people to gain speed in delivery does not necessarily translate into the expected impact, thus being able to reduce efficiency in most scenarios.

The consequence is an increase in people's demotivation and turnover. This turnover can impact people who know about the problem being solved or, more specifically, about a project component, impacting deadlines, maintainability, and all that in a vicious cycle.

The software is “promoted” to legacy, and the solution is to redo the project; however, if the methodology does not change along the way, the software returns to that state. After all, without sound engineering and development practices, many options are more of a hindrance than a help at critical times.

The other point is the waste of knowledge and the lack or low rate of reuse of components, which means that throughout the organization, there is the same component countless times. Thus, a bug must be fixed numerous times: the same job is done countless times instead of being efficient between teams.

Best Practices From the Open-Source World

The open-source world brings a series of methodologies and practices for the corporate world that have already been successfully validated in sizeable open-source foundations such as Apache, Eclipse, and Linux, among others. Examples include:

• Visibility
• Code review
• Tests
• CI/CD
• Software documentation
• Issue tracker

Using this approach creates a tendency to close gaps and break down knowledge silos within an organization.

In other words, it is bringing to the corporate market all the know-how and maturity of a remote-first, distributed, collaborative methodology that focuses on code quality without forgetting delivery. As is the case with the Java language, the JVM, and the Linux kernel, in addition to several Apache Foundation projects such as Cassandra, HBase, and Hive, among others.

What Are the Most Significant Benefits of InnerSource?

Through the use of the methodology, it is understood that in the short and medium term, the InnerSource can bring, in general, the following benefits:

• High-quality code: With more excellent test coverage and devs paying attention to the quality standard, greater flow within continuous integration is possible.
• Comprehensive documentation: A well-documented code and documentation close to the code make the code the source of truth. This allows for a better understanding of the business by applying Domain-Driven Design (DDD) good practices as a ubiquitous language.
• Effective code reuse: With good documentation of both the code and the architecture, its reuse is much easier, not to mention that the onboarding process tends to be less expensive and faster.
• Strong collaboration: As a consequence of the previous points, the model promotes collaboration with fewer frictions.
• Healthy culture: Silos are broken, and communication becomes more transparent. Reports show this also increases the engineer's purpose, reducing the team's turnover.

In other words, several people with different knowledge and specialties know the documentation, access the code, test, reusing, and ensure the quality and reputation of the code.

Does It Work in Production?

These practices are excellent. It is a utopian proposal. Does it exist? Does anyone already adopt it?

The answer is yes! There is even an InnerSource Foundation that contains success stories from several companies worldwide; for example, Microsoft, Gitlab, Adobe, and Capital One.

The benefits are very similar between companies in addition to increasing the efficiency of developing.

Another point is security: almost 40% of devs identify that InnerSource helps to identify security problems.

In its book on InnerSource, PayPal reports more fantastic room for innovation, quality, and scalability of existing solutions within the organization.

Also, according to the State of InnerSource report, 61% of people who responded pointed to knowledge sharing as the most significant benefit of adopting InnerSource. Then, 51% declared that the use increased job satisfaction.

InnerSource in Brazil

In Brazil, there are already companies that already use and adopt the concepts and practices quite fluently. Next, let's see a little more about the applications in Itaú and Zup:

InnerSource at Zup

For example, within Zup, most of the practices applied in its open-source products were used in internal and external projects of the organization.

In this case, we can highlight the architecture documentation part in which the projects were concerned with the application of the C4-model, the use of a Tech Radar in addition to the Architecture Decision Records (ADR).

As a next step, the goal is to scale these practices and many others throughout the organization.

InnerSource at Itaú

At Itaú, many teams already practice InnerSource, and some already practice even before talking about InnerSource on a corporate scale.

More than using InnerSource, it's essential to know why to use InnerSourcethe purpose and the value it will bring  – as we know that InnerSource is not a silver bullet.

One of the first approaches is to start InnerSource, where reuse and collaboration by more than two teams make sense. And if it makes sense to adopt InnerSource practices, we help teams structure themselves for this with their repositories.

For example, we encourage teams to share their components and libraries and promote visibility so other teams can reuse and not build from scratch.

In addition, we seek to understand the potential of InnerSource for teams. Some practice InnerSource to remove the day-to-day bottleneck, others for reasons of team capacity, and others to generate adoption for their components, libraries, or infrastructure templates, for example.

And through this, we generate data for the teams that practice InnerSources, such as product adoption, collaboration, and the number of days the issues are open. In other words, InnerSource is a solution for day-to-day software engineering practices.

Our next step is to create, with several teams, a reuse standard where everyone can bring good practices, regardless of the group, and promote a grander scale in the organization.

Also, based on their experiences, Itaú, Rede and Zup focus on an InnerSource that aims to impact the three organizations further. The goal is to ensure reuse, raise the technical quality of the team, and reduce development time for existing components to focus on innovation directly for the business.

Conclusion

The world of open source is an excellent source of significant success cases such as JVM and Linux, in addition to supremacy regarding developer experience with languages, databases, and other widely used products.

In addition, open source brings a culture that focuses on software quality, clarity in communication, good documentation, continuous delivery in a distributed team, ensuring reuse, and focusing on innovation. This culture in dealing with software development makes many organizations seek the same result. That's why organizations like Itaú, Rede, and Zup aim to increase efforts in InnerSource culture. Achieve even greater heights through more remarkable and more fluid collaboration.