Reducing build times for C/C++ Projects with JFrog’s Conan and Lattix

One of the challenges of large C/C++ projects is the slow build times. There are many techniques you can use to reduce build times: Pimpl Idiom, Forward Declarations, Guard Conditions, reduce interdependencies and using parallelism. We are going to focus on the last two techniques reducing interdependencies with Lattix and using parallel builds (and incremental builds) with JFrog’s Conan. Our services partner, Verifa, has recently had success reducing the build times for several companies using Lattix and Conan together.

What is Conan?

Conan is a C/C++ multiplatform package manager that allows you to share and build native binaries easily. By making it easy to create builds, share packages, and run them in different environments (Windows, Linux, macOS, etc), Conan allows C/C++ applications to be used in CI/DevOps environments like Jenkins. Conan does this by describing how to build a library in any build system and all the dependencies for that library.

What is Lattix?

Lattix is an architectural analysis solution that helps you understand, define and control the dependencies in your source code. Lattix works seamlessly with C/C++ applications in a CI/DevOps environment like Jenkins by continuously checking the source code for architectural issues and allowing you to model different architectures and understand the impact of those changes.

How Conan and Lattix work together to optimize your build

There are several ways that Conan alone can help you improve your build times:

Incremental builds – only rebuild the libraries that need to be rebuilt. These are the libraries that have been affected by a dependent library that has changed

Parallel build – when there is no relationship (dependency) between two or more libraries in the project graph, you can build them in parallel

Lattix can help you improve your build times by reducing interdependencies or making your application more modular and independent. Build times depend on the number of files and the complexity of those files, it is also affected by the number of files each file depends on. For small projects, the total number of files is so small the software architecture is not a concern. As projects grow, build times grow. This is when improving the physical structure (architecture) and managing dependencies of the software are more of a priority. Lattix helps larger projects improve the physical structure and manage dependencies (ie- increasing modularity). One of the many benefits of modularity is that it decreases build times because it reduces the number of libraries that are affected by a change. When a build is less affected less code needs to be rebuilt which translates to faster builds.

Separately, Conan and Lattix both optimize your build. When taken together, they can supercharge your build optimization. While you can see big improvements in build times as soon as you enable parallel builds or incremental builds. The extent of those improvements depends on your project architecture and how many dependencies you have between them. This is where Lattix can help. With Lattix’s patented “what-if” analysis using the dependency structure matrix technology, you can model your builds in Lattix. Lattix will quickly and easily tell you everything that will be impacted by a change to a library and what libraries you can build in parallel. You can also try alternative “what-if” architectures and understand their impact on the build. Projects that are more modular (fewer interdependencies) will see the biggest improvement in parallel and incremental builds. Here is an example of reducing build times by removing dependencies with Lattix and parallelizing the build with Conan:

When library G changes, here is what the incremental build looks like:

Run a Conan Build/Jenkins Task to Build G Run a Conan Build/Jenkins Task to Build E Run a Conan Build/Jenkins Task to Build F and B together in parallel (the dependency structure matrix above on the right tells you that you can run F and B in parallel based on the application’s dependencies) Run a Conan Build/Jenkins Task to Build C Run a Conan Build/Jenkins Task to Build A

So, in this case, the only library that does not need to be rebuilt is D because it’s not affected by a change to library G. We can also see in the dependency structure matrix that for a full build libraries [G,D] and [F,B] can be run in parallel.

Using the Lattix Architect’s “what-if” analysis, if we remove the dependencies between F->E and B->E. We have the following two modules:

We now see that libraries E and G are a separate module. When we make changes to library G, the incremental build looks like the following:

Run a Conan Build/Jenkins Task to Build G Run a Conan Build/Jenkins Task to Build E

All the other libraries [A,B,D,C,F] are unaffected and do not need to be rebuilt. We can also see that for the full build, we can run libraries [G,F,D] and [E,C,B] in parallel.

In this example, removing two dependencies has a significant positive impact on both the incremental (66% decrease in the number of libraries involved) and full builds of this C/C++ application.

Conclusion

The reason that many forward-thinking companies are making large investments in DevOps and CI is that they can reduce the time to develop and release applications. This allows for more testing time which will increase the quality and reliability of the C/C++ applications. Using Conan and Lattix together you can dramatically improve your build times and allowing you to release products faster.