After having analyzed Blender’s context in the first essay and its codebase architecture in the second, we take a look at the quality of the codebase in this third essay. Because Blender is already over 25 years old, and because of the complicated nature of the functionality it aims to deliver, we initially expect Blender to have a large amount of technical debt — the degree of inaccessibility, or, the number of “deficiencies in internal quality that make it harder than it would ideally be to modify and extend the system further.”1 In order to ensure a certain code quality, Blender maintains some rules, such as “no code gets in trunk without documentation” (meaning that contributors have to document new functionality in the Blender Documentation)2, no new warnings are allowed, and code style has to remain consistent within files 3. Numerous other guidelines can be found at 4.
Blender’s release cycle is heavily tuned towards bug fixing 5. Blender aims to release an update every three months, but the actual development of an update takes longer and overlaps with other updates, as shown in the figure below. Each update consists of five phases — Bcon1 through Bcon5 — and starts on the master branch. Bcon1 takes nine weeks and has developers focus mainly on new features and big changes. Bcon2 is four weeks, and is focused on improving, optimizing and stabilizing new and existing features, and only smaller changes and features are made here. From Bcon3 onward, the release gets its own branch, on which four weeks are spent on bug fixing6 in Bcon3, one week is spent preparing the update in Bcon4, and the update is released in Bcon5. Also at the start of an update’s Bcon3 phase, the succeeding update’s Bcon1 phase starts. Developers spent 1-2 on update 1’s Bcon3 phase, and the rest to update 2’s Bcon1 phase. Therefore, approximately 60% - 70% of the time is spent on code improvement.
Blender’s testing pipeline is “a work in progress and isn’t anywhere near full coverage”7, and contributors are invited to add unit tests and increase coverage. Test coverage as a percentage of the codebase is not tracked, as the developers feel that there are “not enough tests to really be worth it at the moment. Focus is more on adding tests for things that we know are likely to break and lead to real bugs, for which such percentages are not always a good measure”8. They have some basic tests for the Python code (mainly UI and add-ons), and they use the Google Test unit testing library for the C/C++ (the majority of Blender’s codebase). Tests for both these parts are called with CMake’s CTest. Google Test needs to be enabled at build-time through a CMake build option (
WITH_GTESTS), and creates testing executables which it then runs.9 Blender publishes nightly unstable builds10, and these are tested with the aforementioned test suites. However, since Blender lacks the resources to test and build every commit11, we can’t speak of Blender using a continuous integration or CI/CD approach.
The Python tests use Python’s own
unittest module. Python’s tests can be run from the command line if Blender and Python are installed.12 The list of Python tests is shown below.
Apart from these, there are exactly one hundred tests for the C/C++ code. 31 of these test
libmv (the Blender-maintained multiview reconstruction library used for Blender’s motion tracking13), 28 tests cover the Cycles rendering engine, 33 cover the various functions and types of the Blender Library (
BLI), and the rest is meant for mesh operators and modifiers, and other I/O-functions, tools, and small internal libraries. Tests for the EEVEE and Blender Workbench rendering engines are disabled by default, because different GPUs yield different test results.14 All tests are shown below.
Let’s take a look at one GTest test and one Python test. The file
BLI_path_util_test.cc (test #112) tests Blender’s path utilities. The
#include "BLI_path_util.h" indicates that this file tests
path_util.c. Besides procedures for setting up and tearing down, the test file contains 24 unit tests, each of which tests one function
path_util.c. Each such unit test, then, tests approximately 5-20 configurations, depending on the function under scrutiny. Ten unit tests investigate the
BLI_path_name_at_index function, seven unit tests investigate
BLI_path_join, and the functions
BLI_path_extension all get one unit test. However,
path_util.c defines 51 functions, meaning upward of 40 functions aren’t tested.
The Python test file
bl_pyapi_mathutils.py (test #46) tests the
source/blender/python/mathutils component. It defines a “test case” (which is essentially the same as a unit test15) for the following datatypes:
KDTree; and for the
tessellate_polygon(polyline) function. However, this component also contains many other functions and object, like
Color, all sorts of intersaction/collision tests, geometrical interpolation functions, and noise generators. Therefore, also this component is not fully covered.
Blender is a very large system. Files or directories where many commits or line changes are occuring are also called “coding hotspots”. These coding hotspots can give one insight into which direction the system is developing. The latest stable version of Blender is version 2.82a, released on March 12th, 2020. Since then Blender has entered the Bcon2 stage of the 2.83 release cycle. The following table shows the number of commits, line insertions and deletions since March 12th for the top level directories.
At first glance it seems like the
release directory seem to be the largest hotspots of coding activity, due to the number of commits and line changes respectively. In case of the
release directory, this is due to the addition of a single file that contains more than 270 thousand lines. Further investigations in the
source directory yields the following table.
blenkernel directories contain the most changes.
blenkernel commits are mostly located in the
intern subdirectory which contains all the source files. The
editors contains more interesting subdirectories as each subdirectory located there is a feature. These distribution of code activity can be seen in the next table.
As can be seen from the table, the
sculpt_paint directories are quite high on the list, meaning that there’s still a lot to work going on for these features. These features also showed up in prior release notes and were highlighted in a (video)[https://www.youtube.com/watch?v=EfF2wDXalgU] posted on Blender’s YouTube channel.
Blender does not have a global roadmap per release cycle due to their size. Instead, each module owner maintains a list of tasks. These lists contain tasks that are to be completed in a single release cycle as well as tasks that may span multiple release cycles. It is therefore rather difficult to keep track of what their goals are for a specific version. They do keep track of the changes that have been made in their (release notes)[https://wiki.blender.org/wiki/Reference/Release_Notes/2.83]. Blender has posted a roadmap for their release planning16 as well as big projects that they would like to tackle before in the year of 2020 17. Blender 2.83 seems to revolve mainly around adding small sized features as well as bug fixes. Larger objectives, such as their upcoming Everything Nodes and Asset Manager projects, are projected to be included in later releases.
Searching through the forums of Blender at 18 and the Blender chat 19, there is little talk of technical debt amongst contributors. It is doubtful that many of them are actively worried about this during discussions, as Blender themselves hold code quality days 20 on each first Friday of each month. These days are fully dedicated on improving the Blender code quality. Any contributer who is interested is allowed to participate in the code quality days. The coordination is done online through the Blender chat. They usually work on:
- adding comments to explain code, giving overviews.
- renaming obscure functions and variables.
- split up big files into more organized smaller ones.
- Uupdate the wiki documentation.
A glance at Blender’s development website21 suggests that they share each task done during code quality days as separate tasks so that other contributors can chime in. In these tasks, it is possible to see discussions of testing new ideas, and code quality discussions between contributers. From these, we can conclude that Blender uses the code quality days in order to help reduce the technical debt and improve the code quality of Blender, rather than doing this per feature being worked on. This iterating through the code on a scheduled basis seems like a more organized way of bringing clarity to the project. Consequently, the code becomes easier to understand for new developers and saves time in implementing new features.
Automated code quality analysis
Static analysis can be used to generate an assessment of code quality. Popular projects for this purpose are SonarQube 22 and SIG 23. We have set up SonarQube to do static analysis on the Blender project, as we only have little results from SIG’s Sigrid, apart from the dependency graph. However, we did hear that — in the subset of components we submitted — there are 1401 cyclic dependencies, 380 indirect cyclic dependencies, and 45 bypassing layers.
The code of Blender mainly consists of C/C++ code. Unfortunately, the (free) community edition of SonarQube does not offer support for these languages. However, the UI and some tooling have been written in Python which is supported. Therefore, SonarQube could be run on Python and some markup languages within the Blender source code.
The analysis consists of 727 files and 101,000 lines of code, of which 98% is Python code. SonarQube lists 84 bugs, but many of these do not affect how Blender runs as a program. For example, there is a 10 years old HTML file that describes Blender’s file format. This file contains many bugs and code smells: the ‘Verdana’ font is used which may not be present on all computers, and
<th> (table header) elements don’t have an
id associated with them so that screen readers may not announce the columns of the table. Furthermore, a dozen bugs of severity “Blocker” — stating that
bpy is not defined — are listed. This is however not a bug, but rather a side-effect of Python being set up in a custom way within Blender itself.
A lot of potential code improvements can be found, however, such as empty code blocks that can be removed, commented-out code, and functions that have a lower case and an upper case variant with the same name.
SonarQube also reports a section for technical debt. Even though over 3,000 instances of code smell are found within the Python codebase, this amount is still low considering the size of the project. The overall technical debt is estimated at 69 days of engineering work, which mostly pertains to the UI code. All in all, SonarQube considers the maintainability of the Python part of Blender’s source code good, as the technical debt ratio is lower than 5%.
In conclusion, Blender adheres to a very strict and streamlined development process. They spent a significant time on code improvement and bug fixes. The small amount of SIG’s Sigrid analysis results tell us that Blender violates a nontrivial number of good practices, while SonarQube tell us that Blender does not have as much technical debt as one would expect from a 25-year-old project. Especially given Blender’s sheer complexity, we think it’s code quality is surprisingly good and well-maintained, which also makes it easier for new developers to helpt contribute to Blender.
Martin Fowler. Technical Debt. https://www.martinfowler.com/bliki/TechnicalDebt.html. ↩
Blender Developer Wiki, New Developer Intro. https://wiki.blender.org/wiki/Developer_Intro/Overview. ↩
Blender Developer Wiki, Contributing Code. https://wiki.blender.org/wiki/Process/Contributing_Code. ↩
Blender Developer Wiki, For All Developers. https://wiki.blender.org/wiki/Developer_Intro/Committer. ↩
Blender Developer Wiki, Release Cycle. https://wiki.blender.org/wiki/Process/Release_Cycle. ↩
These fixes are naturally also merged with the master branch. ↩
Correspondence with Blender developer Brecht van Lommel. ↩
Correspondence with Blender developers. ↩
Blender, Blender LTS and 3.0. https://code.blender.org/2020/02/release-planning-2020-2025/ ↩
Blender, 2020 - Blender Big Projects. https://code.blender.org/2020/01/2020-blender-big-projects/ ↩
Blender, Style guide, Code Quality Day. https://wiki.blender.org/wiki/Style_Guide/Code_Quality_Day. ↩
SonarQube. https://www.sonarqube.org/ ↩
SIG. https://www.softwareimprovementgroup.com/ ↩