After discussing the vision1 of Meteor and its architecture2, it is now time to look at how its quality and architectural integrity is retained during development.
Overall software quality
When talking about software quality in general, a large part of it can be described by the quality of the source code. A good place to start is what Meteor Software themselves say about contributing to the Meteor framework3. An option for contributors is reporting a bug in the source code through clear bug reports. Those provide valuable data which can be used to improve the quality of the code, and therefore the framework4.
Another source of software quality is of course testing such as unit tests and with tools such as Travis CI5, but this is described in more details in other parts of this essay.
There also exists third party programs that can assist with maintaining the quality of the code. A program called Sigrid provides code analysis which allows you to to measure, evaluate and monitor your software maintainability.
Our Sigrid analysis shows that the Meteor framework does not have terrible maintainability, but it does not have great maintainability either, with an overall score of 2.7. According to Sigrid, the Meteor framework’s unit size and complexity as well as component independence and entanglement deserve low ratings. This could theoretically be problematic for developers who want test, as the large unit size and cyclomatic complexity makes it hard for anyone not familiar with the code to write good tests for it, thus unit testing the Meteor framework can be hard and require a lot of time. The low scores could also be a bad sign for maintainability. When a component which a lot of other components depend on gets altered, it is a costly process to ensure and test that the system does not lose functionality.
A closer look at the Sigrid refactoring candidates shows a large issue with the mongoDB_driver, which Meteor Software planned to update, according to their roadmap. It has a really long function with some complicated if-statements, which are hard to test. This could definitely benefit from refactoring, by splitting up some if-statements and breaking up the code into separate functions.
But we should not take Sigrid/SIG as gospel, the tool is still in beta and does not always work properly. To add to this, a lot of files that generated warnings were test files, which have a good reason to be long and complex, and are not part of production code. Matter of fact, a large number of files which contribute to Meteor’s mediocre scores are (test) files in optional packages.
Nevertheless, looking at the code ourselves, some files, such as tools/cli/command-packages.js, could definitely be split up in smaller parts, which should ease oversight and testing.
Other tools, such as Code Climate, provide Meteor with a similar score. But this tool also again takes test, deprecated, and even example code into account, giving a bit of a skewed image as well.
Testing process and pull requests
The main way the Meteor application is tested is via Continuous Integration (CI). For each pull request, the CI must run each test before the changes can be merged. Meteor has its own rules regarding code quality, which is based on the Airbnb guide. The code style is also checked by the CI, by using ESLint6.
There are several different CI configurations which are used.
- Travis is used to run the Package tests. As described in our last post, there is a separation in Meteor’s source code. Namely into the Meteor Core (in the form of individual packages) and the Meteor Tool, which is a Node.js application which ultimately builds and runs Meteor Applications as Node Applications. This CI configuration uses Meteor’s own test runner for packages, TinyTest, to test the functionality of all the core Meteor packages. In order to test these packages, this configuration instructs to Meteor Tool to launch a webserver, which is then visited using Puppeteer, in order to run the tests for both Meteor client and server.
- AppVeyor is used to test the Meteor Tool on a Windows machine. Starting from Meteor 1.6, the Meteor Tool has support for Windows machines7, but because the existing CircleCI uses Linux / Docker, there was no CI for Windows. This configuration only tests valuable, hand-picked tests8. Furthermore, it also tests installing Meteor (which in this case means running from a Git checkout, not a release installation) on Windows.
- “CricleCI Tests” is used to fully test the Meteor Tool. This is a collection of run configurations made up of different tests, which are ran in parallel in CircleCI. Initially, tests were grouped together/split up by name9, but nowadays, after each group has run, the CircleCI measures how long each test took to complete, and rebalances the test groups based on this information10. Presumably this is done to take full advantage of CircleCI parallel containers. The Meteor Tool is tested using the
meteor self-testcommand, as opposed to themeteor test-packagescommand used to test Meteor Packages. This does not invoke TinyTest, but a separate test runner made for testing the Meteor Tool. - “CircleCI Docs” compares the JSDoc annotations in Meteor’s source code, to the information in Meteor’s documentation repository. The goal of these checks is to find places where the documentation no longer lines up with the source code, in order to provide a warning that the documentation needs to be updated.
Of course, it is also entirely possible to run tests locally, by executing either meteor test-packages or meteor self-test in a local console. Note that meteor also has the meteor test command, but that is not used for testing the Meteor framework, but applications built using Meteor.
Now being aware of the test process, let’s look at how a high standard of code is maintained in the Meteor by going over the discussions in the issues and pull requests.
By going over some open issues, there is not much talk about code quality or testing. Most issues indicate when a problem occurs. There are several issues with no comments at all. In the once that have one or two comments mostly provide some extra information (Issue #10874) or a small update to issue regarding which version is used (Issue #10962). Fewer issues have larger discussion, there is a conversation between the developers and the contributor of the issue. Like in Issue #10850, the developers suggest some things to try and the different parties discuss how to get to the solution. Note that the conversations in issues do not discuss code quality etc. Let see how that happens in the pull requests.
Meteor has a few reviewers who are able to actually confirm a pull request and merge it. These reviewers are part of Meteor Software or frequent contributors11. This means that most discussion in a pull request is done between one of Meteor’s reviewers and the contributor, but also fellow contributors sometimes take part.
In our own pull request, PR #10960, we used the Underscore library for a check. This created a dependency which could be avoided and a change was requested to improve this. Similar types of comments to reduce technical debt can be found in PR #10896.
In combination with the CI running the tests and the points of improvement coming from frequent reviewers and other contributors, the amount of technical debt is reduced and the code quality is kept high.
Architectural hotspots
Because Meteor uses large files, it is hard to identify hotspots in the code. The fact that a file has been changed a lot, does not mean that the code inside has a lot of architectural problems since each change can be to another part of the code. There are however some pieces of code which stand out in terms of amount of changes.
First of all the package.js files in packages that use external sources. For example, the mongo package.
This makes sense, since these files get updated when there is a new version available.
Another file which is changed a lot is the bundler.js file which specifies how the tool builds project files.
Although this file has a large impact on the working of the framework, this is not specifically an architectural hotspot.
The task of this file is combining other files, so it makes sense that this is changed a lot.
Roadmap’s impact on architecture
The Meteor Roadmap12 describes functionality and other changes for Meteor for the near- to medium-term future. Most items in this roadmap relate to updating components and changing or extending functionality available by Meteor application developers. Although this might change the resulting architecture in applications, this does not change the architecture of the framework itself. There are however items which impact the architecture of the framework.
A couple of items are targeted on improving the build performance. This means that the code responsible for building, the Meteor Tool, should be adjusted to be more efficient. For this, parts of the code may need to be simplified or split up, resulting in a better architecture.
Another goal on the roadmap is transitioning as much as possible to NPM. This entails publishing code that does not depend on the core or other Meteor exclusive features to NPM as separate packages. Doing so will result in less code in the main repository and a codebase with less loose components. This mindset of publishing separate NPM-packages is also encouraged for new packages.
Technical Debt
At the start of a software project, it can be hard to imagine how it will fare a couple of years into the feature. What works? What doesn’t? Over the years, Meteor has grown quite a bit, and in this section, we will look at what we believe to be the technical debt present in the Meteor architecture, and things we believe that could have been done differently from the start.
Atmosphere
Nowadays, it’s hard to imagine a JavaScript application which doesn’t use NPM. However, up until Meteor version 1.3, Meteor had no full support for NPM13. In fact, up until version 0.9.0, the Meteor developers did not even take third party packages from Atmosphere into account either. Adding these packages used a community-developed tool called Meteorite14.
While Meteor’s separation into different packages makes sense, because it allows (experienced) developers to choose which functionality to include, the use of a separate package manager feels a bit dated, given the existence and popularity of NPM. Of course, moving away from Atmosphere and Meteor’s current packaging system, and towards NPM, would be a massive undertaking. The current Meteor packages are really the core of the Meteor framework, with the Meteor Tool built around the packaging system. On top of that, Atmosphere packages are designed/implemented almost explicitly for Meteor, so migrating them to NPM might not always make sense. However, the move to NPM has been announced by the Meteor developers themselves1315, and as such we consider this refactoring as the largest amount of technical debt.
Blaze
In a similar way, Meteor’s own templating language, Blaze, feels a bit less-than as well, with the introduction of React, Angular and Vue. So much so, that it’s not even listed on Meteor’s homepage anymore, while those three others are. Furthermore, Blaze hasn’t seen a proper update or commit in over a year16. Since Meteor can be used with other user interface rendering libraries, and developers are not limited to using Blaze, it might be strange to consider phasing it out as technical debt. However, we find that from a documentation point of view, Blaze is still considered Meteor’s “main” library. While there are pages on how to use the other libraries (React, Angular, Vue) in combination with Meteor, the other pages in the Meteor Guide still assume Blaze is being used. We would consider reworking the Meteor Guide on including other frond-end technologies as part of the technical debt.