The architecture behind Meteor’s impact

In ‘the vision which makes Meteor shine’¹, we researched the vision behind Meteor. Now the question is, how was this vision realized? For this, the six architectural views have been described by Rozanski and Woods², namely: concurrency, deployment, development, functional, information and operational. In this blog post, we will be taking a closer look at which ones of these are relevant to Meteor and go more into detail on those which are.

Viewpoint	Relevance
Concurrency	A concurrency view describes which parts of the system can run concurrently and what is managed. For Meteor, this viewpoint is relevant especially with regards to the Meteor methods3.
Deployment	The deployment viewpoint is about the environment in which the system is deployed. This is relevant for Meteor, since Meteor has several dependencies.
Development	The development viewpoint is about the software development procedure. Meteor is open source, which makes this view relevant, since both developers from inside and outside Meteor Software work on improving Meteor.
Functional	A functional viewpoint for Meteor is important. It describes the tasks and interactions of Meteor when it is running. This view is useful for all stakeholders, but especially useful for developers who want to understand how Meteor works.
Information	An information view describes the way that Meteor deals with data. The reactiveness of Meteor makes this a relevant view since data needs to be updated fast.
Operational	The operational viewpoint describes how the system will be managed and supported. This is relevant for Meteor since its users need to know how to upgrade to a newer version.

The line between architectural choices in the Meteor framework and architectural choices with regards to how developers are steered to structure their application can be very thin. The Meteor framework’s tasks is taking the code written by an application developer to create source files for running a built application. In this essay, we will mainly focus on the architectural choices made for the framework, not the end user’s application.

The source code

Looking at Meteor’s source code, there is a distinction between the code that is used to develop Meteor web applications and the code which builds, runs and deploys these applications. In the Meteor project, this separation can be summarised in two parts: the Meteor core and the Meteor tool, respectively. In this section, we will look at how these parts are built up and how they depend on and interact with each other. The architectures of both parts differ greatly, as such we will discuss them separately and then compare them.

Meteor Core

Meteor’s core can be found in the ‘packages’ directory. Every subdirectory is its own ‘package’ and provides a part of Meteor’s core functionality. Packages are relatively standalone, but they can depend on other Meteor packages, as well as packages from NPM⁴. Packages are loaded by both clients and the server of an application, but they can do different things depending on the platform⁴. Furthermore, most packages are optional and are not even included in a newly created Meteor project.

Not all Meteor packages are written by Meteor Software: Meteor has its own package manager, Atmosphere, where developers can submit custom packages⁴. This is a relatively big feature of the Meteor ecosystem and you will rarely find a Meteor application without any third-party packages.

Packages

There are different types of packages⁵.

• Build packages, which run at build/compile time of a Meteor application.
• Runtime packages, which run at runtime of a Meteor application.
• Empty packages / Feature Switches. The presence of these packages can be detected by other packages and it changes their behaviour.
• Wrapper packages, which are wrappers around NPM or other non-Meteor Packages.

While it is infeasible to list every core Meteor package, we will give an overview of the most important/interesting ones.

Package	Function
meteor	Defines Meteor environment used throughout the framework and while developing applications. Core package.
ddp	Distributed Data Protocol. Websocket communication between server and clients.
mongo	Database driver. Provides database interaction and reactivity to Meteor applications.
accounts	User account system. Split up into sub-packages: base functionality, passwords, OAuth, etc.
tracker	Enables reactive programming. Reruns computations when data changes.
webapp	HTTP Server that turns a Meteor project into a web application.
tinytest	Test runner for unit testing local Meteor packages.
ecmascript, typescript	Enable developing in EcmaScript and/or TypeScript.

Looking at the different packages that exist in Meteor’s codebase, it becomes clear that there are higher-level packages, which provide the main functionality, and lower-level packages, which provide more utility and ease of programming. In the following diagram we put some higher-level packages on the left and some lower-level packages on the right. Notice how a lot of higher-level packages have dependencies on the lower-level packages. (Note that this diagram barely covers all the available packages. It is meant to provide a general sense of Meteor’s core packages and their dependencies.)

Meteor Tool

The Meteor tool can be found in the ‘tools’ folder. This is Meteor’s CLI which builds and runs Meteor applications, which will be discussed in more detail later. It’s divided into subcomponents, which will briefly be summarised in the following table.

Component	Function
cli	Command line tool. Takes user input and executes commands.
cordova	Cordova integration for turning your application into a native Android/iOS app.
fs	Communication with filesystem. Enables Meteor to run on Windows.
isobuild	Build System. Compiles, bundles, builds and links packages which make up a Meteor application.
meteor-services	Deploying to Galaxy.
packaging	Managing Packages and Accessing Atmosphere.
runners	Running a Meteor app and Mongo instance.
tests	Test cases for Meteor Tool.
tool-testing	Test runner for tests.
utils, console, static-assets, tool-env	Miscellaneous.

Core vs Tool

When we compare the architectures of the Meteor Core and the Meteor Tool, we find the following.

Firstly, the Tool has a closer resemblance to a ‘standard’ JavaScript project. Its components are divided into subdirectories and files, which can interact with each other using require(), as well as regular imports from NPM. Despite this, it still feels like a ‘single’ project. Packages, on the other hand, are more like a separate project per package. Each package is declared using a package.js file, which appears to be Meteor’s version of NPM’s package.json. It declares imports from other Meteor packages or NPM. Every source file used needs to be explicitly declared within this file, and there can even be separate versions for different platforms (web client, Cordova client, server, etc.).

Secondly, this difference in architecture is also apparent during testing. The Meteor Tool is tested using the meteor self-test command. Rather self-explanatory, this runs the tests for the Meteor Tool, using the runner in tool-testing. A Meteor Package is tested using meteor test-packages, which runs a different type of test runner, TinyTest, on a Meteor Package. For the Meteor Tool, every JavaScript file in the tests directory is considered a test suite, for a Meteor Package, test files generally end in _tests.js and are explicitly defined in the package.js file.

Meteor releases

There are a few things Meteor Software takes into consideration when they deploy the Meteor framework to developers⁶. Note that this is not about how Meteor applications (i.e. applications built using Meteor) are deployed, but rather how developers can obtain and maintain the Meteor framework. First, Meteor needs to be installed. OSX and Linux users can simply use the command-line and run an install script. Windows users, on the other hand, need to install Chocolatey⁷. Developers who like to contribute to Meteor itself, or simply run the development version, can clone the Meteor framework from GitHub and install the dependencies. This is running Meteor from a Git checkout. When this is done, developers automatically download a so-called dev_bundle from Meteor’s servers. This dev_bundle is a bundle of code, packages, and tools which provide the functionality of the Meteor tool. This includes but is not limited to:

• Node.js version
• npm version
• MongoDB version
• Packages used by meteor-tool
• Packages used by the server bundle

The standard dev_bundle should be enough for most changes, but more extensive changes might need manual changes to one or more of the fields described above. Should there be damage to your local dev_bundle and you cannot fix it yourself, there is always a way to rebuild it from scratch and re-package the dependencies. There are also security measures for changes to this dev_bundle. Any pull request involving changes to the dev_bundle will be noted by repo collaborators, and a request to have a new dev_bundle built/published will be forwarded to Meteor Software⁸.

How Meteor runs

The essential concept behind Meteor is that it creates code for multiple platforms from one codebase. For this, it comes with a build system: Meteor Tool⁹. This build system is used to compile, run, deploy and publish Meteor applications.

The Meteor framework is designed with a “minimal kernel”. The largest architectural style of Meteor is that almost all functionality is contained in packages which can be imported when needed. These packages can depend on and interact with each other to deliver functionality to the developer’s application when they are imported. Likewise, any code that is placed in an import/ folder is only loaded when it is used, i.e. (lazily) imported in the JavaScript.

Meteor provides both the client and the server side of applications. It allows developers to provide specific code for each and common code for both¹⁰. When building the codebase, code that is placed in a client/ folder (no matter where in the file structure) is not loaded on the server and similarly code placed in a server/ folder is not loaded on the client. The code that is loaded for the client is executed in the browser or on the device of the users of the application. The code that is generated for the server is executed by NodeJS.

While the Meteor framework does not have specific run time dependencies when building the developers code into an application, a developer can incorporate dependencies like for example databases in their application.

Non-functional properties

Meteor comes with a lot of different packages. This could be an issue in terms of required memory, but Meteor provides an option to create a smaller version of Meteor. This minimalistic version contains less packages, but do not include Mongo or DDP¹¹.

Another non-functional property is that Meteor should be easy to use for new users. On the other side of this, Meteor does not want to limit the advanced user in any way. It accomplishes this by making it so that tutorials and documentation only introduce new and more advanced concepts when a new user would be ready for them¹².

Furthermore, Meteor should run on Linux, MacOS and Windows. It was originally only built to run on Linux and MacOS, but via putting filesystem calls to path and fs modules via the files.js library it can also run on Windows¹³. The drawback of this implementation is that some file related operations are slower on Windows.