What is Docker Compose, and why does it matter?

Docker Compose is an orchestration tool that manages multiple docker containers in a single application, allowing the running of application components in isolated environments without unnecessary complexities. Compose automates the building, deploying and running of the docker containers allowing engineers to develop and deploy without building, dependencies, and compatibility issues. Compose achieves such isolation using container-based technology which allows a lower CPU and memory overhead compared to hypervisors and virtual machines.

What will I learn from this blog-post?

This blog-post will discuss a high-level overview of Docker Compose and its motivations. The end-user mental model explains how the user perceives and uses Compose. Then, the capabilities and requirements of Compose are addressed. Later, the stakeholders influencing these requirements and their goals are discussed. Next, the context of Compose in terms of the People, Technical External Systems, Business, and Life-cycle is addressed. Finally, the future roadmap is presented.

• End-User Mental Model
• Key Capabilties and Requirements
• Stakeholders
• Context

End-User Mental Model

How are Docker Compose users actually using Compose?

The End-User mental model of Compose mainly consists of three parts, namely :

1. Service
2. Network
3. Volume

Whenever the user defines a docker-compose.yml file, the end-user has to define those three components.

Service

A service is a docker container defined by the end-user to carry a specific task. Services in Compose can be subdivided into more parts, listed below:

• Image: From which Docker image is the service based on.
• Build: Which Dockerfile specifies the service building process.
• Ports: Which port of the Docker container maps to a port of the host machine.
• Volume: What file system should be mounted to the Docker container file system. The file system could be from the host machine or user defined. The user defined file systems are discussed in more detail here.
• Link: Makes the interdependence of services explicit.

Volume

Volume is configurable; the user can decide to mount on the services file systems. The data in volumes persists between Docker containers restarts.

Network

Network is a user-defined local area network with options to configure which services are part of it. Networks are usually defined for the services to communicate between each other with a network protocol (e.g. UDP, TCP, HTTP).

Key Capabilities and Requirements

Technically speaking, what are the requirements and capabilities of Compose?

The driving idea behind Compose is to enable IaC (infrastructure as code). IaC eliminates manual provisioning of systems. Compose allows local development and facilitates zero time deployment from anywhere. It allows for complex applications to be shared with its customer’s stakeholders and enables personal infrastructure without requiring external IT resources. Compose helps to configure resource allocation, connections with containers, environment variables, state-specific data along with persistent data. ¹

The top-level functional requirements of Compose are: ¹²

Application

• Define complex computational systems with few lines of code
• Define an application with multiple services
• Enable building, starting, launching an application using the command-line tool in detached mode.
• Communicate with the Docker engine to enable running applications such that it behaves like a single container.

Containers

• Create, stop and remove containers
• Display process information of the containers
• Connect a container to other environments
• Operate on the container as a normal Docker container

Networks

• Instructs Docker engine to create a network over which the applications’ containers can communicate
• Remove defined customer network

Volumes

• Configure data volumes
• Define environment variables

Can you give a use-case for more perspective?

Suppose you have a two container system running on two separate hosts. Compose will build systems such that you can run both services from the same host system without putting much effort into network configurations. This enables continuous simplified delivery ¹.

Stakeholders

Who decides these capabilities and requirements?

Major stakeholders influence the decisions made regarding Compose. These are discussed below. ³⁴

Businesses

Docker Inc. is the acquirer of Compose. It is responsible for generating revenue via subscriptions to Compose Support. This means making Compose scalable, ensuring fast software delivery, and acquiring investments to build the open-source platform.

Companies like Circle Cl, FLocker and Codefresh are in partnership with Compose. These partners wish to provide solutions leveraging the Compose API, benefit from co-marketing with Docker Inc. and provide confidence to customers to use and install Compose via their solution ⁵.

End-users

The end-users are using Compose to build/view applications. Their goal is to use Compose for fast performance, UI responsiveness, and easily transform the template into application in minutes⁵.

Customers

Customers are regarded as businesses using Docker Compose within their organization to deploy applications (e.g. Harvest and Key location). Their goals include scalability, easy configuration, and fast development setup.

Some technical goals could be the usage of standard Docker API, multi-container descriptor simple configuration, and Kubernetes integration.^{5 6}

Domain experts

The main developers along with Docker Inc. are most likely the domain experts. Their role is to guide Compose to follow its roadmap and intercept possible dangers within the platform (using their experience).

Developers/Testers

The developers and testers are building and testing Compose. Their goals are to have access to well-written documentation, reliable networking, and versions with working features. They expect less volatility in the system, and not to be dependent on a single machine; both currently unsatisfied by Compose ⁵.

Docker Compose Context

What are the aspects involved in building such platform?

Context is a multivalent world which we define as a combination of Bass’s architectural context categories: technical, business, professional, and life-cycle ⁷ with the intuition provided by Rozanski & Woods ⁸ in their definition:

“The Context view of a system defines the relationships, dependencies, and interactions between the system and its environment—the people, systems, and external entities with which it interacts.”

We analyze four subsets covering the big picture. All subsets consist of interactions between them.

The Present (2020)

People

Since Compose is an open-source system, the word people make more sense than professionals. Compose community is part of the Docker Community ⁹ and follows the same guidelines. Chatting is the main communication channel, mostly centered around Q&A, at the user level. To find insights about the Compose organization, we took a look at the releases and commit history since 2013. The commit history presented in Table 1, shows that a small group of participants did most of the commits. The absence of recent pushes from top developers suggests that new developers started to take ownership over time. According to Sebastiaan van Stijn (Open Source Manager at Docker), the changes of ownership is because the Compose project was initiated by Aanand Prasad and Ben Firsh and later acquired by Docker. Prasad and Firsh moved to different roles inside Docker, and are no longer working on the project.

Nowadays, the maintenance of the project is done within the developer solutions, a team inside Docker where people work on projects in rotation. This helps spreading knowledge and prevents a project becoming dependent on a limited number of people. Releases in 2020 are authored by an automated pipeline. Before, it was done by some of the active top contributors. According to Van Stijn, the developers’ influence in their release cycle depends on the nature of the features/changes. As the main “Docker” CLI (the docker stack deploy), the Docker app plugin, and Compose itself use the same file format, the Docker team tries to keep those projects in sync by functionality added to the Compose file schema. Changes specific to the Compose command-line can be contributed by the community; those will be evaluated and included if they address a common use case.

Technical External systems

External technologies that influence Compose are Docker, Linux distributions, Linux shells, Secure Shells (SSH), Python implementation for YAML pyyaml, orchestrators like Kubernetes and Swarkit, and docker registry servers.

From all these external systems, Docker is the biggest influence for Compose as external technology and on the people side.

Businesses

Online, we found suggestions to use Compose for local dev environment in many IaaS, PaaS and SaaS providers like Digital Ocean, Amazon or Heroku. This indicates that most usage occurs within software companies.

Life Cycle

We analyzed the life of Compose by looking at the bug/features frequency over releases during the last two years. We noticed that Compose is in continuous evolution. Table 2 shows features/bug fixes that occur at a constant rate. Features added in the last two years indicate that Compose orbits around security, optimization, compatibility, and more configuration options.

The Future (+2020)

According to Van Stijn, the roadmap will keep Compose as an easy-to-use solution for developers, focusing on commonly used features and avoiding complexity.

All information below exceeds the 1500 word limit

Thanks

Special thanks to Sebastiaan van Stijn for answering all our questions and guide us in this Compose adventure. Details of the interview can be found below.

Release table Features and Bug fixes:

Author	Number of commits
Joffrey F	1510
Aanand Prasad	1181
Daniel Nephin	780
Ben Firshman	522
Ulysses Souza	164
Mazz Mosley	159
Harald Albers	94
mnowster	70
Steve Durrheimer	38
Nicolas De Loof	32

Table 1: Top 10 Authors, for 2013-2020 in Docker Compose.

Date	Release	Features	Bugfix	Author
2020-02-24	1.26.0-rc2	yes	yes	docker-dsg-cibot
2020-02-03	1.25.4	no	yes	docker-dsg-cibot
2020-01-31	1.25.4-rc2	no	yes	docker-dsg-cibot
2020-01-23	1.25.3	no	yes	docker-dsg-cibot
2020-01-20	v1.25.2	yes	yes	docker-dsg-cibot
2020-01-20	v1.25.2-rc2	yes	yes	docker-dsg-cibot
2020-01-06	1.25.1	yes	yes	ulyssessouza
2019-11-29	1.25.1-rc1	no	yes	ulyssessouza
2019-11-18	1.25.0	yes	yes	ulyssessouza
2019-10-28	1.25.0-rc4	yes	yes	rumpl
2019-10-28	1.25.0-rc3	yes	yes	rumpl
2019-08-07	1.25.0-rc2	yes	yes	ulyssessouza
2019-06-24	1.24.1	no	yes	rumpl
2019-05-23	1.25.0-rc1	yes	yes	ulyssessouza
2019-03-28	1.24.0	yes	yes	ulyssessouza
2019-03-22	1.24.0-rc3	yes	yes	ulyssessouza
2019-01-14	1.24.0-rc1	yes	yes	rumpl
2018-11-28	1.23.2	no	yes	shin-
2018-11-01	1.23.1	no	yes	shin-
2018-10-30	1.23.0	yes	yes	shin-
2018-10-17	1.23.0-rc3	yes	yes	shin-
2018-10-08	1.23.0-rc2	yes	yes	silvin-lubecki
2018-09-26	1.23.0-rc1	yes	yes	shin-
2018-07-17	1.22.0	yes	yes	shin-
2018-07-05	1.22.0-rc2	yes	yes	mnottale
2018-06-21	1.22.0-rc1	yes	yes	shin-
2018-05-02	1.21.2	no	yes	shin-
2018-04-27	1.21.1	no	yes	shin-
2018-04-09	1.21.0	yes	yes	shin-
2018-03-31	1.21.0-rc1	yes	yes	shin-

Table 2: Release relation Bug/Feature in docker-compose for 2018-2020.

Interview Quotes of Sebastiaan van Stijn

“Compose (formerly known as “Fig”) project was started by Aanand Prasad and Ben Firsh. Their company (and the project) was later acquired by Docker, where development was continued. Ben and Aanand moved to different roles inside Docker, and later went on new journeys, so they’re no longer working on the project. Nowadays, the maintenance of the compose project was then moved to the “developer solutions” team inside Docker, which is responsible for various projects and products. Within that team, people work on projects “in rotation”, which helps spreading knowledge and prevents a project depending on a limited number of persons.”

“…it depends on the features or changes nature; given that both the main “docker” CLI (the docker stack deploy), the docker app plugin, and Docker Compose itself use the same file format, we try to keep those projects “in sync” on functionality added to the compose file schema. This means that a change added in the docker CLI should also find its way into docker-compose “standalone” eventually. We may move the compose specification to a separate repository to make it more clear where the “canonical” location is to introduce changes in the schema.

“I’ll be honest; there’s been some feature-creep in the past where features have been accepted for corner-cases. Adding features is a lot easier than removing/deprecating them, so we’ll have to live with some of those, or find a migration path to improve. The goal of Docker projects/products is to make things easy to use and to provide sensible defaults when possible. Some features added in the past don’t “fit” that principle, so it may have to be looked at some point. Finally changes based on user and customer feedback, research; if we identified workflows that can be improved by adding a feature, such features can be added to the roadmap. Regarding control of releases, there is not a build master,that’s a rotating role, not a fixed person.”

“All through GitHub; open a ticket to propose enhancement or feature and/or open a pull request to propose a change. Providing a good description of the use-case that the change addresses is important there, as it helps to decide if the proposed solution makes sense, or if other solutions should be considered/designed.”

“While some things “settle down”, the container ecosystem is still a fast moving “landscape”. We want docker-compose and the compose-file format to be a great, easy-to-use solution for developers. The compose-file format is very popular as it (despite some “odd” choices from the past) has a nice balance between “commonly used features” and not being overly complex. We see adoption of the format for many uses; for some as an “end format” (running containers locally), for others as an intermediate format (deploying to kubernetes, or generating kubernetes YAML’s from the compose file).” - Sebastiaan van Stijn

Written by Manisha Sethia, Kanya Paramita, Andrea Nardi, Lucio Guerchi

‌ ‌ ‌ ‌ ‌