The last blog post was quite a difficult story, wasn’t it. In this blog post, we will conclude our mypy investigation series after quite a rollercoaster ride. The final topic will be a bit simpler (but don’t underestimate it). This last blogpost will investigate the configurability of mypy and why we would want to configure it.
Huh, Configurability?
So, as stated in the first blog post, mypy is a program that can type check other python files. However, this is not the complete story. The configurability of mypy makes it a nifty tool for many end-users (hey, I’ve heard that word before!). Because there are many options available for report generation, typing settings and warnings, there is an option for every taste! The upcoming sections will dive deeper into the stakeholders (what is important for them), and what to configure in mypy.
Even more stakeholders?
As said before, people using mypy are the end-users. During the first blog post, we have specified two different sets of end-users. First, the developers (to be more precise the development teams) using mypy for their project. Secondly, the users of type-checked python programs. This last stakeholder group, for instance, includes all people using Dropbox. However, this stakeholder group is not important for configurability. The development teams, in contrast, are highlighted rather vaguely during the first blogpost. A development team of a project has many people with different interests. For instance managers, testers and coders are roles that are important for development teams.
Since it is clear that these roles have different interests in mypy and its capabilities, we will discuss the different variable aspects of mypy first. Afterward, we will again come back to the stakeholders and describe their interest in the variability aspects of mypy.
Lets switch things up
Now that we know a little more about the stakeholders, let’s talk about the different options of mypy. As said before, mypy has a very broad series of options available to it, to be able to cater to the needs of many different people. The interesting part of variability is to see what users can switch up. Therefore we have created feature diagrams 1 to discuss the options mypy has, using the documentation of mypy 2. In the diagram we have merged options into groups, to make the concept easier to grasp.
In the images, you can see that both the report and main configuration features are hidden. To keep the image clear and readable we have decided to show these parts of the diagram separately. There are two feature diagrams shown above. One of the feature diagrams is for users using the command line interface of mypy, while the other is for users using the daemon.
We will start by discussing some of the higher leaves of the hierarchical tree and gradually descend into the tree.
Daemon or command line
So what about those two diagrams. Is one diagram not enough? Well unfortunately not. While some users might want to run mypy directly and check only some files and just once in a while, other users feel the need to type check continuously. To meet the requirements of both users, mypy gives the option to use either the daemon or run the program from the command line.
Both the daemon as well as the command line configurations have the option to use the configuration file or use plugins. However, they do have their distinct advantages, which we will highlight before going into these features.
Plugins
As the ever-expanding universe, mypy is also possible to expand. The use of plugins makes mypy more flexible. Mainly libraries that do not naturally support the type annotations in PEP484 make it hard to type-check your project. Therefore one could make use of a plugin. A well-known project that can extend mypy is SQLAlchemy. This plugin is useful when working with SQL inside of a project. Mainly the ORM (Object Relation Map) is impossible for mypy to type check correctly and this plugin makes it possible. Very cool!
Configuration File
Both the daemon as well as the command line can use a configuration file. This configuration file contains various options that will be elaborated on in the upcoming sections.
Report generation
Mypy supports a variety of different formats for report generation. Since we have collapsed the options in the overview diagrams, an expanded view of the report generation feature will be given in the diagram below.
In the diagram, we can see that there are 9 options for report generation. The options are specified in the documentation. There is a multitude of usages possible for the reports. One could, for instance, be interested in checking the progress of integrating mypy with a line count report (where the lines of typed code and untyped code are counted), but also a nice overview of typing errors could be given in almost any format (XML, HTML, etc).
Errors/warnings
There is also the option to set up how mypy handles errors and warnings. One could turn certain warnings off or can suppress errors. For instance, if a file has a non-fatal error, this can be either reported or ignored. In the feature diagram, we have distinguished between three main features, but each of these features includes a multitude of options 2.
Typing options
Similar to the errors and warning parts of the feature diagram, for each of the endpoint features under the Typing feature, there are many configurable options. Again we tried to combine them for clarity. With the shown options one could change how mypy behaves when encountering code without types, with the Any type and which types it can ignore. 2
ImportDiscovery
As a developer type checking just one file might be easy. However, if that file uses a lot of different other files via imports, staying on track with the different types in all of these, might be hard. Therefore mypy has options for imports. The behavior can be altered using flags. A lot of previous features are also altered using flags. However, we have chosen to only explain the flags for import discovery, to give the reader a feeling for the concept. Getting the idea is more useful than stating the obvious! The following flags can be set
--namespace-packages: enables using namespace packages (explicitly packages without an__init__.pyor__init__.pyi)--ignore-missing-imports: makes mypy ignore all missing imports--follow-imports {normal,silent,skip,error}: Adjusts how mypy follows imported modules that are not explicitly passed through the command line--python-executable EXECUTABLE: If theEXECUTABLEprovided is PEP 561 compliant mypy will collect type information especially for this executable--no-site-packages: Disable searching for PEP 561 compliant packages--no-silence-site-packages: unsuppress error messages generated within PEP 561 packages
Main configuration
Last but not least, we have features which we collected under the name Main configuration. The features provided in the diagram below are what we distinguished as primary features.
We will go over each of these features to show what you can do with them.
PythonVersion: Make mypy type check as if the program runs under Python version X.YOperatingSystem: Make mypy type check as if the program runs under a specific OSIncrementalMode: Sets variables to make sure the cache is used properly. The cache speeds up the mypy type checking processMiscellaneous--find-occurences CLASS.MEMBER: Prints out all usages of a class member--scripts-are-modules: runs a script (a file without extension.py) as if it is a module
Advanced OptionsPDB: will invoke the Python debugger when a fatal error occursTraceback: Displays a full traceback when a fatal error occursExceptionsFatalError: raises exceptions when a fatal error occursCustomTypingModule: use a custom module instead of thetypingmoduleCustomTypeshedDir: Specifies the directory where typeshed stubs are locatedWarnIncompleteStub: gives a warning when included stubs have missing or incomplete type annotations
Melting it all together
Wow, that was a lot to process. While the information given is quite dense and intriguing, the combination with the stakeholders is what makes it interesting. In the following table, we will describe the coupling between them. Please note that some of the features might not occur in the table. This is mainly because they are not relevant to a specific stakeholder. Also, note that there are a lot of stakeholders in a development team (RUP specifies 25 roles according to IBM 3). We will only cover those that have an interest in one of the variable features.
| Stakeholder | Role description | Variability aspect |
|---|---|---|
| Software Architect | Designs the global architecture | Main configuration variables, use of plugins, import discovery and typing options |
| Designer | Design a specific part or module of the architecture | Import discovery and typing options |
| Implementer | Implements code | Errors/warnings, option to either use the daemon or the command line |
| Test Manager | Makes sure that testing is complete | Report generation options and use of plugins |
| Test Analyst | Selects what to test | Report generation options |
| Test Designer | Designs test and implements them | The complete ConfigFile options |
| Tester | Runs the tests | Needs documentation only |
| Deployment Manager | Oversees deployment | Report variability |
| Project Manager | Oversees the complete project | Report variability |
| Tool specialist | Creates guidelines how to use mypy or other tools | All possible variability is interesting |
Now that we have discussed both the variability options and the stakeholders that benefit from it, we believe the reader will understand our choices made in the table above. Since the main idea behind roles might differ between persons one can always argue whether this table is correct. However, we think it at least gives some insight into why configurability is important.
How it’s made
So, now we know all of the possibilities of mypy, we want to give you an idea of how these options are implemented in mypy. The main source of the options is the configuration file. This configuration file will (or will not) be read in config_parser.py. This parser file uses standard settings if no configuration file is used, or will load the options specified in the configuration file. We could distinguish an observer pattern in this; options are or are not being set. One could see this pattern back in the way the configuration file is used where flags are set to add or remove functions or options. Since everything is loaded when starting mypy, and cannot be changed afterward (except for a few options when using the daemon), we can see this as load time-binding variability. Reconfiguration requires a reboot or restart of mypy.
When developing for mypy one could add options by simply adding functions to the code and adding them in the config parser.
As our last words (we should win the Nobel prize for literature, right?) we want to thank all of you for reading the blogposts. We hope you’ve learned a lot about software architecture; especially about how it influenced mypy. Sadly everything comes to an end and this was our last blog post. So no next topic and no more joking around. However we have a bright note during this farewell, the website we have posted these blogposts on contains many similar blog posts of all kinds of projects. If you are interested in software architecture, we would recommend going there. Again, lots of thanks for reading!