Introducing Alfred: Detect Dangerous Symbols in Python

Here is a common situation: you have a large Python project in which you would like to ban specific functions or modules that you consider dangerous or obsolete. You want to remove them from your existing codebase and avoid them in the future.

Today, we’re open-sourcing Alfred: a flake8 plugin to do just that. It is fast and easy to use, and supports banning any kind of symbol import: function, modules, classes, etc. For the initial release, it only supports Python 3.6.

Internally at Data Theorem, we are using Alfred to automatically detect things like:

• Usage of dangerous dynamic code evaluation functions such as eval() or expr().
• Usage of datetime functions that are not timezone-aware (which can be a big problem in Python).
• Usage of highly-privileged functions (for example to access the database) that should only be used in very specific scenarios.

Alfred allows us to catch these things as part of our Continuous Integration pipeline. Any function call that is flagged in CI is then discussed in order to decide whether the dangerous call should be removed, or if an exception should be made using the # noqa flake8 directive.

Why not basic text search ?

Basic text search or grep would miss a lot of cases. For instance, if you want to ban json.dumps from your project, a simple text search would miss all of the following calls:

import json as j
j.dumps()

from json import dumps
dumps()

A quick example

As an example, let’s run Alfred on the Django code base to see if they use the dangerous eval() and exec() functions.

First, install flake8 and Alfred. For the initial release, only Python 3.6 is supported:

$ cd django
$ pip install flake8-alfred

Then, configure Alfred using a .flake8 or setup.cfg file for the project:

[flake8]
enable-extensions = B1
warn-symbols =
    eval = Using the dangerous function eval!
    exec = Using the dangerous function exec!

Lastly, run flake8 on the Django code base:

$ flake8
core/management/commands/shell.py:77:21: B1 Using the dangerous function exec!
core/management/commands/shell.py:86:13: B1 Using the dangerous function exec!
core/management/commands/shell.py:92:13: B1 Using the dangerous function exec!
contrib/gis/serializers/geojson.py:53:32: B1 Using the dangerous function eval!
db/migrations/questioner.py:138:28: B1 Using the dangerous function eval!

Alfred has detected calls to exec() and eval(). Most of these are legitimate, so after reviewing them we can use the # noqa: B1 flake8 directive to silence the warnings.

The next step would be to add flake8 and Alfred to CI in order to continuously monitor usage of these banned symbols as code gets committed.

More information

For more information, head to the project’s page on GitHub. The project has been released under the MIT license.