The course, Deployment of Machine Learning Models is now live on Udemy

Elegant Flask API Development Part 1

Working with Flask-Injector
25 September 2018

Introduction

Flask is one of the most popular web (micro)frameworks in the Python ecosystem. In contrast to Django’s “batteries included” approach, Flask is lightweight. This means that its core is simple and extensible, and that many of the design decisions are left to the developer. There are many pros to Flask, and to be clear, I think it is a great tool which I personally enjoy using. However, in this post I’m going to discuss one of the microframework’s drawbacks, and how to mitigate it.

If you would like to jump straight to the example code, here’s the github link


Flask Challenges

Flask requires a lot of globals and tightly bound code. To delve into this topic more, let’s first refresh our memory of Flask’s state management. Flask applications have three kinds of state:

  1. Setup state
  2. Application context bound state
  3. Request context bound state

Flask globals that you are likely to find yourself working with include:

  • flask.g and flask.current_app
  • app.config
  • the db object
  • flask.request

In the above list, apart from flask.request, all the objects are bound to the application context state. The flask.request object is bound to the request context.

flask.g is defined in the docs as:

A namespace object that can store data during an application context. This is an instance of Flask.app_ctx_globals_class, which defaults to ctx._AppCtxGlobals. This is a good place to store resources during a request.

When you really break it down, all flask.g does is provide a global for storing state. Initially, this may not seem like a big deal. However, the risk comes as you start to consider lower levels of a larger application. With the ability to pass around bits of state, there is the temptation to write code all over the place that accesses flask.g. This produces coupling between different parts of the code, makes it hard to change things and makes it hard to test components. For tiny programs you won’t even notice this risk, but for bigger projects this can creep up on you, and you may find yourself with code like this:

from functools import wraps

from flask import g, request
from flask.wrappers import Response


def auth(internal_only=False):
    def decorator(func):
        @wraps(func)
        def wrapper(*args, **kwargs):
            # implementation details of `parse_credentials` 
            # not important for this demo code
            credentials = parse_credentials(
                request.headers, internal_only
            )  
            g.credentials = credentials

            return func(*args, **kwargs)

        return wrapper
    return decorator

In the above code snippet, we have an example authentication decorator (Flask lends itself to decorators, so you’ll find yourself writing many in larger apps). We find ourselves bringing in flask.g to hang the credentials for a particular request on. This is the trap that is very easy to fall into as your Flask application grows.

One way to free yourself from needing to use Flask global objects is to use dependency injection.


Dependency Injection

Fundamentally, dependency injection (DI) is about decoupling a service or class from its dependencies by having other objects supply the service or class with those dependencies. This is as opposed to Service Location where the service/class actively finds its own dependencies. One of the definitive articles on this topic is this blog post by Martin Fowler.

The price you pay for adding dependency injection to your application is increased complexity. The pattern tends to be less critical in Python where keyword arguments, the ease of mocking and monkey patching give quite some flexibility. However, the pattern still offers a number of benefits in Python:

  • Clarity: Component dependencies are explicit (almost serves as documentation)
  • Ease of testing: Reduce the use of monkey patching
  • Decoupling: Less friction when swapping a particular dependency implementation (e.g. a database, HTTP client or even a simple config).

In practice, this means that dependency injection shines with larger applications with many dependencies. So how can we set this up in Flask?


Flask Injector

Flask-Injector is built on top of the Injector Python dependency injection framework. Now we will build a small example implementation of Flask-Injector. I’m limiting the dependencies to just redis to make the code as easy to follow as possible, and you can imagine how the dependencies could be far greater in number.

You can you also clone the example from the github repo. Be sure to check the readme for setup instructions. Once we’ve installed the requirements (notably, Flask, redis, Injector and Flask-Injector), we can have a look at creating a FlaskInjector instance. In our app.py file we will need a snippet like this:

# app.py (snippet)
import flask_injector

import providers


INJECTOR_DEFAULT_MODULES = dict(
    redis_client=providers.RedisClientModule(),
)

modules = dict(INJECTOR_DEFAULT_MODULES)

flask_injector.FlaskInjector(
    app=flask_app,
    modules=modules.values(),
)

# code continues...

In the code above we specify modules to be injected, in this case our RedisClientModule (we will go into the code for this soon), and pass these modules and our flask app instance into the FlaskInjector constructor.

Next, when we create our views we will specify which views will have dependencies injected:

# app.py
# other code...

@injector.inject
@app.route('/calculate')
def calculate(redis_client: StrictRedis):
    # redis dependency is injected
    redis_client.set('foo', 'bar')
    value = redis_client.get('foo')
    return value.decode('utf-8')

# other code...

The @injector.inject decorator automatically and transitively provides keyword arguments with their values. In the above case, our calculate view will have the redis client object instance from our providers.py module injected. Let’s now show the complete app.py file, with our app creation factory, views, and a few extra configuration parameters:

# app.py

import flask
import flask_injector
import injector
from redis import StrictRedis

import providers


INJECTOR_DEFAULT_MODULES = dict(
    redis_client=providers.RedisClientModule(),
)


def _configure_dependency_injection(
        flask_app, injector_modules, custom_injector
) -> None:
    modules = dict(INJECTOR_DEFAULT_MODULES)

    if injector_modules:
        modules.update(injector_modules)

    flask_injector.FlaskInjector(
        app=flask_app,
        injector=custom_injector,
        modules=modules.values(),
    )


def create_app(
        *,
        custom_injector: injector.Injector=None,
        injector_modules=None,
):
    app = flask.Flask(__name__)
    app.config.update(
        {'redis_port': 6379,
         'redis_host': 'localhost'}
    )

    @app.route('/')
    def hello_world():
        return 'hello world'

    @injector.inject
    @app.route('/calculate')
    def calculate(redis_client: StrictRedis):
        # redis dependency is injected
        redis_client.set('foo', 'bar')
        value = redis_client.get('foo')
        return value.decode('utf-8')

    _configure_dependency_injection(
        app, injector_modules, custom_injector)

    return app


if __name__=='__main__':
    application = create_app()
    application.run()

In app.py, we have a standard Flask create_app factory. Within this, we setup two simple views (in a larger project, obviously we’d put these in separate files/modules). The second route ‘/calculate’ has the @injector.inject decorator, which we discussed above.

As part of our app creation, we specify _configure_dependency_injection which is a helper function to instantiate the flask_injector.FlaskInjector. This object initializes Injector for the application, and needs to be called after all views, signal handlers, template globals and context processors are registered. We pass in our the flask app we’ve created, the modules we wish to inject (in the above case, this is just redis_client). We do not specify an Injector param, so a new one is created. The custom_injector option comes in useful for testing (more on this later).

Meanwhile, let’s have a look at the provider code:

# providers.py

import flask
import flask_injector
import injector
import redis


class RedisClientModule(injector.Module):

    def configure(self, binder):
        binder.bind(redis.StrictRedis,
                    to=self.create,
                    scope=flask_injector.request)

    # ... class continues

In the above provider code, we bind the StrictRedis object to the create method. For redis, we bind to the request scope. Let’s look at the create method:

    # RedisClientModule code...

    @injector.inject
    def create(
            self,
            config: flask.Config,
    ) -> redis.StrictRedis:
        return redis.StrictRedis(host=config.get('redis_host'),
                                 port=config.get('redis_port'))

Notice above we inject another dependency to the RedisClientModule create method (hence the presence of the @injector.inject decorator) - this time the dependency is flask.Config. Flask-Injector binds key flask dependencies by default, so this is already available without us needing to specify it explicitly. Here’s the link to the sourcecode. The Config dependency is bound to a singleton scope.

That’s our toy app setup and ready to run with dependency injection.


Testing with Flask-Injector

One of the areas where DI really shines is during testing. Here’s how we can setup our pytest tests using our app’s Flask-Injector setup:

# conftest.py
import fakeredis
import injector
import pytest

from ..app import create_app

test_config = {}


@pytest.fixture
def default_dependencies():
    return {
        'redis_client': fakeredis.FakeStrictRedis,
    }


@pytest.fixture
def test_injector():
    return injector.Injector()

# conftest.py continues...

These fixtures can now be passed to our create_app factory to create a test app instance with highly configurable dependencies.

# conftest.py

@pytest.fixture
def app(default_dependencies, test_injector):
    app = create_app(
        custom_injector=test_injector,
        injector_modules=default_dependencies,
    )
    app.testing = True

    with app.app_context():
        yield app


@pytest.fixture
def flask_test_client(app):
    with app.test_client() as test_client:
        yield test_client

Here we are able to easily adjust dependencies for testing using the default_dependencies fixture. In this case, we use a fakeredis instance for testing, but this could easily also be a mock. This same approach could be taken with other dependencies (databases, caches, objects, config etc.) Dependency injection makes mocking very easy during testing, and prevents the need for extensive monkey patching.

This just leaves us with a simple test:


def test_calculate_returns_200(flask_test_client):
    # When
    subject = flask_test_client.get('/calculate')

    # Then
    assert subject.status_code == 200

In this simple example, we’ve just looked an injecting dependencies into views. Flask-Injector also lets you inject dependencies into:

  • before_request handlers
  • after_request handlers
  • teardown_request handlers
  • template context processors
  • error handlers
  • Jinja environment globals (functions in app.jinja_env.globals)
  • Flask-RESTFul Resource constructors
  • Flask-RestPlus Resource constructors

Gotchas / Adjustments

  • 500 errors when there is an error in your provider code (even if it is because authentication failed). Solution: use Flask error handlers to address custom errors.
  • As mentioned earlier, annotations can be used in place of the inject decorator


Next Steps

Checkout the Injector docs, since it completely underpins Flask-Injector.

Then the source code for Flask-Injector is just a few files and very easy to follow.

Category

Tags