Table of Contents¶

Django-db-mutex Documentation¶

Django-db-mutex provides the ability to acquire a mutex lock from the database in Django.

Overview¶

For critical pieces of code that cannot overlap with one another, it is often necessary to acquire a mutex lock of some sort. Many solutions use a memcache lock strategy, however, this strategy can be brittle in the case of memcache going down or when an unconsistent hashing function is used in a distributed memcache setup.

If your application does not need a high performance mutex lock, Django DB Mutex does the trick. The common use case for Django DB Mutex is to provide the abilty to lock long-running periodic tasks that should not overlap with one another. Celery is the common backend for Django when scheduling periodic tasks.

Installation¶

Install Django-db-mutex with your favorite Python package manager:

# Using pip
pip install django-db-mutex

# Or, using pip (from source, in editable form)
pip install -e git://github.com/ambitioninc/django-db-mutex.git#egg=django-db-mutex

Add 'db_mutex' to your INSTALLED_APPS setting:

INSTALLED_APPS = (
    # other apps
    'db_mutex',
)

Examples¶

How to Use Django DB Mutex¶

The Django DB Mutex app provides a context manager and function decorator for locking a critical section of code. The context manager is used in the following way:

from db_mutex import DBMutexError, DBMutexTimeoutError
from db_mutex.db_mutex import db_mutex

# Lock a critical section of code
try:
    with db_mutex('lock_id'):
        # Run critical code here
        pass
except DBMutexError:
    print('Could not obtain lock')
except DBMutexTimeoutError:
    print('Task completed but the lock timed out')

You’ll notice that two errors were caught from this context manager. The first one, DBMutexError, is thrown if the lock cannot be acquired. The second one, DBMutexTimeoutError, is thrown if the critical code completes but the lock timed out. More about lock timeout in the next section.

The db_mutex decorator can also be used in a similar manner for locking a function:

from db_mutex import DBMutexError, DBMutexTimeoutError
from db_mutex.db_mutex import db_mutex

@db_mutex('lock_id')
def critical_function():
    pass

try:
    critical_function()
except DBMutexError:
    print('Could not obtain lock')
except DBMutexTimeoutError:
    print('Task completed but the lock timed out')

Lock Timeout¶

Django DB Mutex comes with lock timeout baked in. This ensures that a lock cannot be held forever. This is especially important when working with segments of code that may run out of memory or produce errors that do not raise exceptions.

In the default setup of this app, a lock is only valid for 30 minutes. As shown earlier in the example code, if the lock times out during the execution of a critical piece of code, a DBMutexTimeoutError will be thrown. This error basically says that a critical section of your code could have overlapped (but it doesn’t necessarily say if a section of code overlapped or didn’t).

In order to change the duration of a lock, set the DB_MUTEX_TTL_SECONDS variable in your settings.py file to a number of seconds. If you want your locks to never expire (beware!), set the setting to None.

Usage with Celery¶

Django DB Mutex can be used with celery’s tasks in the following manner:

from db_mutex import DBMutexError, DBMutexTimeoutError
from db_mutex.db_mutex import db_mutex
from abc import ABCMeta
from celery import Task

class NonOverlappingTask(Task):
    __metaclass__ = ABCMeta

    def run_worker(self, *args, **kwargs):
        """
        Run worker code here.
        """
        raise NotImplementedError()

    def run(self, *args, **kwargs):
        try:
            with db_mutex(self.__class__.__name__):
                self.run_worker(*args, **kwargs):
        except DBMutexError:
            # Ignore this task since the same one is already running
            pass
        except DBMutexTimeoutError:
            # A task ran for a long time and another one may have overlapped with it. Report the error
            pass

Code Documentation¶

db_mutex¶

DBMutex Model¶

Exceptions¶

class db_mutex.exceptions.DBMutexError¶: Thrown when a lock cannot be acquired.

class db_mutex.exceptions.DBMutexTimeoutError¶: Thrown when a lock times out before it is released.

Contributing¶

Contributions and issues are most welcome! All issues and pull requests are handled through github on the ambitioninc repository. Also, please check for any existing issues before filing a new one. If you have a great idea but it involves big changes, please file a ticket before making a pull request! We want to make sure you don’t spend your time coding something that might not fit the scope of the project.

Running the tests¶

To get the source source code and run the unit tests, run:

$ git clone git://github.com/ambitioninc/django-db-mutex.git
$ cd django-db-mutex
$ virtualenv env
$ . env/bin/activate
$ python setup.py install
$ coverage run setup.py test
$ coverage report --fail-under=100

While 100% code coverage does not make a library bug-free, it significantly reduces the number of easily caught bugs! Please make sure coverage is at 100% before submitting a pull request!

Code Quality¶

For code quality, please run flake8:

$ pip install flake8
$ flake8 .

Code Styling¶

Please arrange imports with the following style

# Standard library imports
import os

# Third party package imports
from mock import patch
from django.conf import settings

# Local package imports
from db_mutex.version import __version__

Please follow Google’s python style guide wherever possible.

Building the docs¶

When in the project directory:

$ pip install -r requirements/docs.txt
$ pip uninstall -y django-db-mutex && python setup.py install
$ cd docs && make html
$ open docs/_build/html/index.html

Release Checklist¶

Before a new release, please go through the following checklist:

Bump version in db_mutex/version.py
Git tag the version
Add a release note in docs/release_notes.rst
Upload to pypi

Vulnerability Reporting¶

For any security issues, please do NOT file an issue or pull request on github! Please contact security@ambition.com with the GPG key provided on Ambition’s website.

Release Notes¶

v1.1.0¶

Add tox to support more versions

v1.0.0¶

Drop Django 1.9 support
Drop Django 1.10 support
Add Django 2.0 support
Drop python 2.7 support
Drop python 3.4 support

v0.5.0¶

Add python 3.6 support
Drop Django 1.8 support
Add Django 1.10 support
Add Django 1.11 support

v0.4.0¶

Add python 3.5 support
Drop Django 1.7 support

v0.3.1¶

Fixed docs

v0.3.0¶

Added django 1.9 support

v0.2.0¶

Dropped Django 1.6, added 1.8 support

v0.1.8¶

Fixed migrations to include south_migrations

v0.1.7¶

Fixed upload to pypi

v0.1.6¶

Updated for Django 1.7 compatibility

v0.1.4¶

This release of django-db-mutex includes docs buildout
python 3.3, 3.4 compatibility