django-multitenant

repository·main·Indexed 21 days ago

https://github.com/citusdata/django-multitenant

A Python/Django library providing support for distributed multi-tenant databases like Postgres+Citus. It implements a shared table architecture that automatically adds tenant context (tenant_id) to queries for efficient routing and isolation. Key features include TenantModel and TenantModelMixin for model definition, TenantForeignKey for composite foreign keys, and integration with Django Rest Framework via TenantModelViewSet.

Tokens
10.1K
Snippets
26
Records
30
Agent score
74%

What's inside django-multitenant

  1. Overview of django-multitenant architecture

    main

    django-multitenant is designed for Python/Django applications using distributed multi-tenant databases, such as Postgres with Citus. It enables horizontal scale-out by automatically adding a tenant context to your queries, allowing the database to efficiently route queries to the correct node.

    This library implements the shared table architecture, where all tenants share the same tables. It assumes that all models related to a tenant include a tenant_id column to represent the tenant context.

  2. Handle ForeignKey and OneToOneField constraints

    main

    When migrating models to a multi-tenant architecture, you must choose the correct field type based on the relationship between tables:

    1. Between distributed tables: Use TenantForeignKey or TenantOneToOneField. This ensures the foreign key is composite (including the tenant ID).
    2. Between a distributed table and a reference table: Use standard Django ForeignKey. No changes are required.
    3. Between a distributed table and a local table: Use a standard ForeignKey but set db_constraint=False to drop the database-level constraint, as Citus cannot enforce constraints between distributed and local tables.

    Note on Citus 11.3+: Identity columns for distributed tables must be bigint. TenantModel handles this automatically for new models. For existing models, you must manually update the identity column to bigint and run a migration.

    from django.db import models
    from django_multitenant.fields import TenantForeignKey
    from django_multitenant.models import TenantModel
    
    class Country(models.Model):  # Reference table
        name = models.CharField(max_length=255)
    
    class Account(TenantModel):
        name = models.CharField(max_length=255)
        country = models.ForeignKey(Country, on_delete=models.SET_NULL)  # Standard ForeignKey for reference tables
    
        class TenantMeta:
            tenant_field_name = 'id'
    
    class Task(TenantModel):
        name = models.CharField(max_length=255)
        project = TenantForeignKey(Project, on_delete=models.CASCADE)  # TenantForeignKey for distributed tables
        account = models.ForeignKey(Account, on_delete=models.CASCADE)
    
        class TenantMeta:
            tenant_field_name = 'account_id'
  3. Handle ManyToMany constraints with through models

    main

    In a Citus-backed multi-tenant application, ManyToMany relationships require a through model that includes the tenant column. This ensures the relationship is correctly distributed across shards. Use TenantForeignKey for the fields within the through model.

    class ProjectManager(TenantModel):
        project = TenantForeignKey(Project, on_delete=models.CASCADE)
        manager = TenantForeignKey(Manager, on_delete=models.CASCADE)
        account = models.ForeignKey(Account, on_delete=models.CASCADE)
    
        class TenantMeta:
            tenant_field_name = 'account_id'
    
    class Project(TenantModel):
        account = models.ForeignKey(Account, related_name='projects', on_delete=models.CASCADE)
        managers = models.ManyToManyField(Manager, through='ProjectManager')
        
        class TenantMeta:
            tenant_field_name = 'account_id'
  4. Install django-multitenant and configure the database engine

    main

    To migrate a multi-tenant Django application to use django-multitenant, follow these setup steps:

    1. Add django_multitenant>=2.0.0, <3 to your requirements.txt.
    2. Install the dependency using pip install -r requirements.txt.
    3. Update your settings.py to use the customized PostgreSQL engine provided by the library.

    This engine is required to support the multi-tenant features and Citus integration.

    # In requirements.txt
    django_multitenant>=2.0.0, <3
    
    # In settings.py
    'ENGINE': 'django_multitenant.backends.postgresql'
  5. Implement multi-tenancy using TenantModelMixin

    main

    If you prefer using mixins, follow these steps:

    1. Import django_multitenant.mixins.
    2. Inherit from TenantModelMixin and models.Model.
    3. Define the tenant column using a static variable tenant_id='column_name'.
    4. Use TenantForeignKey for foreign keys to other tenant-aware models.
    5. Ensure the referenced table includes a unique_together constraint that combines the primary key and the tenant_id.

    Note: You can also use TenantManagerMixin with your model's manager to support tenant-aware querying.

    from django_multitenant.mixins import *
    
    class ProductManager(TenantManagerMixin, models.Manager):
        pass
    
    class Product(TenantModelMixin, models.Model):
        store = models.ForeignKey(Store)
        tenant_id='store_id'
        name = models.CharField(max_length=255)
        description = models.TextField()
    
        objects = ProductManager()
    
        class Meta:
            unique_together = ["id", "store"]
    
    class PurchaseManager(TenantManagerMixin, models.Manager):
        pass
    
    class Purchase(TenantModelMixin, models.Model):
        store = models.ForeignKey(Store)
        tenant_id='store_id'
        product_purchased = TenantForeignKey(Product)
    
        objects = PurchaseManager()
  6. Introduce tenant columns to models

    main

    To enable efficient routing and sharding, every model belonging to a tenant must include a tenant column (e.g., account_id). This allows queries to include the tenant ID in the WHERE clause, enabling the database to quickly locate all records for a specific account.

    For standard models

    Add a ForeignKey to the tenant model directly to the child model.

    class Task(models.Model):
        name = models.CharField(max_length=255)
        project = models.ForeignKey(Project, on_delete=models.CASCADE, related_name='tasks')
        # Add the tenant column
        account = models.ForeignKey(Account, related_name='tasks', on_delete=models.CASCADE)

    For ManyToMany models

    To distribute ManyToManyField relationships, you must use an explicit through model that includes the tenant column. This ensures that queries involving the relationship can be routed via the account_id.

    class Project(models.Model):
        name = models.CharField(max_length=255)
        account = models.ForeignKey(Account, related_name='projects', on_delete=models.CASCADE)
        # Use a 'through' model
        managers = models.ManyToManyField(Manager, through='ProjectManager')
    
    class ProjectManager(models.Model):
        project = models.ForeignKey(Project, on_delete=models.CASCADE)
        manager = models.ForeignKey(Manager, on_delete=models.CASCADE)
        # Include the tenant column in the through model
        account = models.ForeignKey(Account, on_delete=models.CASCADE)
    class Task(models.Model):
        name = models.CharField(max_length=255)
        project = models.ForeignKey(Project, on_delete=models.CASCADE, related_name='tasks')
        account = models.ForeignKey(Account, related_name='tasks', on_delete=models.CASCADE)
    
    class Project(models.Model):
        name = models.CharField(max_length=255)
        account = models.ForeignKey(Account, related_name='projects', on_delete=models.CASCADE)
        managers = models.ManyToManyField(Manager, through='ProjectManager')
    
    class ProjectManager(models.Model):
        project = models.ForeignKey(Project, on_delete=models.CASCADE)
        manager = models.ForeignKey(Manager, on_delete=models.CASCADE)
        account = models.ForeignKey(Account, on_delete=models.CASCADE)
  7. Set the Current Tenant via Middleware

    main

    To avoid manually setting the tenant in every view, implement a middleware that sets the tenant based on the authenticated user's account.

    Important: You must call unset_current_tenant() at the end of the request cycle. Because the tenant is stored as a thread-local variable, failing to unset it can lead to data leakage between requests on the same web server thread.

    Steps:

    1. Create a middleware using set_current_tenant and unset_current_tenant from django_multitenant.utils.
    2. Add your middleware to the MIDDLEWARE list in settings.py.
    from django_multitenant.utils import set_current_tenant, unset_current_tenant
    from django.contrib.auth import logout
    
    class MultitenantMiddleware:
        def __init__(self, get_response):
            self.get_response = get_response
    
        def __call__(self, request):
            if request.user and not request.user.is_anonymous:
                if not request.user.account and not request.user.is_superuser:
                    logout(request.user)
                set_current_tenant(request.user.account)
    
            response = self.get_response(request)
            
            # Essential to prevent data leakage in thread-local storage
            unset_current_tenant()
            return response
  8. Create a custom middleware for tenant resolution

    main

    If you need highly customized logic for determining the tenant (e.g., retrieving it from a session variable or a specific user attribute), you can implement your own Django middleware class. Inside the middleware's __call__ method, use django_multitenant.utils.set_current_tenant to apply the tenant to the current context.

    # src/appname/middleware.py
    from django_multitenant.utils import set_current_tenant
    
    class MultitenantMiddleware:
        def __init__(self, get_response):
            self.get_response = get_response
    
        def __call__(self, request):
            if request.user and not request.user.is_anonymous:
                # Your custom logic to set the current tenant
                current_tenant = your_method(request)
                set_current_tenant(current_tenant)
            
            response = self.get_response(request)
            return response
    
    # In settings.py
    MIDDLEWARE = [
        # ...
        'appname.middleware.MultitenantMiddleware'
    ]
  9. Integrate django-multitenant with Django Rest Framework

    main

    To use django-multitenant with Django Rest Framework (DRF), you must configure the middleware, define how the tenant is identified from a request, and use the provided TenantModelViewSet to ensure automatic query scoping.

    1. Configure Middleware

    Add 'django_multitenant.middleware.MultitenantMiddleware' to your MIDDLEWARE setting in settings.py.

    2. Define Tenant Resolution via Monkey Patching

    Since DRF views may not use the standard Django view resolution path for tenant identification, you must monkey patch the django_multitenant.views.get_tenant function. This function should accept a request and return the appropriate tenant object.

    3. Use TenantModelViewSet

    Derive your viewsets from TenantModelViewSet. This ensures that all queries within the viewset are automatically scoped to the current tenant found in request.tenant. You do not need to manually filter the queryset by the tenant.

    # 1. settings.py
    MIDDLEWARE = [
        # other middleware
        'django_multitenant.middleware.MultitenantMiddleware',
    ]
    
    # 2. views.py
    def tenant_func(request):
        # Return the tenant object based on the request
        return Store.objects.filter(user=request.user).first()
    
    # Monkey patching get_tenant function
    from django_multitenant import views
    views.get_tenant = tenant_func
    
    # 3. views.py
    class StoreViewSet(TenantModelViewSet):
        """
        API endpoint that allows groups to be viewed or edited.
        """
        model_class = Store
        serializer_class = StoreSerializer
        permission_classes = [permissions.IsAuthenticated]
  10. Distribute tables in Citus via migrations

    main

    After updating your models, you must create an empty migration to instruct Citus to mark tables for distribution. Use tenant_migrations.Distribute within your migration's operations list.

    • Use reference=True for reference tables (tables that are not distributed).
    • Omit reference=True for distributed tables.

    Troubleshooting Citus 11+ Migrations: If you encounter ERROR: cannot run type command because there was a parallel operation on a distributed table in the transaction, wrap your operations in a RunSQL command to set the mode to sequential:

    operations = [
        migrations.RunSQL("SET LOCAL citus.multi_shard_modify_mode TO 'sequential';"),
        tenant_migrations.Distribute('MyTable'),
    ]
    from django.db import migrations
    from django_multitenant.db import migrations as tenant_migrations
    
    class Migration(migrations.Migration):
        operations = [
            tenant_migrations.Distribute('Country', reference=True),
            tenant_migrations.Distribute('Account'),
            tenant_migrations.Distribute('Manager'),
            tenant_migrations.Distribute('Project'),
            tenant_migrations.Distribute('ProjectManager'),
            tenant_migrations.Distribute('Task'),
        ]
  11. Implement multi-tenancy using TenantModel

    main

    You can implement multi-tenancy by having your models inherit from TenantModel.

    Steps:

    1. Import django_multitenant.fields and django_multitenant.models.
    2. Inherit from TenantModel.
    3. Define the tenant column using a TenantMeta inner class with either tenant_field_name or tenant_id.
    4. Use TenantForeignKey instead of models.ForeignKey for all foreign keys pointing to other TenantModel subclasses.

    Warning: Avoid defining a field named tenant_id directly in the class to prevent collisions with the library's internal logic.

    from django_multitenant.fields import *
    from django_multitenant.models import *
    
    class Store(TenantModel):
        name = models.CharField(max_length=50)
        address = models.CharField(max_length=255)
        email = models.CharField(max_length=50)
        class TenantMeta:
            tenant_field_name = "id"
    
    class Product(TenantModel):
        store = models.ForeignKey(Store)
        name = models.CharField(max_length=255)
        description = models.TextField()
        class Meta:
            unique_together = ["id", "store"]
        class TenantMeta:
            tenant_field_name = "store_id"
    
    class Purchase(TenantModel):
        store = models.ForeignKey(Store)
        product_purchased = TenantForeignKey(Product)
        class TenantMeta:
            tenant_field_name = "store_id"