UtilMeta Python Meta-Framework

repository·main·Indexed 42 days ago

https://github.com/utilmeta/utilmeta-py

A progressive meta-framework for API development in Python that uses type annotations to automate request validation, ORM query optimization, and OpenAPI documentation. It is compatible with major web frameworks including Django, FastAPI, Flask, Sanic, and Tornado.

Tokens
75.8K
Snippets
224
Records
257
Agent score
51%

What's inside UtilMeta

  1. Overview of UtilMeta API & Operations

    main

    The UtilMeta framework includes a built-in API service management system designed to observe and manage both local and online API services. This system provides a suite of operational features including:

    • Data Management: CRUD operations on data.
    • API Management: Oversight of available API services.
    • Observability: Log querying and system monitoring.
    • Quality Assurance: API testing.
    • Reliability: Alerts and incident management.
  2. Understand API Path Declaration rules

    main

    UtilMeta determines the request path for an API function based on the following priority:

    1. Template String: The path provided in the @api decorator (e.g., @api.get('article/{id}')).
    2. Function Name: If no path is provided in the decorator, the function name is used as the path.
    3. HTTP Method Name: If a function name is a valid HTTP method (e.g., get, post, put), its path is automatically set to '/' and cannot be overridden.

    Path Match Priority: When defining multiple routes, place fixed paths (e.g., /article/feed) above variable path parameters (e.g., /article/{slug}) in your class definition. This ensures UtilMeta matches the specific route before falling back to the variable one.

    from utilmeta.core import api
    
    @api.route('/article')
    class ArticleAPI(api.API):
        @api.get
        def feed(self): pass
        # GET /article/feed
        
        @api.get('{slug}')
        def get_article(self, slug: str): pass
        # GET /article/{slug}
        
        def get(self, id: int): pass
        # GET /article?id=<ID>
  3. Use UtilMeta Declarative ORM to define data structures

    main

    UtilMeta's declarative ORM allows you to define the expected structure of query results using a Schema class. By inheriting from orm.Schema[<ModelClass>], you map your database models to a structured schema that controls how data is serialized for clients.

    Key behaviors:

    • Field Mapping: If a field name in the Schema matches a field in the model, it is serialized to the declared type.
    • Relationship Serialization: For foreign keys or relationships, you can specify another Schema class to serialize the entire related object instead of just the ID. This is useful for reducing API calls (e.g., returning an author object within an article response).
    • Expression Fields: You can define fields that represent database expressions (like counts) using model-level functions.
    from utilmeta.core import orm
    from .models import User, Article
    from django.db import models
    
    # Define a schema for the User model
    class UserSchema(orm.Schema[User]):
        username: str
        # An expression field querying the count of related 'articles'
        articles_num: int = models.Count('articles')
    
    # Define a schema for the Article model with nested serialization
    class ArticleSchema(orm.Schema[Article]):
        id: int
        # Serializes the entire related User object using UserSchema
        author: UserSchema
        content: str
  4. Choose a Session storage strategy

    main

    UtilMeta provides several session storage strategies depending on your requirements for speed and persistence:

    • Cache: High performance, uses a cache like Redis. Import CacheSession from utilmeta.core.auth.session.cache.
    • Database: Persistent storage using a database. Import DBSession from utilmeta.core.auth.session.db.
    • Cache + Database (fallback): Hybrid approach. Queries the cache first; if it misses or fails, it falls back to the database. Import CachedDBSession from utilmeta.core.auth.session.cached_db.
    from utilmeta.core.auth.session.cache import CacheSession
    from utilmeta.core.auth.session.db import DBSession
    from utilmeta.core.auth.session.cached_db import CachedDBSession
  5. Implement Dynamic Fields in Schemas using get_runtime

    main

    When a schema needs a field that depends on the context of the current request (e.g., a following boolean that changes based on whether the requesting user follows the target user), use a get_runtime class method.

    This method should return a new subclass of the original schema that overrides the field with a dynamic query expression (using orm.Field and expressions).

    class ProfileSchema(UserBase):
        following: bool = False
    
        @classmethod
        def get_runtime(cls, user):
            if not user:
                return cls
    
            class ProfileRuntimeSchema(cls):
                following: bool = orm.Field(
                    exp.Exists(
                        Follow.objects.filter(
                            following=exp.OuterRef('pk'), 
                            follower=user
                        )
                    )
                )
    
            return ProfileRuntimeSchema
    
    # Usage in API:
    # await ProfileSchema.get_runtime(current_user).ainit(target_instance)
  6. Use Generic and Decorator Hook functions

    main

    Hooks allow you to intercept and modify requests, responses, or errors.

    Generic Hooks

    Override these methods in your Client class for global processing:

    • process_request(self, request: Request) -> Request: Modify the request. Returning a Response instance skips the network call and returns that response directly.
    • process_response(self, response: Response) -> Response: Modify the response. Returning a Request instance triggers a retry/redirect.
    • handle_error(self, error: Error): Handle errors. Returning a Response allows you to return a custom response instead of throwing.

    Decorator Hooks

    Use these for fine-grained control over specific functions:

    • @api.before(func): Pre-processes the request before it is called. The first argument must be the request.Request object.
    • @api.after(func): Processes the response after the call.
    • @api.handle(func): Handles errors thrown during the call chain.

    Execution Order for Default Functions:

    1. @api.before $\rightarrow$ 2. process_request $\rightarrow$ 3. Network Request $\rightarrow$ 4. process_response $\rightarrow$ 5. @api.after.
    class GithubClient(cli.Client):
        @api.get('https://api.github.com/user')
        async def get_user(self) -> response.Response: pass
        
        @api.before(get_user)
        def add_authorization(self, req: request.Request):
            req.headers['Authorization'] = f'token {self.token}'
  7. Declare response templates with `response.Response`

    main

    Use response templates to parse API responses automatically. Declare the template as the return type hint of your request function. The template must inherit from response.Response or be a Union of multiple response classes.

    Key properties for response templates:

    • status: The expected HTTP status code. The client only parses the response into this template if the status matches.
    • result: The type hint for the parsed result data.
    • headers: The type hint for response headers (using a Schema).
    • result_key: The key in the JSON response containing the actual data (e.g., if your API returns {"data": {...}}, set result_key = 'data').
    • message_key: The key for error messages.
    • state_key: The key for business/status codes.
    • count_key: The key for pagination totals.

    Access the unparsed raw response body via response.data.

    from utilmeta.core import cli, api, request, response
    import utype
    
    class UserSchema(utype.Schema):
        id: int
        username: str
    
    class UserResponse(response.Response):
        status = 200
        result: UserSchema
    
    class UserClient(cli.Client):
        @api.post
        def login(
            self, 
            username: str = request.BodyParam,
            password: str = request.BodyParam,
        ) -> UserResponse: pass
  8. Handle multiple response types with `Union`

    main

    If an API returns different structures based on the status code (e.g., success vs. error), use typing.Union in the return type hint. The Client will attempt to parse the response using the templates in the order they are declared in the Union.

    If no template matches, the client will throw an error unless you include response.Response as the final element in the Union to catch all remaining cases.

    from typing import Union
    from utilmeta.core import cli, api, response
    
    class UserResponse(response.Response):
        status = 200
        result: UserSchema
    
    class UserResponseFailed(response.Response):
        status = 403
        message_key = 'error'
    
    class UserClient(cli.Client):
        @api.post
        def login(self, ...) -> Union[UserResponse, UserResponseFailed, response.Response]: pass
  9. Configure the HTTP Runtime Backend

    main

    UtilMeta is a progressive meta-framework that supports multiple Python web frameworks as runtimes. You can switch the underlying implementation by passing the framework module to the backend argument in the UtilMeta constructor. Supported backends include:

    • Django (with Django REST framework)
    • Flask (with APIFlask)
    • FastAPI (with Starlette)
    • Sanic
    • Tornado
    from utilmeta import UtilMeta
    from utilmeta.core import api
    import django
    
    class RootAPI(api.API):
        @api.get
        def hello(self):
            return 'world'
    
    service = UtilMeta(
        __name__,
        name='demo',
        backend=django,    # or flask / starlette / tornado / sanic
        api=RootAPI,
        route='/api'
    )
    
    app = service.application()  # returns a WSGI app
    
    if __name__ == '__main__':
        service.run()
  10. Control field visibility with Field Modes

    main

    The mode parameter in orm.Field controls when a field is used during different CRUD operations:

    • 'r' (Read/Retrieve): Returned in query results.
    • 'w' (Write/Update): Used to update existing records.
    • 'a' (Add/Create): Used when creating new records.

    You can combine modes (e.g., 'rwa'). Use no_input=True to allow a field to be set by the developer in code (like an author_id from the current session) while preventing it from being accepted from client request data.

    Predefined Schema Modes: UtilMeta provides shorthand modes for common operations:

    • orm.A: Mode 'a' (Creation/POST).
    • orm.W: Mode 'w' (Update/PUT).
    • orm.WP: Mode 'w' that ignores required attributes (Partial Update/PATCH).
  11. Define general parameters for an API class

    main

    If multiple endpoints within an api.API class require the same parameter (such as a path parameter defined in the class route), you can declare it as a class property. This makes the parameter available to all methods in the class without repeating the declaration in every function signature.

    To define a general parameter, assign a value from utilmeta.core.request to a class property. Common request parameter types include:

    • request.PathParam: For path parameters.
    • request.QueryParam: For query string parameters.
    • request.HeaderParam: For request header parameters.
    • request.CookieParam: For cookie parameters.

    Example: Using @api.route to define a path template and capturing a slug for all methods in the class.

    from utilmeta.core import api, request
    
    @api.route('{slug}/comments')
    class CommentAPI(api.API):
        slug: str = request.PathParam
    
        def __init__(self, *args, **kwargs):
            super().__init__(*args, **kwargs)
            # self.slug is now available for use in __init__ and all methods
            self.article = Article.objects.filter(slug=self.slug).first()
    
        def get(self):
            return CommentSchema.serialize(Comment.objects.filter(article=self.article))
  12. Use API Hooks for Request Lifecycle Management

    main

    UtilMeta API classes support hooks to execute logic before or after endpoints. Use @api.before('*') to run logic before all methods in a class (e.g., to fetch a resource based on a path parameter and attach it to self), or @api.before(method_name) to target specific methods. This is useful for pre-loading data or validating existence.

    @api.route('profiles/{username}')
    class ProfileAPI(API):
        @api.before('*')
        async def handle_profile(self):
            profile = await User.objects.filter(username=self.username).afirst()
            if not profile:
                raise exceptions.NotFound(f'profile({repr(self.username)}) not found')
            self.profile = profile
    
        @api.get
        async def get(self, user: Optional[User] = API.user_config):
            return await ProfileSchema.ainit(self.profile)