Jina-Serve

Jina AI Cloud serves as the central portal and single entrypoint for managing your entire Jina AI ecosystem. You can perform CRUD (Create, Read, Update, Delete) operations, manage access control, handle personal access tokens, and manage subscriptions for the following resources:

• Data: Includes docarray stores and Finetuner artifacts.
• Executors: The individual units of computation in your services.
• Flows: The orchestrated pipelines that connect Executors.
• Apps: Managed via the Jina Apps platform.

An jina.Executor is a self-contained microservice exposed via gRPC or HTTP. It is a Python class that processes Documents (using docarray.BaseDoc and docarray.DocList as data structures).

Key principles:

1. Subclassing: Always subclass directly from jina.Executor.
2. Methods: An Executor can contain any number of functions.
3. Endpoints: Functions decorated with @requests are exposed as services. These can be regular functions or coroutines (async def) and can handle single Documents or batches.
4. State (Beta): Functions decorated with @write (placed above the @requests decoration) are considered to update the Executor's internal state.

A jina.Flow is an orchestration layer that connects multiple jina.Executors into a processing pipeline. Documents flow through this pipeline to be processed by the Executors.

Key characteristics:

• Orchestration: It acts as an interface to configure and launch a microservice architecture.
• Gateway: Every Flow automatically launches a Gateway service, which exposes the underlying services through an API (HTTP, gRPC, or WebSockets).
• Scalability: Flows allow you to scale individual Executors independently.
• Composition: A Flow is essentially a set of Deployments.

A jina.Deployment is an orchestration component that manages a single jina.Executor (or a set of replicated Executors) to accomplish a task. While Executors do the heavy lifting of processing Documents, the Deployment acts as the interface to configure and launch your microservice architecture.

Key benefits:

• Scaling: Deployments allow you to scale Executors independently based on your requirements.
• Orchestration: They enable easy integration with cloud-native orchestrators like Kubernetes to manage your services.

Executor Hub is a service that allows you to containerize, reuse, and share jina.Executor instances. It functions similarly to a Docker registry, enabling you to:

• Pull prebuilt Executors: Use existing services to reduce development complexity.
• Push custom Executors: Share your own Executors privately or publicly.
• Streamline containerization: Turn your jina.Executor into a ready-for-the-cloud containerized service automatically.

Dynamic batching allows Jina to accumulate incoming requests and group them into a single batch before sending them to a jina.Executor.

This is primarily used for inference tasks where model inference is more efficient when processed in batches (e.g., utilizing GPU resources).

How it works: Incoming requests with the same request parameters are queued together. The batch is dispatched to the Executor when either:

1. The number of accumulated Documents reaches the preferred_batch_size.
2. The timeout duration is exceeded.

Note: While it can work with parametrized requests, it is best suited for endpoints that receive consistent parameters.

Jina-serve collects specific metadata to understand usage patterns and improve features. The collected data includes:

• Versions of jina-serve and its dependencies;
• A hashed unique user identifier (uid);
• A hashed unique session identifier (session-id);
• Boolean events marking the start of components like a Flow, Gateway, Runtime, or Client.

Jina-serve automatically detects the installed version of DocArray and adapts its internal methods and APIs accordingly. However, developers must write their Executors and Clients differently depending on whether they are using docarray < 0.30 or docarray >= 0.30.

• DocArray < 0.30: Uses a fixed Document schema. The user must adapt their data to this predefined schema.
• DocArray >= 0.30: Uses a flexible schema built on top of Pydantic. Users define their own schemas by subclassing BaseDoc, allowing the data to adapt to the user's needs rather than forcing a specific structure.