PipServices Documentation

website·Indexed 31 days ago

http://docs.pipservices.org/

Documentation and tutorials for building microservices with PipServices v4, featuring guides on getting started, messaging (MQTT, Kafka), data validation, and distributed locks using Redis. Supports multiple languages including Node.js, Python, Go, Dart, and .NET.

Tokens
1.2M
Snippets
5.6K
Records
5.6K
Agent score
49%

What's inside PipServices

  1. Understand Pip.Services microservice architecture components

    v4

    A standard Pip.Services microservice typically consists of several core components to handle metadata, logging, networking, and health monitoring. Key components include:

    • ContextInfo: Provides the name and description of the microservice.
    • ConsoleLogger: Handles writing logs to stdout.
    • LogCounters: Manages the logging of performance counters.
    • HttpEndpoint: A shared component that allows multiple services to operate on a single HTTP port simultaneously.
    • HeartbeatRestService: Provides a GET /heartbeat endpoint to verify if the microservice is running.
    • StatusRestService: Provides a GET /status endpoint to retrieve the current status of the microservice.
    • Custom Controllers and REST Services: User-implemented logic (e.g., HelloWorldController and HelloWorldRestServices) that define the actual business functionality of the service.
  2. Use TypeDescriptor for type reflection in Python

    v4
    The TypeDescriptor class from pip_services4_commons.reflect is used to uniquely identify a type across different libraries. It stores the name of the type and the library it belongs to, which is essential for reflection and serialization tasks in pipservices.
    from pip_services4_commons.reflect import TypeDescriptor
    
    # Create a type descriptor for 'classA' in 'library1'
    type1 = TypeDescriptor("classA", "library1")
  3. Implement a ProcessContainer for a HelloFriend microservice

    v4
    In a PipServices three-tier architecture, the ProcessContainer acts as the entry point for a microservice. It defines the process name, the configuration file path, and registers the necessary service factories (the business logic) and communication factories (e.g., HTTP) required for the service to run.
    export class HelloFriendProcess extends ProcessContainer {
        public constructor() {
            super("hello-friend", "HelloFriend microservice");
            this._configPath = "./config.yaml"
            this._factories.add(new HelloFriendServiceFactory());
            this._factories.add(new DefaultHttpFactory());
        }
    }
  4. Implement a Three-Tier Architecture Factory in Python using PipServices

    v3
    In a PipServices three-tier architecture, a Factory class is used to decouple the service implementation from its definition. By using Descriptor objects, you can map specific roles (View/HTTP Service, Controller, and Persistence) to their respective implementation classes. This allows the application to swap implementations (e.g., changing MySQL persistence to MongoDB) without changing the business logic.
    from pip_services3_commons.refer import Descriptor
    from pip_services3_components.build import Factory
    
    class HelloFriendServiceFactory(Factory):
        def __init__(self):
            super(HelloFriendServiceFactory, self).__init__()
    
            # Define Descriptors for the three tiers
            # Descriptor(name, type, technology, version, version)
            HttpServiceDescriptor = Descriptor('hello-friend', 'service', 'http', '*', '1.0')  # View
            ControllerDescriptor = Descriptor('hello-friend', 'controller', 'default', '*', '1.0')  # Controller
            PersistenceDescriptor = Descriptor('hello-friend', 'persistence', 'mysql', '*', '1.0')  # Persistence
    
            # Register the implementation classes to the descriptors
            self.register_as_type(HttpServiceDescriptor, HelloFriendRestService)  # View implementation
            self.register_as_type(ControllerDescriptor, HelloFriendController)  # Controller implementation
            self.register_as_type(PersistenceDescriptor, HelloFriendPersistence)  # Persistence implementation
  5. Implement a HelloFriendController in Go for a three-tier architecture

    v3
    The HelloFriendController acts as the business logic layer in a three-tier architecture. It manages the coordination between the API layer and the persistence layer. It handles configuration via Configure, dependency injection of the persistence layer via SetReferences, and implements business methods like Greeting (which retrieves a random friend from persistence) and Create (which persists a new friend).
    type HelloFriendController struct {
    	defaultName string
    	persistence *HelloFriendPersistence
    }
    
    func NewHelloFriendController() *HelloFriendController {
    	c := &HelloFriendController{}
    	c.defaultName = "Pip User"
    	return c
    }
    
    func (c *HelloFriendController) Configure(ctx context.Context, config *cconf.ConfigParams) {
    	c.defaultName = config.GetAsStringWithDefault("default_name", c.defaultName)
    }
    
    func (c *HelloFriendController) SetReferences(ctx context.Context, references cref.IReferences) {
    	res, err := references.GetOneRequired(cref.NewDescriptor("hello-friend", "persistence", "*", "*", "1.0"))
    	if err != nil {
    		panic(err)
    	}
    
    	c.persistence = res.(*HelloFriendPersistence)
    }
    
    func (c *HelloFriendController) Greeting(ctx context.Context) (string, error) {
    	filter := cdata.NewFilterParamsFromTuples("type", "friend")
    	selectedFilter, err := c.persistence.GetOneRandom(ctx, "123", *filter)
    	if err != nil {
    		return "", err
    	}
    
    	return "Hello, " + selectedFilter.Name + "!", nil
    }
    
    func (c *HelloFriendController) Create(ctx context.Context, correlationId string, item MyFriend) (MyFriend, error) {
    	return c.persistence.Create(ctx, correlationId, item)
    }
  6. Implement a basic CRUD microservice with Pip.Services (Python)

    v4
    This example demonstrates how to build a microservice using the Pip.Services framework in Python, implementing a 'HelloFriend' service with a REST controller and pluggable persistence layers (MySQL and PostgreSQL). The architecture follows a Controller-Service-Persistence pattern where the specific database implementation is determined at runtime via configuration/environment variables.
    from pip_services4_data.data import IStringIdentifiable
    from pip_services4_http.controller import RestController
    from pip_services4_components.config import IConfigurable
    from pip_services4_components.refer import IReferences, IReferenceable, Descriptor
    from pip_services4_components.build import Factory
    from pip_services4_container.container import ProcessContainer
    from pip_services4_http.build import DefaultRpcFactory
    from pip_services4_mysql.persistence import IdentifiableMySqlPersistence
    from pip_services4_postgres.persistence import IdentifiablePostgresPersistence
    from pip_services4_data.query import FilterParams
    from typing import Optional
    from abc import ABC
    import bottle
    
    # 1. Data Model
    class MyFriend(IStringIdentifiable):
        def __init__(self, id: str, type: str, name: str):
            self.id = id
            self.type = type
            self.name = name
    
    # 2. Persistence Interface
    class IMyDataPersistence(ABC):
        def get_one_random(self, trace_id: Optional[str], filter: FilterParams) -> MyFriend:
            raise NotImplemented()
        def create(self, trace_id: Optional[str], item: MyFriend) -> MyFriend:
            raise NotImplemented()
    
    # 3. MySQL Implementation
    class HelloFriendPersistence1(IdentifiableMySqlPersistence, IMyDataPersistence):
        def __init__(self):
            super(HelloFriendPersistence1, self).__init__('myfriends3')
        def _define_schema(self):
            self._clear_schema()
            self._ensure_schema(f"CREATE TABLE IF NOT EXISTS `{self._table_name}` (id VARCHAR(32) PRIMARY KEY, `type` VARCHAR(50), `name` TEXT)")
        def _compose_filter(self, filter: FilterParams):
            filter = filter or FilterParams()
            type = filter.get_as_nullable_string('type')
            name = filter.get_as_nullable_string('name')
            condition = ''
            if type: condition += f"`type`='{type}'"
            if name: condition += f"`name`='{name}'"
            return condition
        def get_one_random(self, trace_id: str, filter: FilterParams) -> MyFriend:
            return super().get_one_random(trace_id, self._compose_filter(filter))
    
    # 4. Service Logic
    class HelloFriendService(IConfigurable, IReferenceable):
        def __init__(self):
            self.__persistence = None
        def set_references(self, references: IReferences):
            self.__persistence = references.get_one_required(Descriptor("hello-friend", "persistence", "*", "*", "1.0"))
        def greeting(self):
            filter_param = FilterParams.from_tuples("type", "friend")
            selected_friend = self.__persistence.get_one_random(None, filter_param)
            return f"Hello, {selected_friend.name} !"
        def create(self, trace_id: Optional[str], item: MyFriend) -> MyFriend:
            return self.__persistence.create(trace_id, item)
    
    # 5. REST Controller
    class HelloFriendRestController(RestController):
        def __init__(self):
            super().__init__()
            self._base_route = "/hello_friend"
            self._service = None
        def set_references(self, references):
            self._service = references.get_one_required(Descriptor('hello-friend', 'service', 'default', '*', '1.0'))
        def register(self):
            self.register_route(method="GET", route="/greeting", handler=self.greeting)
            self.register_route(method="GET", route="/greeting_create", handler=self.create)
        def greeting(self):
            return self.send_result(self._service.greeting())
        def create(self):
            item = MyFriend(bottle.request.query["id"], bottle.request.query["type"], bottle.request.query["name"])
            return self.send_result(self._service.create(self._get_trace_id(), item))
    
    # 6. Factory and Process Container
    class HelloFriendControllerFactory(Factory):
        def __init__(self):
            super().__init__()
            self.register_as_type(Descriptor('hello-friend', 'controller', 'http', '*', '1.0'), HelloFriendRestController)
            self.register_as_type(Descriptor('hello-friend', 'service', 'default', '*', '1.0'), HelloFriendService)
            self.register_as_type(Descriptor('hello-friend', 'persistence', 'mysql', '*', '1.0'), HelloFriendPersistence1)
            self.register_as_type(Descriptor('hello-friend', 'persistence', 'postgres', '*', '1.0'), HelloFriendPersistence2)
    
    class HelloFriendProcess(ProcessContainer):
        def __init__(self):
            super().__init__('hello-friend', 'HelloFriend microservice')
            self._config_path = './configDTC.yaml'
            self._factories.add(HelloFriendControllerFactory())
            self._factories.add(DefaultRpcFactory())
    
    if __name__ == '__main__':
        import os
        os.environ["MYSQL_ENABLED"] = "True" # Toggle between MySQL and Postgres
        runner = HelloFriendProcess()
        runner.run()
  7. Implement a ClientFacadeFactory for microservice clients in .NET

    v3
    In a microservice facade architecture, a ClientFacadeFactory is used to register and manage the instantiation of various service clients. By extending the Factory class and using Descriptor objects, you can map specific service identifiers (like 'pip-services-accounts') and implementation types (like 'memory' or 'null') to their corresponding client classes. This allows the facade to dynamically resolve the required clients for the underlying microservices it orchestrates.
    using Pip.Services.SampleFacade.Clients.Version1;
    using PipServices3.Commons.Refer;
    using PipServices3.Components.Build;
    
    namespace Pip.Services.SampleFacade.Build
    {
    	public class ClientFacadeFactory: Factory
    	{
    		// Define descriptors for the clients the facade depends on
    		public static Descriptor SettingsNullClientV1Descriptor = new Descriptor("pip-services-settings", "client", "null", "*", "1.0");
    		public static Descriptor AccountsMemoryClientV1Descriptor = new Descriptor("pip-services-accounts", "client", "memory", "*", "1.0");
    		public static Descriptor PasswordNullClientV1Descriptor = new Descriptor("pip-services-passwords", "client", "null", "*", "1.0");
    		public static Descriptor RolesMemoryClientV1Descriptor = new Descriptor("pip-services-roles", "client", "memory", "*", "1.0");
    		public static Descriptor SessionsMemoryClientV1Descriptor = new Descriptor("pip-services-sessions", "client", "memory", "*", "1.0");
    		public static Descriptor EmailSettingsMemoryClientV1Descriptor = new Descriptor("pip-services-emailsettings", "client", "memory", "*", "1.0");
    		public static Descriptor SitesMemoryClientV1Descriptor = new Descriptor("pip-services-sites", "client", "memory", "*", "1.0");
    		public static Descriptor InvitationsMemoryClientV1Descriptor = new Descriptor("pip-services-invitations", "client", "memory", "*", "1.0");
    		public static Descriptor BeaconsMemoryClientV1Descriptor = new Descriptor("beacons", "client", "memory", "*", "1.0");
    
    		public ClientFacadeFactory()
    		{
    			// Register the descriptors to their actual implementation types
    			RegisterAsType(SettingsNullClientV1Descriptor, typeof(SettingsNullClientV1));
    			RegisterAsType(AccountsMemoryClientV1Descriptor, typeof(AccountsMemoryClientV1));
    			RegisterAsType(PasswordNullClientV1Descriptor, typeof(PasswordsNullClientV1));
    			RegisterAsType(RolesMemoryClientV1Descriptor, typeof(RolesMemoryClientV1));
    			RegisterAsType(SessionsMemoryClientV1Descriptor, typeof(SessionsMemoryClientV1));
    			RegisterAsType(EmailSettingsMemoryClientV1Descriptor, typeof(EmailSettingsMemoryClientV1));
    			RegisterAsType(SitesMemoryClientV1Descriptor, typeof(SitesMemoryClientV1));
    			RegisterAsType(InvitationsMemoryClientV1Descriptor, typeof(InvitationsMemoryClientV1));
    			RegisterAsType(BeaconsMemoryClientV1Descriptor, typeof(BeaconsMemoryClientV1));
    		}
    	}
    }
  8. Implement a three-tier architecture with Pip.Services in C#

    v3

    A three-tier architecture in Pip.Services separates the application into three distinct layers to ensure maintainability and scalability:

    1. View (Tier 1): A RestService that handles HTTP requests, defines routes, and delegates business logic to the controller.
    2. Controller (Tier 2): A logic layer that implements IConfigurable and IReferenceable. It processes data and coordinates between the view and the persistence layer.
    3. Persistence (Tier 3): A data access layer (e.g., inheriting from IdentifiableMySqlPersistence) that handles database schema definition and raw data queries.

    These layers are decoupled using a Factory for dependency injection and a ProcessContainer to manage the lifecycle of the microservice.

    // Data object
    [DataContract]
    public class MyFriend : IStringIdentifiable
    {
        [DataMember(Name = "id")]
        public string Id { get; set; }
    
        [DataMember(Name = "type")]
        public string Type { get; set; }
    
        [DataMember(Name = "name")]
        public string Name { get; set; }
    }
    
    // Tier 1: View
    public class HelloFriendRestService: RestService
    {
        protected HelloFriendController controller;
    
        public HelloFriendRestService() : base()
        {
            _baseRoute = "/hello_friend";
        }
        
        public override void SetReferences(IReferences references)
        {
            base.SetReferences(references);
            controller = references.GetOneRequired<HelloFriendController>(new Descriptor("hello-friend", "controller", "*", "*", "1.0"));
        }
    
        private async Task Greeting(HttpRequest req, HttpResponse res, RouteData routeData)
        {
            var result = await controller.Greeting();
            await SendResultAsync(res, result);
        }
    
        public override void Register()
        {
            RegisterRoute(method: "GET", route: "/greeting", action: Greeting);
        }
    }
    
    // Tier 2 : Controller
    public class HelloFriendController : IConfigurable, IReferenceable
    {
        private HelloFriendPersistence persistence;
    
        public void SetReferences(IReferences references)
        {
            persistence = references.GetOneRequired<HelloFriendPersistence>(new Descriptor("hello-friend", "persistence", "*", "*", "1.0"));
        }
    
        public async Task<string> Greeting()
        {
            var filter = FilterParams.FromTuples("type", "friend");
            var selectedFilter = await persistence.GetOneRandomAsync(null, filter);
            var name = selectedFilter!= null ? selectedFilter.Name : null;
            return $"Hello, {name} !";
        }
    }
    
    // Tier 3 = Persistence
    public class HelloFriendPersistence: IdentifiableMySqlPersistence<MyFriend, string>
    {
        public HelloFriendPersistence() :base("myfriends3") { }
    
        protected override void DefineSchema()
        {
            ClearSchema();
            EnsureSchema($"CREATE TABLE IF NOT EXISTS {_tableName} (id VARCHAR(32) PRIMARY KEY, `type` VARCHAR(50), `name` TEXT)");
        }
    }
  9. Understand the PipServices Dart toolkit architecture

    v4

    The PipServices Dart toolkit is organized into three tiers of modules: Core, Basic, and Optional.

    Core Modules (Required):

    • Commons: Provides basic primitives and patterns for symmetrical development across different programming languages.
    • Components: Contains standard components for logging, performance counters, synchronization, configuration, and caching. Implementations for these are found in extension modules.

    Basic Modules (Commonly used in microservices):

    • Container: Used to compose microservices from loosely-coupled components.
    • Data: Provides components for in-memory and file system data storage.
    • RPC: Handles synchronous communication via local calls and HTTP/REST protocol.
    • Messaging: Handles asynchronous communication and local queues.

    Optional Modules (Technology-specific):

    • Cloud: Azure, AWS
    • Databases/Storage: MongoDB, Redis, Memcached
    • Communication: GRPC, ActiveMQ, MQTT
    • Monitoring/Logging: ElasticSearch, Prometheus
  10. Understand the core components of a Pip.Services Python microservice

    v3

    A standard Pip.Services microservice in Python typically consists of several infrastructure components and custom business logic. The core components include:

    • ContextInfo: Determines the name and description of the microservice.
    • ConsoleLogger: Handles writing logs to stdout.
    • LogCounters: Manages the logging of performance counters.
    • HttpEndpoint: A shared component that allows multiple services to operate on a single HTTP port simultaneously.
    • HeartbeatRestService: Provides a GET /heartbeat endpoint to monitor if the microservice is running.
    • StatusRestService: Provides a GET /status endpoint to retrieve the current status of the microservice.
    • Custom Controllers: (e.g., HelloWorldController) contain the business logic and are linked to REST services via descriptors.
    • Custom REST Services: (e.g., HelloWorldRestServices) expose the controller's functionality over HTTP.
  11. Implement user authentication and session management in a Microservice Facade

    v3
    In a microservice facade architecture, the facade layer orchestrates calls between multiple specialized microservices (Accounts, Passwords, Roles, and Sessions) to handle complex user workflows. This implementation demonstrates how to manage user sign-in, sign-out, and session lifecycle by aggregating responses from these backend services.
    public async signin(req: any, res: any): Promise<void> {
        let login = req.param('login');
        let password = req.param('password');
    
        try {
            // 1. Find user account via Accounts Client
            let account = await this._accountsClient.getAccountByIdOrLogin(null, login);
            if (account == null) {
                throw new BadRequestException(null, 'WRONG_LOGIN', 'Account ' + login + ' was not found');
            }
    
            // 2. Authenticate via Passwords Client
            let result = await this._passwordsClient.authenticate(null, account.id, password);
            if (result == false) {
                throw new BadRequestException(null, 'WRONG_PASSWORD', 'Wrong password for account ' + login);
            }
    
            // 3. Retrieve roles via Roles Client
            let roles: string[] = [];
            if (this._rolesClient) {
                roles = await this._rolesClient.getRolesById(null, account.id);
            }
    
            // 4. Open session
            await this.openSession(req, res, account, roles);
        } catch (err) {
            this.sendError(req, res, err);
        }
    }