FormSG Documentation

repository·develop·Indexed 18 days ago

https://github.com/opengovsg/formsg

An open-source form builder application designed to digitize paper processes, featuring encryption for storage and integrations with Singapore government authentication services like SingPass and CorpPass. This documentation covers self-hosting, local development using Docker, and the @opengovsg/formsg-sdk for verifying webhooks and decrypting submissions in Node.js environments.

Tokens
109.3K
Snippets
399
Records
503
Agent score
63%

What's inside FormSG

  1. Overview of FormSG Self-Hosting

    develop

    FormSG is a feature-rich form builder designed for secure citizen data collection. Self-hosting FormSG allows organizations to:

    • Maintain complete data sovereignty within their own jurisdiction.
    • Integrate with existing systems and identity providers.
    • Meet specific compliance requirements for regulatory environments.
    • Reduce vendor lock-in while maintaining operational control.

    Note: This repository contains self-hosting documentation. For end-user guides on how to create and manage forms, visit https://guide.form.gov.sg/.

  2. Monitoring and Logging Pattern requirements

    develop

    The Monitoring and Logging Pattern provides visibility into application performance and security. It requires:

    • Application performance monitoring (APM).
    • Security event logging and alerting.
    • Infrastructure metrics collection.
    • Centralized log aggregation.

    Conceptual Configuration Example:

    monitoring:
      metrics:
        - container_cpu_usage
        - container_memory_usage
        - application_response_time
        - database_connections
        - failed_authentications
      
      alerts:
        - high_cpu: "> 80% for 5 minutes"
        - high_error_rate: "> 5% for 2 minutes"
        - failed_health_checks: "> 2 in 1 minute"
      
      logs:
        - application_logs
        - access_logs  
        - security_events
        - audit_trail
    # Conceptual Pattern - Adapt to your infrastructure tools
    monitoring:
      metrics:
        - container_cpu_usage
        - container_memory_usage
        - application_response_time
        - database_connections
        - failed_authentications
      
      alerts:
        - high_cpu: "> 80% for 5 minutes"
        - high_error_rate: "> 5% for 2 minutes"
        - failed_health_checks: "> 2 in 1 minute"
      
      logs:
        - application_logs
        - access_logs  
        - security_events
        - audit_trail
  3. Storage and Database Pattern requirements

    develop

    The Storage and Database Pattern provides secure, scalable storage for forms, attachments, and application data. It requires:

    • A MongoDB-compatible document database.
    • S3-compatible object storage for file uploads.
    • A session store for user authentication.
    • Backup and disaster recovery procedures.

    Conceptual Configuration Example:

    database:
      type: mongodb_compatible
      connection: mongodb://user:pass@cluster/formsg
      features_required:
        - transactions
        - ttl_indexes
        - aggregation_pipelines
    
    storage: 
      type: s3_compatible
      buckets:
        attachments: formsg-attachments-prod
        images: formsg-images-prod
        static_assets: formsg-static-prod
      features_required:
        - presigned_urls
        - lifecycle_policies
        - server_side_encryption
    
    sessions:
      type: redis_compatible
      ttl: 86400
      encryption: true
    # Conceptual Pattern - Adapt to your infrastructure tools
    database:
      type: mongodb_compatible
      connection: mongodb://user:pass@cluster/formsg
      features_required:
        - transactions
        - ttl_indexes
        - aggregation_pipelines
    
    # Object storage pattern  
    storage:
      type: s3_compatible
      buckets:
        attachments: formsg-attachments-prod
        images: formsg-images-prod
        static_assets: formsg-static-prod
      features_required:
        - presigned_urls
        - lifecycle_policies
        - server_side_encryption
    
    # Session storage pattern
    sessions:
      type: redis_compatible  # or database-backed
      ttl: 86400  # 24 hours
      encryption: true
  4. Container Service Pattern requirements

    develop

    The Container Service Pattern runs the FormSG application with auto-scaling and health monitoring. Key requirements include:

    • A container with health checks.
    • Auto-scaling based on CPU/memory utilization.
    • Environment variables sourced from a secure configuration store.
    • A load balancer for high availability.

    Conceptual Configuration Example:

    service_name: formsg
    container_image: your-registry/formsg:latest
    port: 5000
    health_check_path: /api/v3/admin/forms
    environment_variables:
      NODE_ENV: production
      DB_HOST: ${database_connection_string}
      SESSION_SECRET: ${secure_session_secret}
    scaling:
      min_instances: 2
      max_instances: 10
      cpu_threshold: 70%
    # Conceptual Pattern - Adapt to your infrastructure tools
    service_name: formsg
    container_image: your-registry/formsg:latest
    port: 5000
    health_check_path: /api/v3/admin/forms
    environment_variables:
      NODE_ENV: production
      DB_HOST: ${database_connection_string}
      SESSION_SECRET: ${secure_session_secret}
    scaling:
      min_instances: 2
      max_instances: 10
      cpu_threshold: 70%
  5. Deploy FormSG using the Pattern-Based Migration Philosophy

    develop

    FormSG is architected for AWS, but it is cloud-agnostic in theory. When deploying to other clouds (GCP, Azure, or On-Premise), do not look for direct product replacements. Instead, map FormSG's requirements to standard infrastructure patterns:

    FormSG RequirementCloud-Agnostic Pattern
    AWS ECSAny container orchestration service
    AWS S3Any object storage with an S3-compatible API
    MongoDB AtlasAny MongoDB-compatible database
    AWS SESAny SMTP service
  6. Configure Site Banners and Maintenance modes

    develop

    FormSG allows setting notification banners via environment variables. You can specify the banner type using prefixes: info:, warn:, or error:. If no prefix is provided, it defaults to info:.

    Banner Precedence for Public Form Routes: SITE_BANNER_CONTENT > IS_GENERAL_MAINTENANCE > IS_SP_MAINTENANCE > IS_CP_MAINTENANCE

    Banner Precedence for Private Form Routes: SITE_BANNER_CONTENT > ADMIN_BANNER_CONTENT

    Banner Precedence for Login Page: SITE_BANNER_CONTENT > IS_LOGIN_BANNER

    SITE_BANNER_CONTENT=info:This is an info banner. You can also add links in the text like https://example.com. There is also a dismiss button to the right of the text.
  7. FormSG compliance support features

    develop

    FormSG includes built-in capabilities to support government compliance requirements:

    Data Protection

    • Encryption: Client-side encryption for sensitive form data.
    • Access controls: Role-based access with audit trails.
    • Data retention: Configurable policies for data lifecycle.
    • Data export: Capability to export data for compliance reporting.

    Audit and Accountability

    • Comprehensive logging: All user actions are logged with timestamps.
    • Non-repudiation: Digital signatures for form submissions.
    • Access tracking: Monitoring who accessed what data and when.
    • Change management: Tracking all form modifications.

    Privacy Protection

    • Minimal data collection: Tools to collect only necessary data.
    • Consent management: Support for form-level privacy notices.
    • Data minimization: Configurable field validation and limits.
    • Right to deletion: Capabilities to support data deletion requests.
  8. Security-Relevant Architecture Decisions in FormSG

    develop

    FormSG's design decisions impact how you should manage and customize your deployment. Understanding these helps maintain a secure posture:

    Design DecisionSecurity BenefitCustomization Impact
    3-Tier ArchitectureClear security boundariesMaintain network segmentation
    Stateless API DesignEasier to scale securelySession store becomes critical
    Component ModularityReplace insecure componentsValidate replacement security
    Environment-Based ConfigNo secrets in codeSecure secrets management required
  9. Load Balancing and Networking Pattern requirements

    develop

    The Load Balancing and Networking Pattern securely exposes FormSG while protecting backend services. It requires:

    • TLS termination with approved certificates.
    • A Web Application Firewall (WAF).
    • Network segmentation between public and private resources.
    • DDoS protection and rate limiting.

    Conceptual Configuration Example:

    load_balancer:
      type: application_load_balancer
      listeners:
        - port: 443
          protocol: HTTPS
          certificate: your-gov-domain-cert
        - port: 80
          protocol: HTTP
          action: redirect_to_https
    
    networking:
      public_subnets:
        - load_balancer
      private_subnets:
        - application_containers
        - database
      security_groups:
        lb_to_app: port_5000_only
        app_to_db: port_27017_only
    # Conceptual Pattern - Adapt to your infrastructure tools
    load_balancer:
      type: application_load_balancer
      listeners:
        - port: 443
          protocol: HTTPS
          certificate: your-gov-domain-cert
        - port: 80
          protocol: HTTP
          action: redirect_to_https
      
    # Network security pattern
    networking:
      public_subnets:
        - load_balancer
      private_subnets:
        - application_containers
        - database
      security_groups:
        lb_to_app: port_5000_only
        app_to_db: port_27017_only
  10. How FormSG handles translations using i18next

    develop

    FormSG uses a hook-based approach for internationalization (i18n) powered by i18next, react-i18next, and i18next-icu for pluralization.

    To display localized text, you must "externalize" it by replacing hardcoded strings with calls to the useTranslation hook. This allows the UI to update dynamically when the user changes languages without requiring a component remount.

    Key concepts:

    • Externalization: Replacing hardcoded text with translation keys.
    • Namespacing: Using keyPrefix to organize translations (e.g., grouping all form-related strings under features.form).
    • ICU Format: Used for complex logic like pluralization (e.g., {count, plural, =1{# item} other{# items}}).
    • Variables: Placeholders like {threshold} are used within strings and must be preserved in translation files to be replaced at runtime.
    import { useTranslation } from 'react-i18next'
    
    const MyComponent = () => {
      // Get the translation function with the correct namespace and prefix
      const { t } = useTranslation('translation', {
        keyPrefix: 'features.form',
      })
    
      // Use the translation function to render strings
      return (
        <div>
          <h1>{t('title')}</h1>
          <p>{t('description')}</p>
          <button>{t('buttons.submit')}</button>
        </div>
      )
    }
  11. Assess technical feasibility for FormSG self-hosting

    develop

    Self-hosting FormSG requires specific technical skills. While not a hard requirement for day one, teams should aim to acquire or possess the following capabilities:

    Minimum Technical Skills Needed:

    • Basic Linux server administration
    • Docker or container orchestration experience
    • Environment variable and secrets management
    • Working with Node.js applications
    • Understanding of web application security

    Team Capability Models:

    • Small teams (1-2 people): Full-stack developers capable of learning DevOps.
    • Medium teams (2-4 people): A mix of development and infrastructure skills.
    • Large teams (5+ people): Specialized roles in DevOps, development, and security.
  12. Understand how multi-language detection works

    develop

    The script attempts to identify four primary languages used in Singapore. Because detecting language is complex (especially when multiple languages appear in the same text block), it uses different strategies:

    • Chinese and Tamil: Identified using dedicated Unicode character ranges.
    • English and Malay: Identified using a heuristic approach that looks for frequently occurring words specific to these languages (since both use Latin characters).
    • Unknown: Any text that does not match the specific language patterns is categorized as unknown. This often occurs when English forms contain non-standard or "funky" words.

    The script generates reports for two specific categories:

    1. Forms where multiple languages are detected.
    2. Forms where only one language is detected, and that language is not English.