Open Charge Map (OCM) System
repository·master·Indexed 40 days ago
https://github.com/openchargemap/ocm-systemBackend systems for Open Charge Map, a global public registry of electric vehicle charging locations. Includes the .NET 6.x API, the OCM API Mirror Service for high-performance read operations via MongoDB, server-side import processing for OCPI providers, and localization tools.
What's inside Open Charge Map
- Developers can use the Open Charge Map API to access the global registry of electric vehicle charging locations and build custom applications. Detailed API documentation and developer guides can be found at https://openchargemap.org/site/develop/.
Use the current Open Charge Map Mobile and Web App
masterThe repository
openchargemap/ocm-system/Appcontains the legacy version of the Open Charge Map web application. For the current, actively maintained version of the mobile and web app, use the following repository:What is the OCM API Mirror Service?
masterThe OCM API Mirror Service is a .NET Core worker that runs a complete read-only clone of the Open Charge Map (OCM) API. It functions by syncing data changes from the master API into a local MongoDB instance, allowing for high-performance read operations without querying the master SQL database.How failed import tracking and retries work
masterThe worker manages failed imports using an in-memory cache to prevent resource wasting and cascading failures:
- Failure Trigger: An import is marked as failed only when an actual exception occurs. If an import completes but
importedOK = false(indicating no changes were needed), it is not treated as a failure. - Quarantine: When a failure occurs, the provider is added to the failed imports cache with a timestamp and is skipped for the next 24 hours (controlled by
IMPORT_DUE_THRESHOLD_HOURS). - Recovery:
- If an import succeeds, the provider is immediately removed from the failed imports cache.
- If the 24-hour threshold expires, the provider is automatically removed from the cache and becomes eligible for retry in the next cycle if it is still due.
- Failure Count: The worker tracks how many times a provider has failed for diagnostic purposes.
- Failure Trigger: An import is marked as failed only when an actual exception occurs. If an import completes but
How the OCM Import Worker scheduling works
masterThe OCM Import Worker uses an intelligent, API-driven scheduling system to process data providers. The scheduling logic follows these rules:
- Cycle Execution: Every
ImportRunFrequencyMinutes, the worker fetches theDataProviderlist from the OCM API. - Filtering: It filters providers based on the
EnabledImportsconfiguration and identifies those due for import (whereDateLastImportedisNULLor older than theIMPORT_DUE_THRESHOLD_HOURS, which defaults to 24 hours). - Failure Exclusion: Providers that have failed an import within the last 24 hours are excluded from the current cycle.
- Priority Queue: Providers are processed in order of oldest to newest
DateLastImported. The provider with the oldest timestamp has the highest priority. - Wait Behavior: If no providers meet the criteria, the worker logs the status and waits for the next cycle without performing unnecessary imports.
- Cycle Execution: Every
Install and build the OCM API on Linux
masterFollow these steps to set up the environment and build the API from source:
- Install .NET 6.0 SDK:
sudo snap install dotnet-sdk --classic --channel=6.0
Verify with `dotnet --version`. 2. **Install MongoDB**: Install the latest version for your system and ensure the service is set to run on startup. 3. **Clone and Build**: ```bash git clone https://github.com/openchargemap/ocm-system cd ocm-system/API/OCM.Net/OCM.API.Web dotnet buildsudo snap install dotnet-sdk --classic --channel=6.0 git clone https://github.com/openchargemap/ocm-system cd ocm-system/API/OCM.Net/OCM.API.Web dotnet build- Install .NET 6.0 SDK:
Add an OCPI Provider via Code (Legacy)
masterUse this approach only if the provider requires custom processing logic. This involves creating a new class that inherits from
ImportProvider_OCPI.Steps:
- Add DataProvider definition in database.
- Add/get operator id for the target network.
- Store credential in keyvault.
- Implement the OCPI import provider in
OCM.Import.Common.Providers.OCPI. - Add import provider to OCM.Import.Worker:
- Update
appsettings.jsonunderEnabledImports. - Add the provider to
OCM.Import.Common.ImportManager.
- Update
Implementation Details:
- Download and verify the OCPI feed manually to check credentials and data format.
- Add a test with a copy of the JSON feed in
OCM.API.Tests - ImportTeststo ensure data consistency. - Ensure the provider maps to its own network operator and any other relevant operators via
GetOperatorMappings().
using System.Collections.Generic; namespace OCM.Import.Providers.OCPI { public class ImportProvider_MyProvider : ImportProvider_OCPI, IImportProvider { public ImportProvider_MyProvider() : base() { ProviderName = "myprovider"; OutputNamePrefix = "myprovider"; IsAutoRefreshed = true; IsProductionReady = true; CredentialKey = "OCPI-MYPROVIDER"; // or null if no auth needed DefaultOperatorID = 1234; Init(dataProviderId: 99, "https://api.example.com/ocpi/2.2/locations"); } public override Dictionary<string, int> GetOperatorMappings() { return new Dictionary<string, int>() { { "My Operator", 1234 } }; } } }Verify core application workflows
masterThe following test plan outlines the critical user journeys required to validate the Open Charge Map application functionality:
Search and Discovery
- Search Near Me: Verify that selecting 'Search Near Me' displays search progress and correctly renders search results.
- POI Details: Selecting a search result must display the Point of Interest (POI) details.
- Map Integration: Returning to the Map view via the Menu Icon must render markers for all current search results. Selecting a marker must display the POI details.
- Data Refresh: After adding comments or photos, refreshing search results must ensure new content appears in the POI details.
User Authentication and Contributions
- Authentication Flow: Attempting to 'Add Comment' should prompt a Sign In. Completing the Sign In from the main menu should return the user to the app in a 'Signed in' state.
- Commenting: Once signed in, navigate to POI details, select 'Add Comment', enter details, and verify submission.
- Photo Uploads: Navigate to POI details, select 'Add Photo', enter details, and verify submission.
Location and Profile Management
- Location Management: Verify the ability to edit an existing location and add a new location.
- Favorites: Verify the ability to add and remove a favorite location.
Build the OCM Import Worker image
masterYou can build the OCM Import Worker image using either standard Docker commands from the repository root or via Docker Compose.
Using Docker: Run the build command from the
ocm-systemroot directory. You can specify a build configuration (e.g.,Release) or perform a clean build using--no-cache.Using Docker Compose: Navigate to the
Import/OCM.Import.Workerdirectory and rundocker-compose build.# Build using Docker from repository root docker build -f Import/OCM.Import.Worker/Dockerfile -t ocm-import-worker:latest . # Build using Docker Compose cd Import/OCM.Import.Worker docker-compose build # Build with specific configuration docker build \ -f Import/OCM.Import.Worker/Dockerfile \ --build-arg BUILD_CONFIGURATION=Release \ -t ocm-import-worker:v1.0.0 \ . # Build with no cache (clean build) docker build --no-cache \ -f Import/OCM.Import.Worker/Dockerfile \ -t ocm-import-worker:latest \ .Prerequisites for running the OCM API Mirror
masterTo run the API Mirror Service, your host must meet the following requirements:
- Runtime: A host capable of running the latest .NET Core runtime (specifically .NET 6.x for current builds).
- Database: MongoDB installed on the host with read/write permissions enabled for local connections.
- Hardware (Production): At least 2GB of main RAM and adequate swap space configured. Lack of swap space on Linux can cause the service to hang.
Set up the OCM Localisation environment
masterTo set up a new development machine for OCM Localisation, install the necessary dependencies usingnpm installwithin the/Localisation/directory. This will fetch required tools likegrunt.npm installDebug the OCM Import Worker container
masterIf the container is running or failing, use these methods to inspect the internal state:
- Interactive Shell: Start a new container with a bash entrypoint or execute bash in a running container.
- File Inspection: Use
lsto check paths orcatto readappsettings.json. - Environment Check: Run
envto verify loaded variables. - Resource Monitoring: Use
docker statsfor real-time usage ordf -hfor disk space.
# Execute shell in running container docker exec -it ocm-import-worker /bin/bash # Check application files docker exec ocm-import-worker ls -la /app docker exec ocm-import-worker cat /app/appsettings.json docker exec ocm-import-worker env # Monitor performance docker stats ocm-import-worker docker exec ocm-import-worker df -h