OneKE Knowledge Extraction System

Docker provides a reliable and stable environment for OneKE. This method is recommended for consistent deployment.

Prerequisites

• Docker installation
• NVIDIA Container Toolkit
• GPU support (recommended CUDA version: 12.4)

Steps

1. Clone the repository and enter the directory:

   git clone https://github.com/zjunlp/OneKE.git
   cd OneKE

2. Pull the official image:

   docker pull zjunlp/oneke:v4

3. Launch the container:

   docker run --gpus all -v ./OneKE:/app/OneKE -it oneke:v4 /bin/bash

Mapping Local Models If you are using locally deployed models, you must map the local model path to a container path:

docker run --gpus all \
  -v ./OneKE:/app/OneKE \
  -v your_local_model_path:/app/model/your_model_name \
  -it oneke:v4 /bin/bash

Note: Once inside the container, the working directory is /app/OneKE. Changes made to the local ./OneKE folder will sync to the container automatically.

Identify subject-predicate-object triples to construct Knowledge Graphs (KG).

Constraint Formats:

• Entity types only: ["Person", "Place"]
• Entity and Relation types: [["Person", "Place"], ["LivesIn", "BornIn"]]
• Subject, Relation, and Object types: [["Person"], ["LivesIn"], ["Place"]]

Building a Knowledge Graph: To automatically construct a KG (e.g., in Neo4j), add the construct field to your YAML configuration.

Warning: If you do NOT want to build a KG, you must remove or comment out the construct field to avoid database connection errors.

Steps:

1. Configure ./examples/config/Triple2KG.yaml with the construct block.
2. Run the extraction script:

config_file=./examples/config/Triple2KG.yaml
python src/run.py --config $config_file

construct: # (Optional) If you want to construct a Knowledge Graph, you need to set the construct field, or you must delete this field.
  database: Neo4j # your database type.
  url: neo4j://localhost:7687 # your database URL
  username: your_username # your database username.
  password: "your_password" # your database password.

The OneKE Web UI provides a graphical interface for configuring and running information extraction tasks. The interface is organized into several functional sections:

Initial Page

• 🎲 Quick Start with an Example 🎲: Loads a simple, predefined example for immediate testing.
• Submit: Executes the configured extraction task.
• Clear: Resets the interface to its initial state.

Model Configuration

Configure the LLM powering the extraction:

• 🪄 Enter your Model: Input a model name (e.g., gpt-4o-mini, o3-mini, deepseek-chat, deepseek-reasoner) or a local file path for local models.
• 🔑 Enter your API-Key: The API key for the selected model (not required for local models).
• 🔗 Enter your Base-URL: Custom API endpoint. Leave empty to use the default URL.

Task Configuration

• 🎯 Select your Task: Choose between Traditional IE (NER, RE, EE, and Triple) or Open Domain IE (Web News Extraction, Book Knowledge Extraction, etc.). Selecting Triple allows for Knowledge Graph construction via Neo4j.
• 🧭 Select your Mode: Choose the extraction strategy.
  • direct mode: Recommended for long text.
  • standard mode: Recommended for short text requiring high accuracy.
  • customize: Allows for custom agent strategies.
  • Not Required: Use if no agents are needed.

Text Input

• 📂 Use File: Toggle to switch between manual text entry and file uploads.
• 📖 Upload a File: Supports formats like *.pdf* and *.html* (must enable 📂 Use File first).
• 📖 Text: A text area for entering raw text in any language.

Set Instruction

• 🕹️ Instruction: Define what to extract (e.g., Please help me extract all the person names). Note: Base-type tasks allow free definitions, while RE, NER, EE, and Triple use predefined instructions.
• ❗Constraint: Optional type and property constraints for extraction objects.
• 💰 Update Case: Check to update the Case Repository.
• 🪙 Truth: Provide ground truth data (e.g., {"relation_list": [{"head": "Guinea", "tail": "Conakry", "relation": "country capital"}]}) to help the LLM.

Get Result

After clicking Submit, the UI displays:

• 🤔 Generated Schema: The automatically generated extraction schema in Python class format.
• 😉 Final Answer: The extraction results in structured JSON format.
• 📝 KG Visualization: A visual representation of the knowledge graph (available for Triple tasks using Neo4j).

You can perform knowledge extraction using the Command Line Interface by providing a YAML configuration file.

1. Prepare Configuration Create or modify a YAML file (examples are available in examples/config). The configuration is divided into two main sections:

• model: Defines the category (e.g., ChatGPT, DeepSeek, LLaMA, Qwen, ChatGLM, MiniCPM, OneKE), model_name_or_path, api_key, and base_url.
• extraction: Defines the task (e.g., Base, NER, RE, EE), instruction, use_file, file_path, output_schema, mode (quick, detailed, customized), and update_case.

2. Execute Extraction Run the following command from the OneKE directory:

config_file=your_yaml_file_path
python src/run.py --config $config_file

3. Deploy Local Models via vLLM To use local models with vLLM acceleration, first deploy the model, then run the extraction:

# Ensure vllm_serve is set to TRUE in your config file
config_file=your_yaml_file_path
python src/models/vllm_serve.py --config $config_file
python src/run.py --config $config_file