spacy-llm

repository·main·Indexed 23 days ago

An experimental library that integrates Large Language Models (LLMs) into the spaCy NLP framework. It enables developers to prototype NLP tasks using prompting and parsing before transitioning to supervised models. Supported tasks include Named Entity Recognition, text classification, lemmatization, relationship extraction, sentiment analysis, span categorization, summarization, entity linking, and translation. It interfaces with API providers like OpenAI, Cohere, Anthropic, Google PaLM, and Microsoft Azure AI, as well as open-source models via Hugging Face and LangChain.

Tokens: 6.1K
Snippets: 22
Records: 29
Agent score: 80%

What's inside spacy-llm

How to write a custom model
main
A model is responsible for the interaction with the LLM. While tasks handle prompt creation and response parsing, the model handles the execution.
Internally, spacy-llm follows this pattern:
1. prompts = task.generate_prompts(docs)
2. responses = model(prompts)
3. docs = task.parse_responses(docs, responses)
To implement a custom model, register a function using @registry.llm_models. The returned object should be a callable that accepts an iterable of prompts and yields an iterable of responses. While built-in tasks and models typically use str, you can use arbitrary objects if your task and model are designed to be compatible.
```
from spacy_llm.registry import registry
import random
from typing import Iterable

@registry.llm_models("RandomClassification.v1")
def random_textcat(labels: str):
    labels = labels.split(",")
    def _classify(prompts: Iterable[str]) -> Iterable[str]:
        for _ in prompts:
            yield random.choice(labels)

    return _classify
```
https://github.com/explosion/spacy-llm/blob/main/usage_examples/README.md
Configure Dolly model sizes in the config file
main
The pipeline uses Dolly models hosted on Huggingface. While dolly-v2-3b is used by default, you can update the configuration file to use larger model sizes for different performance requirements. Supported models include:
- dolly-v2-3b (default)
- dolly-v2-7b
- dolly-v2-12b
https://github.com/explosion/spacy-llm/blob/main/usage_examples/textcat_dolly/README.md
Understand the structure of a spacy-llm configuration file
main
A spacy-llm configuration file defines an llm component which requires two primary parameters: task and model.
- task: Defines how the prompt is structured and how the LLM's output is parsed into a spaCy Doc (e.g., storing entities in doc.ents or categories in doc.cats).
- model: Defines which model to use (open-source or third-party API) and the connection method (REST, LangChain, HuggingFace, etc.).
Configuration is based on spaCy's configuration system, making it modular and extensible.
```
[components]

[components.llm]
factory = "llm"

# Defines the prompt and parsing logic
[components.llm.task]
...

# Defines the model and connection method
[components.llm.model]
...
```
https://github.com/explosion/spacy-llm/blob/main/usage_examples/README.md
Use negative examples to improve NER accuracy
main
The spacy.NER.v3 task uses Chain-of-Thought reasoning. To improve performance, provide both positive and negative examples in your few-shot data. Negative examples explicitly show the model what is not an entity for your specific use case.
When defining a negative example, set is_entity to false and use label: "==NONE==". Use the reason property to explain why a specific span was rejected. This helps the LLM understand context-dependent boundaries.
Example JSON structure for negative examples:
```
[
    {
        "text": "You can't get a great chocolate flavor with carob.",
        "spans": [
            {
                "text": "chocolate",
                "is_entity": false,
                "label": "==NONE==",
                "reason": "is a flavor in this context, not an ingredient"
            },
            {
                "text": "carob",
                "is_entity": true,
                "label": "INGREDIENT",
                "reason": "is an ingredient to add chocolate flavor"
            }
        ]
    }
]
```
https://github.com/explosion/spacy-llm/blob/main/usage_examples/ner_v3_openai/README.md
Provide examples for few-shot annotation
main
To perform few-shot learning, use a configuration file designed for few-shot tasks (e.g., fewshot.cfg) and provide a file containing training examples. spacy-llm supports multiple formats for these examples, including .yaml, .yml, .jsonl, and .json.
https://github.com/explosion/spacy-llm/blob/main/usage_examples/multitask_openai/README.md

Quickstart: Text classification using a config file

main

For more control over parameters, use spaCy's config system. Define the llm component, its task (e.g., spacy.TextCat.v3), and its model (e.g., spacy.GPT-4.v2) in a .cfg file. Use spacy_llm.util.assemble to load the pipeline from the config.

[nlp]
lang = "en"
pipeline = ["llm"]

[components]

[components.llm]
factory = "llm"

[components.llm.task]
@llm_tasks = "spacy.TextCat.v3"
labels = ["COMPLIMENT", "INSULT"]

[components.llm.model]
@llm_models = "spacy.GPT-4.v2"

from spacy_llm.util import assemble

nlp = assemble("config.cfg")
doc = nlp("You look gorgeous!")
print(doc.cats)
# {"COMPLIMENT": 1.0, "INSULT": 0.0}

Quickstart: Text classification in Python
main
You can perform quick experiments by adding an llm_textcat pipe to a blank spaCy model. This uses the built-in text classification task and the default OpenAI GPT-3.5 model. You must ensure your OpenAI API key is set as an environment variable.
```
import spacy

nlp = spacy.blank("en")
llm = nlp.add_pipe("llm_textcat")
llm.add_label("INSULT")
llm.add_label("COMPLIMENT")
doc = nlp("You look gorgeous!")
print(doc.cats)
# {"COMPLIMENT": 1.0, "INSULT": 0.0}
```
https://github.com/explosion/spacy-llm/blob/main/README.md
How to write a custom task
main
A task represents an action you want an LLM to perform. To implement a custom task, you must create a class and register it using the llm_tasks registry. The class must implement two specific methods:
1. generate_prompts(docs: Iterable[Doc]) -> Iterable[str]: Transforms spaCy Doc objects into a list of string prompts to be sent to the model.
2. parse_responses(docs: Iterable[Doc], responses: Iterable[str]) -> Iterable[Doc]: Parses the LLM's string outputs back into spaCy Doc objects, typically by updating attributes like doc.ents or doc.cats.
For a barebones implementation to use as a template, refer to spacy.NoOp.v1.
```
from spacy_llm.registry import registry
from spacy_llm.util import split_labels
from typing import Iterable, List

@registry.llm_tasks("my_namespace.MyTask.v1")
def make_my_task(labels: str, my_other_config_val: float) -> "MyTask":
    labels_list = split_labels(labels)
    return MyTask(labels=labels_list, my_other_config_val=my_other_config_val)


class MyTask:
    def __init__(self, labels: List[str], my_other_config_val: float):
        ...

    def generate_prompts(self, docs: Iterable[Doc]) -> Iterable[str]:
        ...

    def parse_responses(
        self, docs: Iterable[Doc], responses: Iterable[str]
    ) -> Iterable[Doc]:
        ...
```
https://github.com/explosion/spacy-llm/blob/main/usage_examples/README.md
Run a multi-task spaCy pipeline via CLI
main
You can execute a spaCy pipeline containing multiple LLM-backed tasks (such as classification and entity detection) using a Python script. The script typically accepts the input text, a configuration file (.cfg), and an optional file containing examples for few-shot learning.
https://github.com/explosion/spacy-llm/blob/main/usage_examples/multitask_openai/README.md
Set up OpenAI credentials for spacy-llm
main
To use OpenAI models (like GPT) for tasks such as text classification, you must provide your OpenAI API key and organization ID via environment variables. Obtain these from the OpenAI platform.
```
export OPENAI_API_KEY="sk-..."
export OPENAI_API_ORG="org-..."
```
https://github.com/explosion/spacy-llm/blob/main/usage_examples/textcat_openai/README.md
Run the relation extraction pipeline via CLI
main
You can execute the relation extraction pipeline using the run_pipeline.py script. The script requires the input text and a path to a configuration file. For few-shot learning, you must also provide a path to an examples file (e.g., .jsonl, .json, .yml, or .yaml).
Zero-shot usage:
```
python run_pipeline.py "[TEXT]" [PATH_TO_CONFIG]
```
Few-shot usage:
```
python run_pipeline.py "[TEXT]" [PATH_TO_CONFIG] [PATH_TO_EXAMPLES]
```
```
# Zero-shot example
python run_pipeline.py \
    "Laura just bought an apartment in Boston." \
    ./zeroshot.cfg

# Few-shot example
python run_pipeline.py \
    "Laura just bought an apartment in Boston." \
    ./fewshot.cfg \
    ./examples.jsonl
```
https://github.com/explosion/spacy-llm/blob/main/usage_examples/rel_openai/README.md
Migrate from 0.3.x to 0.4.x config paradigm
main
Version 0.4.x introduced a significant refactor, moving from a backend-centric configuration to a model-centric configuration.
Global Changes
- The registry name changed from @llm_backends to @llm_models.
- The api attribute has been removed.
Migrating REST-based Models (Default)
REST models are used for hosted LLMs. In 0.4.x, you specify the model type via the @llm_models registry and the specific model variant via the name attribute.
0.3.x (Old):
```
[components.llm.backend]
@llm_backends = "spacy.REST.v1"
api = "OpenAI"
config = {"model": "gpt-3.5-turbo", "temperature": 0.3}
```
0.4.x (New):
```
[components.llm.model]
@llm_models = "spacy.GPT-3-5.v1"
name = "gpt-3.5-turbo"
config = {"temperature": 0.3}
```
Migrating HuggingFace Models
HF models in 0.4.x (e.g., spacy.Dolly.v1) now distinguish between initialization arguments and inference arguments using config_init and config_run.
0.3.x (Old):
```
[components.llm.backend]
@llm_backends = "spacy.Dolly_HF.v1"
model = "databricks/dolly-v2-3b"
config = {}
```
0.4.x (New):
```
[components.llm.model]
@llm_models = "spacy.Dolly.v1"
name = "dolly-v2-3b"
config_init = {}
config_run = {}
```
Migrating LangChain Models
LangChain models follow the same pattern as REST models, but use the langchain.[API].[version] registry format.
0.3.x (Old):
```
[components.llm.backend]
@llm_backends = "spacy.LangChain.v1"
api = "OpenAI"
config = {"temperature": 0.3}
```
0.4.x (New):
```
[components.llm.model]
@llm_models = "langchain.OpenAI.v1"
name = "gpt-3.5-turbo"
config = {"temperature": 0.3}
```
```
# 0.4.x REST Example
[components.llm.model]
@llm_models = "spacy.GPT-3-5.v1"
name = "gpt-3.5-turbo"
config = {"temperature": 0.3}
```
https://github.com/explosion/spacy-llm/blob/main/migration_guide.md

spacy-llm

What's inside spacy-llm

Global Changes

Migrating REST-based Models (Default)

Migrating HuggingFace Models

Migrating LangChain Models