Second Me

The Direct Preference Optimization (DPO) workflow is a four-stage process used to optimize models based on preference signals. The stages are:

1. SFT Model Deployment: Deploy a Supervised Fine-Tuning (SFT) model via llama.cpp and expose it through an API endpoint.
2. DPO Data Synthesis: Use the deployed SFT model to generate specialized DPO training data.
3. Model Training: Train the model using the synthesized data with configurable hyperparameters.
4. Merge Weights: Merge the resulting Adapter Weights back into the Base Model (if using LoRA).

Second Me is an open-source prototype designed to create an "AI Self"—a digital identity that preserves your context, personality, and interests.

Core Concepts

• AI-Native Memory: Uses Hierarchical Memory Modeling (HMM) and Me-Alignment algorithms to capture your identity and context through training on your own data.
• Second Me Network: Allows you to scale your intelligence by running your AI self on a distributed network, enabling secure context sharing with other apps and users.
• Privacy & Control: Designed to be trained and hosted locally, ensuring your data remains under your control while providing an interface for global AI collaboration.
• Use Cases:
  • Roleplay: Switching personas to represent you in different scenarios.
  • AI Spaces: Collaborating with other Second Me instances to solve problems or brainstorm ideas.

Long Chain-of-Thought (CoT) mode enables multi-step reasoning during the data synthesis pipeline when using DeepSeek R1 as the base model. This feature is designed to produce synthetic data with extended reasoning chains, improving the long-context reasoning capabilities of trained models.

CoT mode is specifically integrated with DeepSeek-R1 and follows a specific prompt structure for the generated data:

<think>reasoning_content</think>
<answer>final_content</answer>

Supported data types for CoT synthesis include:

• SelfQA data
• Preference data
• Diversity data

Different embedding models produce vectors with different dimensions. When switching models, you must be aware of these dimensions to understand potential mismatches in your vector database (ChromaDB).

| Model | Dimension |
|-------|----------|
| OpenAI text-embedding-ada-002 | 1536 |
| OpenAI text-embedding-3-small | 1536 |
| OpenAI text-embedding-3-large | 3072 |
| Ollama snowflake-arctic-embed | 768 |
| Ollama nomic-embed-text | 768 |
| Ollama mxbai-embed-large | 1024 |

When stream: true is used, the API returns Server-Sent Events (SSE). Each chunk follows the OpenAI chat.completion.chunk format. To extract the text, navigate to choices[0].delta.content.

{
  "id": "chatcmpl-123",
  "object": "chat.completion.chunk",
  "created": 1677652288,
  "model": "gpt-3.5-turbo",
  "system_fingerprint": "fp_44709d6fcb",
  "choices": [
    {
      "index": 0,
      "delta": {"content": "Hello"},
      "finish_reason": null
    }
  ]
}

MCP services return responses as a Server-Sent Events (SSE) stream in an OpenAI-compatible format. Each event contains a fragment of the generated response. The stream concludes when an event containing [DONE] is received.

data: {"id":"chatcmpl-123","object":"chat.completion.chunk","created":1694268190,"model":"lpm-registry-model","system_fingerprint":null,"choices":[{"index":0,"delta":{"content":" world!"},"finish_reason":null}]}

data: {"id":"chatcmpl-123","object":"chat.completion.chunk","created":1694268190,"model":"lpm-registry-model","system_fingerprint":null,"choices":[{"index":0,"delta":{},"finish_reason":"stop"}]}

data: [DONE]

To manually release a new version of the package, you must first bump the version in pyproject.toml. Then, use build and twine to create and upload the distribution archives.

# Install build tools
pip install build twine

# 1. Bump version in pyproject.toml

# 2. Build the package
python -m build

# 3. Upload to PyPI
python -m twine upload dist/*

You can enable or disable CoT mode through the following methods:

1. CLI: Use the --is_cot flag when running train_for_user.sh.
2. Service Initialization: Set is_cot=True when initializing trainprocess_service.py.

# Using the CLI via train_for_user.sh
./train_for_user.sh --is_cot True

To use custom models with Second Me, you must first install Ollama. You can download it from https://ollama.com/download. Once installed, use the following CLI commands to manage your models:

• ollama pull <model_name>: Download a specific model.
• ollama serve: Start the Ollama service.
• ollama ps: List currently running models.
• ollama list: List all downloaded models.
• ollama rm <model_name>: Remove a model.
• ollama show <model_name>: Display model details (architecture, context length, etc.).

ollama pull qwen2.5:0.5b
ollama serve

Verify the model's responses by running the test script.

Execution: Run the following command from the project root directory:

python lpm_kernel/L2/mlx_training/test_mlx.py

Customization: The default prompt in test_mlx.py is configured for a specific Chain-of-Thought (COT) model. You should modify the payload object in the script to match your training objectives and desired prompt format.

# Example of modifying the payload in test_mlx.py
payload = {
    "messages": [
        {
            "role": "system",
            "content": "Your custom system prompt here..."
        },
        {
            "role": "user",
            "content": "Your test question here"
        }
    ],
    "temperature": 0.7
}

If you are developing the gguf package, install it in editable mode so changes to the source code are reflected immediately. You may need to upgrade pip first if your installation requires setup.py for editable mode.

# Upgrade pip if necessary
pip install --upgrade pip

# Install in editable mode
cd /path/to/llama.cpp/gguf-py
pip install --editable .

Before running the DPO workflow, you must perform the following manual configurations:

1. API Configuration: Set your API_KEY and BASE_URL in lpm_kernel/L2/dpo/utils.py.
2. Personal Bio: Manually populate the global bio section, including details such as interests and occupation.
3. Model Format: Ensure your SFT model is converted to the gguf format before deployment.