DeepSeek Coder

Training data must follow a specific JSON-serialized format. Each line in your dataset file should be a JSON string containing two required fields:

1. instruction: The task or prompt provided to the model.
2. output: The target response or code completion.

For a reference implementation of the data structure, you can follow the Evol-Instruct-Code-80k-v1 format.

To solve mathematical problems using the Program-Aided Language (PAL) approach, use a prompt that instructs the model to use Python for computation and to display the final result in LaTeX.

Prompt Structure:

1. Instruction: "Let's use python to solve math problems. Display the final result in LaTeX."
2. Question: The mathematical problem to be solved.
3. Solution: A Python function named solution() that performs the calculation and returns the result (often formatted as a string or LaTeX).

Let's use python to solve math problems. Display the final result in LaTeX.

Question: [Mathematical Question]

```python
def solution():
    # ... computation logic ...
    return result

To evaluate a base model (e.g., DeepSeek-Coder-1.3b-Base) on the HumanEval dataset using multiple GPUs, use the eval_pal.py script via accelerate.

Note: If you are testing different programming languages, you must update the execution paths in humaneval/execution.py to ensure the environment can run the generated code correctly.

Required arguments:

• --logdir: The model name or path.
• --language: The programming language to evaluate (e.g., python).
• --dataroot: The root directory of the dataset.

MODEL_NAME_OR_PATH="deepseek-ai/deepseek-coder-1.3b-base"
DATASET_ROOT="data/"
LANGUAGE="python"
python -m accelerate.commands.launch --config_file test_config.yaml eval_pal.py --logdir ${MODEL_NAME_OR_PATH} --language ${LANGUAGE} --dataroot ${DATASET_ROOT}

To evaluate the model's ability to solve mathematical problems using the PAL (Program-Aided Language models) method, install the following required Python packages:

pip install sympy==1.12 pebble timeout-decorator transformers

Since DeepSeek Coder uses a HuggingFace Tokenizer, you can generate GGUF models by using a specific branch of llama.cpp that supports these pre-tokenizers.

Steps:

1. Clone llama.cpp and checkout the regex_gpt2_preprocess branch.
2. Build llama.cpp.
3. Use convert-hf-to-gguf.py to convert the model.
4. Use ./quantize to apply quantization (e.g., q4_0).

git clone https://github.com/DOGEwbx/llama.cpp.git
cd llama.cpp
git checkout regex_gpt2_preprocess
make
python3 -m pip install -r requirements.txt
# generate GGUF model
python convert-hf-to-gguf.py <MODEL_PATH> --outfile <GGUF_PATH> --model-name deepseekcoder
# use q4_0 quantization as an example
./quantize <GGUF_PATH> <OUTPUT_PATH> q4_0
./main -m <OUTPUT_PATH> -n 128 -p <PROMPT>

To use the DeepSeek Coder models, install the required Python dependencies using pip:

pip install -r requirements.txt

For high-throughput inference, use the vLLM library.

Text Completion: Pass a list of strings to llm.generate(). Chat Completion: Use tokenizer.apply_chat_template with tokenize=False to convert messages into formatted prompts before passing them to llm.generate().

from vllm import LLM, SamplingParams

tp_size = 4 # Tensor Parallelism
sampling_params = SamplingParams(temperature=0.7, top_p=0.9, max_tokens=100)
model_name = "deepseek-ai/deepseek-coder-6.7b-base"
llm = LLM(model=model_name, trust_remote_code=True, gpu_memory_utilization=0.9, tensor_parallel_size=tp_size)

prompts = ["If everyone in a country loves one another,"]
outputs = llm.generate(prompts, sampling_params)
print([output.outputs[0].text for output in outputs])

You can evaluate a DeepSeek Coder model on the MBPP dataset using the eval_pal.py script. The provided example uses accelerate.commands.launch to distribute the workload across 8 GPUs using a configuration file named test_config.yaml.

Required Arguments:

• --logdir: The path/name of the model being evaluated.
• --dataroot: The directory containing the MBPP dataset.

MODEL_NAME_OR_PATH="deepseek-ai/deepseek-coder-1.3b-base"
DATASET_ROOT="data/"
LANGUAGE="python"
python -m accelerate.commands.launch --config_file test_config.yaml eval_pal.py --logdir ${MODEL_NAME_OR_PATH} --dataroot ${DATASET_ROOT} 

To set up the environment for evaluating DeepSeek-Coder models on code generation benchmarks, install the following Python packages:

pip install accelerate
pip install attrdict
pip install transformers
pip install pytorch

To evaluate instruction-based models (e.g., deepseek-coder-33b-instruct), use the eval_instruct.py script. This script allows you to specify the model, output path, and language.

Required arguments:

• --model: The full model identifier (e.g., deepseek-ai/deepseek-coder-33b-instruct).
• --output_path: The file path where the results will be saved in .jsonl format.
• --language: The programming language to evaluate.
• --temp_dir: A directory for temporary files used during evaluation.

LANG="python"
OUPUT_DIR="output"
MODEL="deepseek-coder-33b-instruct"

CUDA_VISIBLE_DEVICES=0,1 python eval_instruct.py \
    --model "deepseek-ai/$MODEL" \
    --output_path "$OUPUT_DIR/${LANG}.$MODEL.jsonl" \
    --language $LANG \
    --temp_dir $OUPUT_DIR