Manage GPT context length and input size
mainEach GPT model has a specific context length. If the model starts to 'forget' context (especially with less common languages), you can:
- Reduce the
max_input_lengthin your configuration to split the input into smaller, more manageable requests. - Use a model with a larger context length (e.g.,
gpt-4oorgpt-4o-miniwhich support 128,000 tokens).