repository·main·Indexed 25 days ago
https://github.com/cosmosshadow/gptpdfA tool that uses large visual models, such as GPT-4o, to parse PDF documents into high-fidelity Markdown. It preserves typography, math formulas, tables, and images, and supports customizable prompts and parallel processing via the parse_pdf function.
To isolate products from the reaction mixture:
Install the gptpdf package using pip to begin parsing PDFs into markdown using large visual models.
pip install gptpdfpython alias configured in your shell, you may need to temporarily unalias it to ensure the virtual environment's Python is used.To improve the conversion rate of S25DH (Steroid C25 dehydrogenase) with hydrophobic substrates like cholest-4-en-3-one, cholecalciferol, or cholesterol, optimize the ratio of the solubilizer HBC (2-hydroxypropyl-β-cyclodextrin) and the organic co-solvent EGME (2-methoxyethanol).
Using EGME as a substitute for 1,4-dioxane can significantly increase conversion rates and substrate loading. Based on experimental results:
After setting up the environment, navigate to the test directory. Before running the tests, you must export the environment variables defined in the .env file. You can then execute the test suite using python test.py.
cd test
# Export environment variables from .env
export $(grep -v '^#' .env | sed 's/^export //g' | xargs)
python test.pyS25DH enzymes are oxygen-sensitive, particularly in their reduced state. To ensure long-term catalyst performance and prevent enzyme inactivation:
If the default prompts do not yield optimal results for your specific model, you can pass a custom prompt dictionary to parse_pdf. The dictionary supports three keys: prompt, rect_prompt, and role_prompt.
prompt = {
"prompt": "Custom prompt text",
"rect_prompt": "Custom rect prompt",
"role_prompt": "Custom role prompt"
}
content, image_paths = parse_pdf(
pdf_path=pdf_path,
output_dir='./output',
model="gpt-4o",
prompt=prompt,
verbose=False,
)You can provide custom prompts to parse_pdf() to change the language or the formatting style of the output. The function uses three distinct prompt types:
prompt (User Prompt): Controls how the model converts image content to Markdown (e.g., language requirements, formula styles like $$ $$).rect_prompt (Rectangle Prompt): Instructs the model on how to handle specific identified regions (e.g., "If the region is a table, use ![]() format").role_prompt (System Prompt): Defines the model's persona (e.g., "You are a PDF document parser").You can customize the prompt, rect_prompt, and role_prompt to adapt the model's behavior to specific requirements or different models.
content, image_paths = parse_pdf(
pdf_path=pdf_path,
output_dir='./output',
model="gpt-4o",
prompt="自定义主提示词",
rect_prompt="自定义矩形区域提示词",
role_prompt="自定义角色提示词",
verbose=False,
)Use the parse_pdf function to convert a PDF file into Markdown content and extract associated images. You must provide an OpenAI API key either as an argument or via the OPENAI_API_KEY environment variable.
from gptpdf import parse_pdf
api_key = 'Your OpenAI API Key'
content, image_paths = parse_pdf(pdf_path, api_key=api_key)
print(content)The parse_pdf function parses a PDF file into a Markdown file and returns the Markdown content along with a list of all extracted image paths.
Signature:
def parse_pdf(
pdf_path: str,
output_dir: str = './',
prompt: Optional[Dict] = None,
api_key: Optional[str] = None,
base_url: Optional[str] = None,
model: str = 'gpt-4o',
verbose: bool = False,
gpt_worker: int = 1
) -> Tuple[str, List[str]]:Parameters:
pdf_path (str): Path to the PDF file.output_dir (str, default: './'): Output directory to store all images and the Markdown file.api_key (Optional[str], optional): OpenAI API key. If not provided, the OPENAI_API_KEY environment variable will be used.base_url (Optional[str], optional): OpenAI base URL. If not provided, the OPENAI_BASE_URL environment variable will be used. This allows using other services with OpenAI-compatible interfaces (e.g., GLM-4V).model (str, default: 'gpt-4o'): OpenAI API formatted multimodal large model. Supports models like qwen-vl-max, GLM-4V, Yi-Vision, or Azure OpenAI (by setting base_url to the Azure endpoint and using the deployed model name).verbose (bool, default: False): When enabled, the content parsed by the large model is displayed in the command line.gpt_worker (int, default: 1): Number of GPT parsing worker threads. Increase this to speed up parsing on high-performance machines.prompt (dict, optional): A dictionary to provide custom prompts. The dictionary can contain:prompt: Guides the model on processing/converting text content in images.rect_prompt: Handles specific marked areas like tables or images.role_prompt: Defines the model's role for the parsing task.Returns:
Tuple[str, List[str]]: A tuple containing the Markdown string and a list of image file paths.content, image_paths = parse_pdf(
pdf_path=pdf_path,
output_dir='./output',
model="gpt-4o",
prompt=prompt,
verbose=False,
)Two primary methods are used to monitor S2SDH activity: