LLM Sherpa

Unlike standard parsers, LayoutPDFReader preserves document structure during chunking. This ensures:

• List items and their preceding paragraphs are kept together.
• Table items are chunked together.
• Contextual information from section headers and nested headers is included in the text chunks.

This is particularly useful for RAG (Retrieval Augmented Generation) applications to maintain context within limited LLM window sizes.

Install the llmsherpa library using pip.

pip install llmsherpa

LLM Sherpa provides 'smart chunking' that maintains the integrity of related text. When iterating through doc.chunks(), use chunk.to_context_text() to get text that includes contextual information from section headers and nested headers. This is ideal for building vector indices for Retrieval Augmented Generation (RAG).

from llama_index.core import Document, VectorStoreIndex

index = VectorStoreIndex([])
for chunk in doc.chunks():
    # to_context_text() includes contextual information from section headers
    index.insert(Document(text=chunk.to_context_text(), extra_info={}))

query_engine = index.as_query_engine()
response = query_engine.query("list all the tasks that work with bart")
print(response)

You can integrate LayoutPDFReader with LlamaIndex by iterating through the document chunks and inserting them into a VectorStoreIndex. Use chunk.to_context_text() to ensure the chunk includes hierarchical context.

from llama_index.core import Document
from llama_index.core import VectorStoreIndex

index = VectorStoreIndex([])
for chunk in doc.chunks():
    index.insert(Document(text=chunk.to_context_text(), extra_info={}))
query_engine = index.as_query_engine()

response = query_engine.query("list all the tasks that work with bart")
print(response)

You can navigate the document hierarchy using doc.sections() to find specific headers. To extract the full content of a section including all its nested subsections, use section.to_html(include_children=True, recurse=True). This HTML can then be passed to an LLM for targeted summarization.

from IPython.core.display import display, HTML
selected_section = None
# find a section in the document by title
for section in doc.sections():
    if section.title == '3 Fine-tuning BART':
        selected_section = section
        break

# use include_children=True and recurse=True to fully expand the section.
# include_children only returns at one sublevel of children whereas recurse goes through all the descendants
HTML(section.to_html(include_children=True, recurse=True))

To parse a PDF with layout information, initialize LayoutPDFReader with the LLMSherpa API URL and use the read_pdf method. The method accepts either a URL or a local file path.

from llmsherpa.readers import LayoutPDFReader

llmsherpa_api_url = "https://readers.llmsherpa.com/api/document/developer/parseDocument?renderFormat=all"
pdf_url = "https://arxiv.org/pdf/1910.13461.pdf" # also allowed is a file path e.g. /home/downloads/xyz.pdf
pdf_reader = LayoutPDFReader(llmsherpa_api_url)
doc = pdf_reader.read_pdf(pdf_url)

If you need to process the full parsed structure manually, you can access the complete JSON response returned by the LLMSherpa service via the .json attribute on the document object.

doc.json

Access tables in a document using doc.tables(). You can convert a specific table to HTML using table.to_html() and pass that string to an LLM to perform complex analysis or answer questions about the tabular data.

from llama_index.llms import OpenAI
context = doc.tables()[5].to_html()
resp = OpenAI().complete(f"read this table and answer question: which model has the best performance on squad 2.0:\n{context}")
print(resp.text)

To access the complete raw JSON response returned by the LLM Sherpa service, use the .json attribute on the document object.

doc.json