Skip to content

File Parser optimised for LLM Ingestion with no loss 🧠 Parse PDFs, Docx, PPTx in a format that is ideal for LLMs.

License

Notifications You must be signed in to change notification settings

QuivrHQ/MegaParse

Folders and files

NameName
Last commit message
Last commit date

Latest commit

 

History

95 Commits
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 

Repository files navigation

MegaParse - Your Mega Parser for every type of documents

Quivr-logo

MegaParse is a powerful and versatile parser that can handle various types of documents with ease. Whether you're dealing with text, PDFs, Powerpoint presentations, Word documents MegaParse has got you covered. Focus on having no information loss during parsing.

Key Features 🎯

  • Versatile Parser: MegaParse is a powerful and versatile parser that can handle various types of documents with ease.
  • No Information Loss: Focus on having no information loss during parsing.
  • Fast and Efficient: Designed with speed and efficiency at its core.
  • Wide File Compatibility: Supports Text, PDF, Powerpoint presentations, Excel, CSV, Word documents.
  • Open Source: Freedom is beautiful, and so is MegaParse. Open source and free to use.

Support

  • Files: ✅ PDF ✅ Powerpoint ✅ Word
  • Content: ✅ Tables ✅ TOC ✅ Headers ✅ Footers ✅ Images

Example

megaparse.mp4

Installation

pip install megaparse

Usage

  1. Add your OpenAI API key to the .env file

  2. Install poppler on your computer (images and PDFs)

  3. Install tesseract on your computer (images and PDFs)

from megaparse.Converter import MegaParse

megaparse = MegaParse(file_path="./test.pdf")
document = megaparse.load()
print(document.content)
megaparse.save_md(content, "./test.md")

(Optional) Use LlamaParse for Improved Results

  1. Create an account on Llama Cloud and get your API key.

  2. Call Megaparse with the llama_parse_api_key parameter

from megaparse.Converter import MegaParse

megaparse = MegaParse(file_path="./test.pdf", llama_parse_api_key="llx-your_api_key")
document = megaparse.load()
print(document.content)

BenchMark

Parser Diff
LMM megaparse 36
Megaparse with LLamaParse and GPTCleaner 74
Megaparse with LLamaParse 97
Unstructured Augmented Parse 99
LLama Parse 102
Megaparse 105

Lower is better

Next Steps

  • Improve Table Parsing
  • Improve Image Parsing and description
  • Add TOC for Docx
  • Add Hyperlinks for Docx
  • Order Headers for Docx to Markdown

Star History

Star History Chart