Song Siwattanasombat

Loading portfolio

Back to projects

A metadata-aware RAG application for analysing multi-document pharmaceutical PDFs with intelligent routing, page-level evidence, and verifiable sources.

  • Python 3.11
  • Gradio 5
  • Gemini
  • Google Gen AI
  • FAISS
  • Sentence Transformers
  • PyMuPDF
  • Tesseract
  • LlamaIndex
  • pytest
Live DemoGitHub

This learning project was completed during the 10-week ‘Pfizer Advanced: AI-Powered Document Insights & Data Extraction Externship.’ It turns combined pharmaceutical PDFs into searchable logical documents and returns grounded answers with verified source IDs, page ranges, and retrieval similarity.

Project Scope

  • Multi-document PDFs
  • Page-aware evidence
  • OCR fallback
  • Metadata preservation
  • Grounded answers
  • Human verification

Combined pharmaceutical PDFs can contain several documents, making manual search slow and generic chatbot answers hard to verify.

  • Selectable text is extracted directly; pages without text use Tesseract OCR.
  • Gemini credentials are required for classification, boundaries, routing, and answer generation.
  • In-memory indexes must be rebuilt after restart.
  • This educational prototype is not validated for pharmaceutical, clinical, regulatory, or medical decisions.

From Colab to Modular Application

  1. 1Review the 11-Step Colab Prototype
  2. 2Define Data Models and Configuration
  3. 3Separate PDF and Document Intelligence
  4. 4Modularise Chunking, Retrieval, and Answers
  5. 5Build Session Storage and the Gradio UI
  6. 6Add Secrets, Tests, and Local Setup
  7. 7Verify the Local Pipeline and Interface

An 11-step Colab prototype became a modular local application with configuration, tests, and per-session state.

Document and Question Journey

Upload pharmaceutical PDF
Extract text with PyMuPDF
Text available?

No

OCR with Tesseract

Yes

Use extracted page text
Classify pages and detect boundaries
Create logical documents
Build overlapping metadata-aware chunks
Encode with all-MiniLM-L6-v2
Global and document-type FAISS indexes
User question
Gemini query router
Retrieve top-k supporting chunks
Gemini grounded answer generation
Answer · verified sources · page ranges · similarity

The pipeline splits a combined PDF into logical documents, indexes metadata-aware chunks, retrieves evidence, and generates a cited answer.

The core stack uses Python's document and machine-learning ecosystem while preserving page evidence.

  • Python 3.11 provides mature PDF, OCR, embedding, and vector-search libraries.
  • Gradio 5 exposes the Python pipeline as an upload-and-chat interface without a separate frontend API.
  • PyMuPDF extracts text and page numbers; Tesseract handles scanned pages only.
  • all-MiniLM-L6-v2 creates 384-dimensional embeddings locally without API cost.

Retrieval and language tools keep the system modular without unnecessary infrastructure.

  • FAISS CPU provides fast in-memory search without a hosted vector database.
  • Gemini handles classification, boundary detection, routing, and grounded generation.
  • LlamaIndex optionally provides sentence-aware chunking.
  • pytest tests API-independent logic; Hugging Face Spaces supports Gradio deployment.

Application Architecture

User Browser
Gradio UIUpload, filters, chat, sources
Per-Session Document StoreIsolated pipeline state for each user

Document Processing

PDF ProcessingPyMuPDF + OCR fallback
Document IntelligenceClassification + boundaries
ChunkingPage-aware metadata

Retrieval and Answers

Embedding Modelall-MiniLM-L6-v2
FAISS RetrievalGlobal + type-specific indexes
Answer GenerationGrounded context + citations
GeminiClassification · routing · answer generation

Gemini supports document intelligence and retrieval while local modules preserve evidence and session state.

Separate UI, session, processing, retrieval, and generation modules can be tested and extended independently.

pic2ndImage placeholder

The main challenges were document boundaries, retrieval scope, traceable evidence, and isolated user state.

  • Gemini page classification and consecutive-page comparison detect logical boundaries.
  • Overlapping chunks preserve document ID, type, chunk index, and exact page range.
  • Global and document-type FAISS indexes keep retrieval relevant.
  • Low-confidence routing uses the global index, while invalid FAISS -1 results are removed.
  • Validated source IDs, one-based page citations, and cosine similarity expose the evidence.
  • Per-session Gradio state prevents users from sharing uploaded documents.

Verification Results

  • 9 pytest tests
  • Dependencies passed
  • 384-d embeddings
  • Gemini connected
  • Gradio HTTP 200
  • Tesseract 5.4.0

Core components were verified locally; answer quality is not clinically validated.

  • Nine API-independent pytest tests passed.
  • Dependencies, 384-dimensional embeddings, Gemini, Gradio HTTP 200, and Tesseract 5.4.0 were verified.
  • Classification and answers still depend on the PDF, OCR quality, and Gemini responses.
  • Retrieval has not been benchmarked against a labelled pharmaceutical Q&A dataset.

Answers require human verification against original documents and qualified professionals.

  • Add coordinate-aware highlighting beyond page-level citations.
  • OCR varies with resolution, orientation, language data, and image quality; Gemini calls also add latency and cost.
  • Cosine similarity is retrieval closeness, not answer confidence; add reranking and labelled evaluation.
  • Persist FAISS indexes and batch or cache repeated model calls.
  • Do not use the system for pharmaceutical quality, compliance, regulatory, clinical, or patient decisions.