Build cautious local literature retrieval and RAG

16 GB RAM is sufficient for retrieval. A local 8B model generally benefits from 16 GB unified memory or suitable GPU memory.

Component	Role	Price	Link
PyMuPDF 1.28.0	page-preserving PDF text extraction	Free / 免费open-source package; confirm AGPL or commercial licensing fits the deploymentchecked 2026-08-02	https://pypi.org/project/PyMuPDF/1.28.0/
rank-bm25 0.2.2	transparent local lexical retrieval	Free / 免费Apache-2.0 Python packagechecked 2026-08-02	https://pypi.org/project/rank-bm25/0.2.2/
Ollama 0.32.5	optional local answer model	Free / 免费local runtime; model license, hardware and electricity are separatechecked 2026-08-02	https://github.com/ollama/ollama/releases/tag/v0.32.5

Save as retrieve.py

import glob, re, sys, fitz
from rank_bm25 import BM25Okapi
rows=[]
for path in glob.glob('papers/*.pdf'):
    with fitz.open(path) as doc:
        for page_no, page in enumerate(doc, 1):
            text=' '.join(page.get_text().split())
            for i in range(0, len(text), 1200):
                chunk=text[i:i+1400]
                if chunk: rows.append((path, page_no, chunk))
tok=lambda s: re.findall(r'\w+', s.lower())
index=BM25Okapi([tok(r[2]) for r in rows]); query=' '.join(sys.argv[1:])
for rank, idx in enumerate(sorted(range(len(rows)), key=lambda i:index.get_scores(tok(query))[i], reverse=True)[:8],1):
    path,page,text=rows[idx]; print(f'[E{rank}] {path} p.{page}\n{text}\n')

Retrieve before asking the model

uv venv --python 3.12 .venv
. .venv/bin/activate
uv pip install 'PyMuPDF==1.28.0' 'rank-bm25==0.2.2'
python retrieve.py 'your precise research question' > evidence.txt
# Read evidence.txt first. Only then pass the contract and retrieved passages together.
(cat grounded_prompt.txt; printf '\n'; cat evidence.txt) | ollama run qwen3:8b

Save as grounded_prompt.txt after appending evidence.txt

Answer only from the supplied evidence blocks. Every factual sentence must end with one or more exact labels such as [E2]. If the passages conflict, describe the conflict. If they do not answer the question, say '本地文献证据不足' and stop. Never invent a title, author, year, DOI, quotation or page. Distinguish direct findings from your synthesis. End with an evidence table mapping each claim to its file and page.

Question: <paste question>

Evidence:
<paste evidence.txt>

Known pitfalls

• Retrieval hallucination occurs when the model cites a retrieved label that does not support its sentence; manually compare every claim with the labelled passage.
• A plausible citation label is not proof that the source exists or supports the claim.
• Lexical retrieval can miss synonyms and multilingual terminology; repeat searches with documented query variants.
• PDF extraction can scramble columns and omit figures or equations.
• Do not treat a local collection as exhaustive literature coverage.
• Model output must never silently fill gaps in the retrieved evidence.