The problem
A chat model can sound confident even when it has not seen your document. If you ask about your college rulebook, internship policy, or a startup pitch deck, the model may answer from general patterns instead of your file.
For a document QA app, that is not enough. Your user needs answers that come from the uploaded documents and can be checked.
One idea
Do not ask the model to remember your documents. Retrieve the relevant parts first, then ask the model to answer only from those parts.
Memory vs retrieval
Model memory means the information learned during training, plus whatever is in the current chat. It is broad, but not reliable for your private PDFs, notices, invoices, resumes, or project docs.
Retrieval means your app searches your own document collection at answer time. It finds small pieces of text that match the question, puts them into the prompt, and asks the model to answer using that evidence.
Tiny prompt difference
A memory-only prompt asks for an answer directly. A retrieval prompt includes evidence and clear rules.
Memory-only:
Question: What is the late submission penalty?
Answer the question.
Retrieval-grounded:
Evidence:
[doc: assignment_policy.txt, lines 12-14]
Late submissions lose 10% marks per day. Submissions more than 5 days late are not accepted.
Question: What is the late submission penalty?
Answer using only the evidence. If the evidence is not enough, say you do not know.Common mistake
Many first RAG projects upload a PDF, ask the model questions, and assume the answer is grounded. That is not a QA system. That is a chat prompt with a file nearby.
A recruiter inspecting your repo should see the flow: documents, chunks, search, evidence in prompt, answer, citations, and a small grounding check.
Think like an examiner
In an exam, an answer without a source is weak. In document QA, the source is the retrieved text. Your app should make it easy to ask: where did this answer come from?
This is why your final repo will include sample documents, example questions, cited answers, and a short evaluation. You are proving that the app can answer from evidence.
Basic RAG flow
Do each one yourself, then tap it to tick it off. The ticks are only a checklist for you: they are not marked or scored.
0 of 5 done
Do this now
Do each one yourself, then tap it to tick it off. The ticks are only a checklist for you: they are not marked or scored.
0 of 5 done