Start with a real user task
Pick a narrow workflow: internal FAQ with citations, support draft assist, or a document Q&A for a defined corpus. Avoid ChatGPT clone as a portfolio pitch because interviewers have seen hundreds.
Write success criteria users would recognise: answer groundedness, time saved, escalation rate. Mentors reject vanity demos without a user sentence you can say out loud.
Freeze scope for the first version. Tool-using agents can wait until retrieval quality is measured and failure cases are listed honestly.
Build retrieval before prompt theatre
Chunking, embeddings, metadata filters and citation behaviour matter more than a clever system prompt. Document how documents enter the index and how you refresh them when content changes.
Compare at least one baseline — keyword or simple embed — against your main approach. Interviewers ask what you tried first and why you moved on.
LLM mentor depth for model choice and prompting sits at mentorkhoj.com/find/llm-mentor; GenAI app architecture sits at mentorkhoj.com/find/genai-mentor.
Log a handful of retrieved chunks for failing questions. Retrieval autopsy teaches more than another system-prompt rewrite and gives MentorKhoj mentors concrete review material.
Document evals and failure cases
Create a small golden set of questions with expected behaviours. Score groundedness, refusal quality and latency. Automate what you can; hand-label what you must without pretending automation covers judgement.
List failure cases: outdated docs, ambiguous queries, hallucinated citations. Show mitigations you tried and what still breaks.
If evals are missing, the project is a toy. MentorKhoj sessions should open with your golden set, not your UI theme or chat bubble colours.
Version your golden set. When you change chunking or prompts, record metric deltas so you can prove the change helped instead of relying on vibes.
Practical six-week GenAI roadmap
Week 1: user task, corpus, success metrics. Week 2: ingestion and baseline retrieval. Week 3: improved RAG and citations. Week 4: eval harness and failure log. Week 5: hardening and cost or latency notes. Week 6: write-up and mentor mock interview.
Keep a decision log: chunk size changes, model swaps, prompt edits — with metric impact. That log becomes interview gold because it shows engineering judgement.
Claim ₹100 mentor credit early so weekly reviews happen before week-six panic when the write-up is still empty.
Before you call the project done, record a five-minute walkthrough covering user task, retrieval path, eval numbers and top failure case for your next mentorkhoj.com/ai demo review.
Production thinking without overclaiming
Discuss auth, rate limits, cost ceilings and observability even in a demo. Precision beats fake production-ready badges that collapse under one follow-up.
Note privacy constraints for your corpus. Hiring managers care whether you thought about data boundaries before you shipped a public demo link.
Ship a minimal UI only after the retrieval-eval loop works. Pretty chat boxes do not hide weak grounding from someone who asks for citation behaviour.
Agent workflows come after measured RAG
Add tools only when the base retrieval path is measured. Agents amplify bad retrieval into confident wrong actions, which is worse in an interview story than a simple RAG with known limits.
If you do add an agent step, document tool choice, guardrails and eval cases for tool misuse. Mentors will ask how you prevent loops and unsafe calls.
Keep the user task narrow even with agents. Broad autonomy demos rarely survive scrutiny without deep eval investment.
Publish a short architecture note with data flow and eval summary even if the UI is minimal. Interviewers often read that note more carefully than the chat interface.
Walk the GenAI path on MentorKhoj
Book a free demo at mentorkhoj.com/ai, match via mentorkhoj.com/find/genai-mentor, and use mentorkhoj.com/find/llm-mentor when model-side depth is the bottleneck.
Claim credit at mentorkhoj.com/claim-free-credit, then keep weekly path-walker reviews until you can defend architecture and evals live.
End state: one narrow GenAI app, measured quality, documented failures, and a talk track you can deliver under pressure without reading the README.
Version your golden set and log retrieved chunks for failing questions. Retrieval autopsy teaches more than another prompt rewrite and gives mentors concrete review material at mentorkhoj.com/find/genai-mentor.
Record a five-minute walkthrough covering user task, retrieval path, eval numbers and top failure case before your next mentorkhoj.com/ai session. That artefact doubles as interview practice.
Prefer a narrow corpus you can refresh over a giant scraped dump you cannot explain. Mentors on mentorkhoj.com/find/llm-mentor and mentorkhoj.com/find/genai-mentor will ask how documents enter and leave the index.
Measure latency and cost per answer even in a demo. Hiring managers ask these questions early, and MentorKhoj reviews should include a one-line cost note beside quality metrics.
Refuse to expand into multi-agent theatre until single-hop retrieval quality is stable on your golden set for two consecutive weekly reviews.
