Most students learning generative AI write a prompt, try it twice and move on. Developers test. Here is a weekend project that turns prompt testing into code: a tiny Node.js script that runs one prompt over a list of test inputs and checks each answer against simple rules.

It doesn't depend on any one AI provider. You plug in whichever API you have access to.

What you will build

  • cases.json: 10 test inputs, each with rules the answer must pass
  • test.mjs: sends each input with your prompt, checks the rules, prints a pass/fail table

Step 1: the test cases 

[ { "input": "Library closes at 6 pm on Saturday from next week.", "mustInclude": ["6 pm", "Saturday"], "maxWords": 25 }, { "input": "Fee deadline for semester 5 is extended to 20 November.", "mustInclude": ["20 November"], "maxWords": 25 } ]

The prompt's job here: summarise a college notice in one short sentence without losing dates and times. Write at least 10 cases, including awkward ones (two dates, Hindi-English mixed text, an empty notice).

Step 2: the checker 

// test.mjs (Node 18+; .mjs so import and top-level await work) import fs from "node:fs"; const PROMPT = "Summarise this college notice in one sentence. Keep every date and time exactly as written."; const cases = JSON.parse(fs.readFileSync("cases.json", "utf8")); async function ask(text) { // Replace with your provider's API call. It must return the model's reply as a string. throw new Error("connect your AI API here"); } function check(answer, c) { const problems = []; for (const word of c.mustInclude || []) { if (!answer.includes(word)) problems.push(`missing "${word}"`); } const words = answer.trim().split(/\s+/).length; if (c.maxWords && words > c.maxWords) problems.push(`${words} words`); return problems; } let passed = 0; for (const c of cases) { const answer = await ask(`${PROMPT}\n\nNotice: ${c.input}`); const problems = check(answer, c); if (problems.length === 0) passed++; console.log(problems.length ? "FAIL" : "PASS", "|", c.input.slice(0, 40), "|", problems.join(", ")); } console.log(`${passed}/${cases.length} passed`);

Keep your API key in an environment variable, never in the file, and never commit it to GitHub.

Step 3: improve the prompt, not the tests

Run it, look at the failures, change one thing in PROMPT, run again. Write down each change and the new score. After three or four rounds you will have a short, honest record like "v1 7/10, v2 9/10 (added 'keep dates exactly')". That record is a far better interview story than "I know ChatGPT".

Ideas to extend it

  • Run each case 3 times, because AI answers vary.
  • Add a mustNotInclude rule to catch made-up details.
  • Save results to a CSV and compare two models.

I put this together for students at Dzimark Training in Mohali, where the AI programmes cover prompt engineering, AI agents and AI-powered full-stack work. If you are in the Tricity and want the structured version, the course outline is on the Dzimark Training AI programmes page.