LLM APIs - lesson 3 | HackIndia Skills

LLM APIs - lesson 3 | HackIndia Skills | HackIndia

Example

Server route

This example shows the shape you want in a Node-style server route. Keep your real product rules in instructions. Keep the user’s text in input.

Older tutorials may show Chat Completions first. For this track, use Responses API for new text features.

import OpenAI from "openai";

const client = new OpenAI({
  apiKey: process.env.OPENAI_API_KEY,
});

export async function POST(req) {
  const { prompt } = await req.json();

  const stream = await client.responses.create({
    model: "gpt-4o-mini",
    instructions: "You are a concise study assistant. Explain in simple steps.",
    input: prompt,
    stream: true,
  });

  const encoder = new TextEncoder();

  return new Response(
    new ReadableStream({
      async start(controller) {
        try {
          for await (const event of stream) {
            if (event.type === "response.output_text.delta") {
              controller.enqueue(encoder.encode(event.delta));
            }
          }
          controller.close();
        } catch (error) {
          controller.error(error);
        }
      },
    }),
    {
      headers: {
        "Content-Type": "text/plain; charset=utf-8",
      },
    }
  );
}

Example

UI reader

The browser calls only your route. It reads the response body chunk by chunk and appends decoded text to state.

This keeps the API key out of the client and still gives a live typing effect.

async function askAI(prompt) {
  setAnswer("");

  const res = await fetch("/api/ai/stream", {
    method: "POST",
    headers: { "Content-Type": "application/json" },
    body: JSON.stringify({ prompt }),
  });

  if (!res.ok || !res.body) {
    setAnswer("Something went wrong. Try again.");
    return;
  }

  const reader = res.body.getReader();
  const decoder = new TextDecoder();

  while (true) {
    const { value, done } = await reader.read();
    if (done) break;
    setAnswer((prev) => prev + decoder.decode(value, { stream: true }));
  }
}

Stream replies to the UI

One idea

Why stream

Data path

Server route

UI reader

Common mistake

Make it demoable

Do this now

Quick check