BuntokBuntok

AI Module#

Built-in AI integration with streaming support, response caching, and prompt injection. Works with OpenAI, Anthropic, and any OpenAI-compatible API.

streamAI#

Transform any AI stream into a Vercel AI SDK compatible response (Data Stream Protocol v1):

typescript
import { streamAI } from "@buntok/core";
import OpenAI from "openai";

const openai = new OpenAI();

app.post("/chat", async (ctx) => {
  const { message } = await ctx.body();

  const stream = await openai.chat.completions.create({
    model: "gpt-4",
    messages: [{ role: "user", content: message }],
    stream: true,
  });

  return streamAI(ctx, stream, {
    onCompletion: (fullText) => {
      console.log("Full response:", fullText);
    },
  });
});

streamAI API#

ParameterTypeDescription
ctxContextBuntok request context
streamAsyncIterableAI response stream (OpenAI, Anthropic, etc.)
options?objectonCompletion callback
Info
Supported chunk shapes: chunk.choices[0].delta.content (OpenAI), chunk.message.content (Anthropic), or plain string. If your AI provider uses a different chunk shape, content extraction may fail silently (empty chunks).

injectSystemPrompt#

Safely inject a system prompt while removing any existing system messages from user input (prevents prompt injection):

typescript
import { injectSystemPrompt } from "@buntok/core";

const userMessages = [
  { role: "user", content: "Hello!" },
];

// Inject system prompt (prepends it, removes existing system messages)
const messages = injectSystemPrompt(
  userMessages,
  "You are a helpful assistant. Be concise."
);
// Result:
// [
//   { role: "system", content: "You are a helpful assistant. Be concise." },
//   { role: "user", content: "Hello!" }
// ]
Warning
injectSystemPrompt removes any role: "system" messages from user input. This prevents users from overriding your system prompt.

AICache#

Cache AI responses to save costs. Hashes the last few messages to generate cache keys:

typescript
import { AICache } from "@buntok/core";
import { Cache } from "@buntok/core/cache";

const aiCache = new AICache(new Cache());

app.post("/chat", async (ctx) => {
  const { messages } = await ctx.body();

  // Check cache first
  const cached = await aiCache.get(messages);
  if (cached) {
    return ctx.json({ response: cached, cached: true });
  }

  // Call AI
  const response = await callAI(messages);

  // Cache result (default TTL: 3600s)
  await aiCache.set(messages, response, 3600);

  return ctx.json({ response, cached: false });
});

AICache API#

MethodDescription
aiCache.get(messages)Get cached response (returns null if miss)
aiCache.set(messages, response, ttl?)Cache a response

Full Example#

typescript
import { Buntok } from "@buntok/core";
import { streamAI, injectSystemPrompt, AICache } from "@buntok/core";
import { Cache } from "@buntok/core/cache";
import OpenAI from "openai";

const app = new Buntok();
const openai = new OpenAI();
const aiCache = new AICache(new Cache());

const SYSTEM_PROMPT = "You are a helpful coding assistant. Be concise.";

app.post("/chat", async (ctx) => {
  const { messages } = await ctx.body();

  // Inject system prompt safely
  const safeMessages = injectSystemPrompt(messages, SYSTEM_PROMPT);

  // Check cache
  const cached = await aiCache.get(safeMessages);
  if (cached) {
    return ctx.json({ response: cached, cached: true });
  }

  // Stream response
  const stream = await openai.chat.completions.create({
    model: "gpt-4",
    messages: safeMessages,
    stream: true,
  });

  return streamAI(ctx, stream, {
    onCompletion: async (fullText) => {
      await aiCache.set(safeMessages, fullText);
    },
  });
});

app.listen(1212);
PreviousDate LibraryNext Vector Search