3

Idyllic Chat Bot — Streaming AI Chat on Next.js and Vercel KV

The AI chat component of the Idyllic web app — a Next.js 13 App Router chatbot that streams OpenAI responses from an edge route, persists per-user history in Vercel KV and signs users in with GitHub via Auth.js.

GitHub Website

Overview

The conversational AI piece of the Idyllic web app, built in mid-2023 when streaming chat on the edge was new. It starts from Vercel's open-source Next.js AI Chatbot template, which I adapted for Idyllic's authentication and deployment. Signed-in users hold multi-turn conversations whose replies stream token by token, revisit their history from a sidebar and share any conversation via a public link.

Live at idyllic-chat-bot.vercel.app.

How it works

  • Edge chat route. POST /api/chat runs on the edge runtime. It rejects unauthenticated requests outside local development, then calls OpenAI's chat completions API (gpt-3.5-turbo, temperature 0.7, stream: true) and pipes the response back through the Vercel AI SDK's OpenAIStream and StreamingTextResponse, so the UI renders tokens as they arrive.
  • Bring-your-own key. A request can carry a preview API key, which is used in place of the server key. That lets people try the bot without the deployment paying for every token.
  • Persistence in Vercel KV. When a completion finishes, the route stores the whole conversation as a Redis hash (chat:{id}) and indexes it in a per-user sorted set (user:chat:{userId}) ordered by time. History loads with a pipelined hgetall, so the sidebar is one round trip regardless of how many chats a user has.
  • Server Actions. Listing, opening, deleting, clearing and sharing chats are Next.js Server Actions rather than bespoke API endpoints, keeping the data access on the server and the client thin.
  • Sharing. A shared chat gets a public /share/[id] page with its own generated Open Graph image, so links unfurl with the conversation's title.
  • Auth. Auth.js (NextAuth) with the GitHub provider and JWT sessions; middleware protects the app routes.
  • UI. React Server Components, shadcn/ui on Radix primitives, Tailwind, and Markdown rendering with GitHub-flavoured Markdown, math and syntax-highlighted code blocks.

What I'd change today

  • Move from the deprecated openai-edge client to the current AI SDK provider model, which also makes switching between OpenAI and Anthropic models a one-line change.
  • Add per-user rate limiting in front of the chat route (Upstash Ratelimit on the same Redis), and cap conversation length sent as context.
  • Replace the template README with Idyllic-specific docs.

Built with

  • Next.js 13 (App Router, React Server Components, Server Actions, edge runtime)
  • Vercel AI SDK, OpenAI Chat Completions (gpt-3.5-turbo)
  • Vercel KV (Redis), Auth.js with GitHub OAuth
  • Tailwind CSS, shadcn/ui, Radix UI, react-markdown