Gate the chat with a client-side keyword filter before spending a token

#1—Context

A public 'chat with me' box is an open invitation to use it as a free ChatGPT. Every off-topic message costs tokens and eats into the rate limit that protects the endpoint from abuse.

#2—Options considered

A
Let everything through — Burns budget on off-topic questions and invites the endpoint to be used as a free general-purpose LLM.
B
Server-side LLM classifier — Accurate, but adds a model call per message — paying tokens to decide whether to pay tokens.
C
Client-side keyword gatekeeper — A cheap check in the browser rejects clearly off-topic questions instantly, before any request leaves the client. Free and zero-latency.

#3—Decision

A keyword gatekeeper in the client checks each question against on-topic and off-topic vocabularies and blocks the clearly-irrelevant ones with a polite redirect — before any network call. On-topic questions proceed to the rate-limited, validated API route.

#4—Trade-offs

Keyword matching is crude: some off-topic phrasings slip through and some on-topic ones get wrongly blocked. Acceptable for a first-line filter — the server-side rate limit and input validation are the real guardrails.

#5—Revisit trigger

If false-blocks start frustrating genuine visitors, replace the keyword list with a small, cheap server-side classifier while keeping the instant client check as a pre-filter.

#6—Takeaways

01
Push cheap filtering to the edge; reserve the expensive path for traffic that deserves it.
02
Crude-but-instant often beats correct-but-costly for a first-line guard — as long as a real backstop (rate limit) sits behind it.
03
Don't spend a model call to decide whether to spend a model call.

#7—Full write-up

Why a filter at all

An open chat box on a public site gets used as a free LLM. Each off-topic message costs tokens and burns the rate limit meant to stop abuse. The cheapest possible defense is to never send the request in the first place.

Why keywords, not a classifier

A server-side classifier would be more accurate, but it means a model call to decide whether to make a model call. A keyword check in the browser is free and instant. It's crude, so it sits in front of — not instead of — the server rate limit and input validation that are the actual backstop.

Chat with Shoaib

Chat with Shoaib

Gate the chat with a client-side keyword filter before spending a token

Why a filter at all

Why keywords, not a classifier