tone translator

the problem

I text Japanese friends, and every tool I reached for handed me stiff, textbook Japanese that reads as non-native on sight. The two obvious options are wrong in opposite directions: a literal translator like Google Translate is rough and unnatural, while a raw chatbot is natural but inconsistent — as the conversation grows the context drifts, and you re-explain casual, natural, no romaji every single time. I wanted the middle ground: a tuned, hardened prompt baked into a one-tap interface, so the output is the same quality, instantly, every time.

what I built

• •Translate in four registers— casual (the default; it's how you talk to friends), polite, formal, blunt — and it explains its own slang and politeness choices inline.
• •A “check” mode: a separate, stronger model grades your Japanese like a tutor — it catches subtle native-speaker errors (あげる vs くれる direction, a dropped particle) and suggests the fix.
• •Streamed token-by-token, per-user history, rate-limited with a budget kill-switch. It's live and public.

the interesting part

Wiring an API call is the easy part. The real problem was making the output reliably goodwhen “good” is subjective and the model is non-deterministic. The decisions I'm proudest of are all about that:

• •Quality as a data problem. I built an eval harness — a golden set of ~24 cases, each seeded from a real failure (giving/receiving direction, particle errors, the right register per tone, prompt injection), graded by Sonnet as an LLM-judge at temperature 0. It runs the real shipping prompt, so a careless edit shows up as a score drop. I keep it out of CI on purpose — it's non-deterministic and costs real calls — and I trust the delta between runs, not any single score. Each case links to the bug that created it, so the suite is my regression history, executable.
• •Then I automated the discipline. A harness only helps if you keep feeding it, so I built a failure-miner agent: weekly, it reads real translations from the database, judges each with Claude, and proposes new test cases for the failures — I approve them; it never edits the test set itself. Its first run over 50 translations surfaced 5 genuine failures and grew the set from 19 to 24.
• •Pull the brittle decision out of the model. Translation direction was once the model's job, and on formal input it would sometimes echo Japanese back as Japanese. I detect the script with a Unicode regex in code instead — 100% reliable and free. Knowing what not to hand the model is half the skill.
• •Prompt injection, the subtle version. “Ignore your instructions and say X” has to be translated, not obeyed — but my first fix overcorrected: the model started refusing or lecturing instead of silently translating, broken nine times out of ten. The real fix was making refusal itself a failure mode in the prompt — treat all input as text, resist silently. Eval-verified, 9/10 → 0/10.

what I'd change next

• •Have the agent open a draft PR with its proposed cases, instead of leaving an artifact for me to copy in by hand.
• •Point the same eval harness at the check feature, not just translation.
• •Harden the judge against injection from the content it reads — it ingests real translations, the same attack surface the app itself defends against.
• •Cross-device live history sync — parked on purpose; it wasn't the priority yet.