Adjay Mistry-Katz

Consumer web + native iOS · AI with guardrails

Lyra — a language platform that teaches

A language-learning platform that actually teaches — grammar, conjugation, and vocabulary in pedagogical order with real spaced repetition — rather than testing random words. Four languages are live (French, Spanish, Italian, Portuguese) on a React web app and a native SwiftUI iOS companion sharing one API. The course is drawn as a galaxy: CEFR level = galaxy, unit = star, lesson = planet.

6 languages live
90+ units · ~3,000 vocab items
FSRS-4.5 scheduler
20 exercise formats
Azure + Whisper speech
Web + native iOS

Lyra's web course view: the whole French course drawn as three visible galaxies of stars against a starfield, labelled French A1 with 30 of 30 lessons done, French A2 with 4 of 38, and French B1 with 0 of 30, plus a 'fading' marker for items due for review — **A whole language, one screen.** CEFR level is a galaxy, a unit is a star, a lesson is a planet in orbit around it. Zoom is the navigation model — tap a galaxy to fall into its stars, tap a star to land on its planets — so a learner's position in 128 lessons is a place rather than a progress bar. The comet bottom-left is the review queue.

The map is the data model

Nothing about the galaxy is decorative: star positions, orbital radii and planet ordering are derived deterministically from the course structure itself — unit index, lesson count, prerequisite chain — so the same course always draws the same sky, and adding a unit moves nothing that already exists. The layout functions are the most heavily mirrored code in the platform: the Swift implementation is a named twin of the web one and is pinned by exact-value tests, because a learner who switches devices has to recognise the same constellation.

Lyra's iOS course map zoomed to the French A2 galaxy: a vertical chain of star systems labelled with unit names — Le conditionnel présent, Les phrases avec si, Le plus-que-parfait, Les pronoms relatifs — each with lesson planets in orbit and a lesson count, next to a comet marked 'four fading, tap to review' — **Inside a galaxy.** Each star is a unit, sized and lit by how much of it is learned, chained in pedagogical order — you can see the course's dependency structure without a syllabus. Planets carry their lesson number, and the comet is the spaced repetition queue asking for attention.

Lyra's iOS unit view for 'L'imparfait I', with a plain-English description of what the unit teaches, a zero percent progress ring, and four numbered lesson planets orbiting a sun, each labelled with its topic — the nous-stem rule, être à l'imparfait, la météo, la campagne d'autrefois — **Inside a star.** The unit states what it will teach you in plain English, then its lessons orbit as planets in the order you should take them — grammar rule, the irregular verb, then the two topics that use them. Twelve items from earlier units are fading and can be reviewed from here.

A lesson teaches before it tests

Each lesson runs as a fixed sequence of phases — grammar explanation, conjugation tables, vocabulary, then drills — rather than a shuffled quiz. That ordering is the pedagogical claim of the product: presentation is a first-class schedulable item, so a conjugation table you already know can be marked as such and demoted in the queue, and the twenty exercise formats only ever draw on vocabulary the course has already introduced.

A Lyra lesson in its conjugation phase: a header for 'Être à l'imparfait' in unit L'imparfait I, a three-step phase bar reading Grammar, Conjugation, Vocabulary, and the full imparfait conjugation of être with a control to mark the table as already solid — **The conjugation phase.** The phase bar is the lesson's contract — grammar, then conjugation, then vocabulary, then drills. The controls beside the table let the learner mark it already solid, hard, or unclear; that feeds the scheduler rather than just skipping the screen.

A 'tap the mistake' exercise: the French sentence 'Sophia avait très timide, mais maintenant elle est bavarde' with its English translation, the word 'avait' selected and marked correct, and feedback buttons reading 'too easy', a tick, a question mark, and 'bad exercise' — **One of twenty exercise formats,** graded deterministically on-device for instant feedback and re-graded server-side as the authority. The four buttons after the answer are the interesting part: *too easy* and *hard* feed the spaced-repetition rating, and *bad exercise* files a content flag that lands in a triage queue against the source unit.

Scheduling is the product

FSRS-4.5 spaced repetition runs server-side only: clients submit graded results, never schedules. Ratings come from grading, not self-report — wrong, almost, correct, and too-easy map to scheduler ratings — and grammar and conjugation concepts are first-class schedulable items alongside vocabulary. Deterministic grading runs client-side for instant feedback and is mirrored server-side as the authority; AI grading is a capped fallback for open answers only, never for choice formats.

Lyra's progress screen: a five-day streak, four items due now, twenty-four answers today, an offer to take an adaptive placement test, and a course mastery list where each unit shows a bar and counts of items learned, learning, marked known and confirmed at production level — **Mastery is reported per item state, not per lesson.** Each unit breaks down into learned, learning, marked-known and — separately — confirmed at *production* level, because recognising a word and being able to produce it are different pieces of knowledge with different schedules.

Lyra's practice tab, 'Beyond the planets': timed drills called Conjugation sprint and Gender sprint presented as meteor showers, and a 'Holiday crash course' asteroid belt of standalone topics like greetings and ordering food, each with an item count — **Content that doesn't fit the course gets its own space.** Timed drills are meteor showers; standalone survival topics are asteroid belts you can drop into with no course progress at all. Both draw from the same item pool and the same scheduler, so practice outside the syllabus still moves the review queue.

Content is a build pipeline

Courses are authored as unit JSON through a validated pipeline: generated authoring briefs, machine-checked invariants (vocabulary introduced before it's used, cross-unit dedupe, per-format payload schemas), and idempotent publishing that regenerates only changed rows by content hash — a full-language publish fell from twenty minutes to under thirty seconds on shared hosting. AI-generated exercises fill demand into a shared, hash-deduped cache, validated before serving and constrained to vocabulary already taught; in-app flag reports feed a triage loop back into the source content.

AI with guardrails

Prompts are composed server-side and every provider call passes one instrumented choke point logging tokens, cost, latency, and errors. The floating tutor receives the learner's screen context as a server-resolved sentence — never client free text — closing an obvious injection surface. Speech uses two providers with defined degrade paths: Azure pronunciation assessment scores read-aloud drills, Whisper transcribes free speech for debriefs; audio is consent-gated, usage-capped, and discarded after scoring.

Two clients, one contract

The iOS app is a genuine second client, not a wrapper: ported components carry named Swift twins, and every API-contract change on the web side requires a parity-log entry before it lands. The mirror rule from the fitness platform, formalised into process.

Multi-platform product · Claude throughout

Vega — an AI fitness platform

A fitness platform built for friends and family: React web app, native SwiftUI iOS app, Android client, Apple Watch workout scheduling, and home-screen widgets — all thin fronts over one PHP API and MySQL database. Claude is integrated across roughly nineteen surfaces: a streaming coach that queries Apple Health through client-executed tool use, post-session debriefs, multi-week training plan generation, and vision-based schedule import.

4 clients · 1 backend
Swift · Kotlin · React · PHP
~19 AI surfaces
SSE streaming proxy
HealthKit + WorkoutKit
TestFlight via Xcode Cloud

Vega's week view on iOS: a coach review verdict of 'slightly behind' with a paragraph of reasoning, a weigh-in and volume summary strip, then each day's sessions with planned and actual pace side by side — **The week, on iOS.** The coach's verdict sits at the top — read against planned-vs-actual volume, not vibes — then every session for the week: runs with planned pace next to what the Watch actually recorded, gym, spin and yoga inline. Ticking a session off here is the same write the web app and the Watch make.

Vega's trends screen: a recovery ring reading 61 percent 'moderate', HRV above baseline, a target exertion band, a sleep-stage breakdown, and 30-day charts for recovery and sleep score — **Readiness, computed not guessed.** HRV, resting heart rate and sleep stages come from HealthKit; the recovery score, the strain figure and the target-exertion band are the app's own arithmetic against the athlete's own baseline. The AI is handed the result — it never estimates readiness itself.

One backend, four clients, and the mirror rule

Four clients in three languages can't share a library, but some logic has to live on every client — readiness scoring constants, run-split labelling, the debrief instructions. The answer is a documented mirror rule: every deliberately ported piece of code names its twin on the other platform, and touching one side means finding and changing the other. Convention as architecture — the cheapest consistency mechanism that survives a solo maintainer, chosen over codegen or a shared spec because the mirrored surface is small and the docs live next to the code.

A Vega home-screen widget showing today's next session — a 5K time trial with target pace — a weather-adjusted pace note, the recovery ring, a one-paragraph coach read, sleep stages and the week's running volume; a live activity timer runs in the dynamic island above — **The widget is the whole day at a glance** — next session and its target pace, the weather adjustment applied to that pace, recovery, sleep, and week-to-date volume, with a live activity running in the Dynamic Island during the session itself. It renders from a cached snapshot the app refreshes, so it stays correct without waking the network.

Planning: arithmetic in code, structure in the model

Plan generation takes a race, a date, current weekly volume and a PB, and returns a multi-week periodised block. The division of labour is the important part: every number is computed by the app, not by the model. Daniels VDOT gives the target paces for each training zone, the app detects athlete level from logged volume, and both go into the prompt verbatim. Claude is asked for structure and progression — phase ordering, session mix, how the block tapers — because that's the part where judgement helps and arithmetic errors are the part users notice.

Vega's training plan generator: goal type and sport selectors, marathon race setup with goal time, current PB, start and race dates, current weekly volume, and a panel of target paces computed by the app from Daniels VDOT with a warning that the goal pace is faster than the current threshold pace — **Plan inputs.** The pace panel is labelled with where its numbers came from — Daniels VDOT 44.6, derived from the entered PB — and the warning underneath is a deterministic check, not model output: a goal marathon pace faster than current threshold pace is flagged in code before a plan is ever generated.

One week of a generated marathon plan expanded: four runs with distances, pace ranges, weather-adjusted pace notes and per-session fuelling detail, then gym, spin and yoga blocks where three gym sessions are marked 'reduced — key cardio session tomorrow' — **One generated week, expanded.** Week 6 of 15 in the build phase: session prescriptions with pace ranges, per-session fuelling worked out in grams and timing, and forecast temperature folded into each session's target pace for equal effort. The gym, spin and yoga blocks resolve against the running plan rather than sitting beside it — three strength sessions are auto-reduced because a key cardio session lands the next day.

Closing the loop on a session

A finished session is where the platform earns its keep. The Watch supplies splits and heart-rate zones through HealthKit, the app pairs them against what was prescribed, and the debrief is generated from that comparison plus the athlete's own one-line note. The summary is then persisted as context for later features — the weekly read and the next plan revision consume the debrief, not the raw splits, which keeps the downstream prompts small and keeps the coach's memory consistent with what it previously said.

A post-run debrief: the athlete's note 'first pace session in a while but this felt good, not too hard', then a generated paragraph analysing the marathon-pace block against its heat-adjusted target, a regenerate button, the original prescription with fuelling and conditions adjustment, and a plan-versus-actual table of splits — **A marathon-pace session, debriefed.** The generated paragraph reconciles three things the athlete shouldn't have to: actual pace against the heat-adjusted target rather than the original goal, the recovery score the session was run on, and their own comment. Underneath it, the prescription as issued and the plan-vs-actual split table it's judged against. Split labelling and expansion logic is mirrored code — web and iOS have to agree on it.

Tiered model routing

Every call defaults to Haiku. Plan generation — thousands of tokens of structured output — routes to Sonnet; screenshot import uses vision-capable Sonnet; chat exposes explicit Haiku/Sonnet/Opus escalation that resets after every message, with live token and cost readouts under each reply, so spending more is always a visible choice. Routing decisions are corrected by evidence: when Haiku kept confusing similar product model names in one feature, that single surface moved up a tier — and same-input calls were deduplicated so the sampled answer stays consistent.

Vega's coach chat on iOS: the question 'it's 32 degrees, should i really still do this run' answered with advice to shift the target to effort, time it for the coolest window, hydrate, and stop on warning signs; below the reply a readout reads 'sonnet · 21 in + 658 out · $0.0099' and a Haiku/Sonnet tier selector sits above the input — **Coach chat, with the meter running.** The answer is grounded in the athlete's actual plan — it knows this session is a protected one and that Wednesday's intervals went fine — because the coach queries Apple Health and plan state through client-executed tool use. The tier selector and the per-message token and cost line under the reply are deliberate: escalating to a bigger model is a choice the user makes and can see the price of.

Centrally governed prompts

Every AI surface registers in one prompt registry: a default instruction, a per-user override, and data toggles controlling exactly which context blocks get injected — goals, schedule, training history, plan progress. The settings UI renders governance cards automatically from the registry, and a preview button shows the exact assembled prompt the model will see. Domain knowledge ships as token-efficient JSON specs — running, strength, and cycling methodology — rather than prose, so one knowledge base serves every generator.

Vega's AI prompts settings page: governance cards for the post-exercise debrief and the gym debrief, each showing a locked admin system prompt in full, a user context box with an 'improve' button, and a collapsed 'data included' toggle list — **Prompt governance, rendered from the registry.** One card per AI surface: the base instruction is admin-owned and locked, the user gets an additive context box, and *data included* expands to the exact context blocks that will be injected. Adding a feature to the registry adds its card here — there's no second place to keep in sync.

A goals editor with three reorderable plain-text goals — run a sub 3:15 marathon in October, retain muscle mass while marathon training, keep doing yoga for recovery after previous lower back pain — above the note that these are included in every AI prompt — **Goals are plain sentences, ordered by priority**, and they go into every prompt on every surface. It's the cheapest personalisation mechanism that works: no taxonomy to maintain, and the ordering is what the coach uses to decide which goal loses when two of them conflict.

The weekly read

The reports tab is where the deterministic layer and the AI layer are visibly separate. Fitness score, training load, plan compliance and consistency are all computed metrics with their own sparklines; the coach's weekly read is generated on top of them and is stamped with the model that wrote it and what it cost. Every AI block in the product carries that stamp, and a thumbs-up/down that lands in the eval platform.

Vega's reports overview: six metric tiles with sparklines — fitness score, weight, weekly volume, training load, plan compliance and eight-week consistency — above a 'coach's weekly read' panel stamped 'haiku · $0.0049' with feedback, redo and clear controls — **Computed metrics, then the model's read of them.** Six tiles, each a deterministic calculation with its own trend; the generated weekly read sits directly beneath, labelled *haiku · $0.0049*. Attribution and cost on the surface itself, not buried in a log.

Prompt caching, and its limitation

Chat resends its large system block every turn, so it's marked for ephemeral prompt caching. Per-message cost accounting made the limitation visible: the system prompt is assembled from live user data, so any change to the injected context invalidates the cached prefix, and infrequent surfaces fall outside the cache window entirely. The wins are real inside an active conversation and near zero elsewhere — the restructure, a stable cacheable core with volatile context trailing it, is queued behind measurement in the eval platform.

Developer tooling · LLM-as-judge, validated

Eval platform — closing the loop

Two products and roughly twenty AI features meant every prompt change was a guess: most outputs were one-shot streams, invisible after the fact. The eval platform closes that loop, following the Hamel Husain / Shreya Shankar methodology — capture real traces, find the failure modes by hand, then make LLM judges prove they agree with a human before trusting them to score at scale.

Next.js App Router + TypeScript
Neon Postgres + Drizzle
Vercel
One trace schema, 3 apps
Full prompt + completion capture
Cohen's κ judge validation

Capture without touching the product

Apps push traces by fire-and-forget HTTP, flushed after the user already has their response. App-generated IDs make ingestion idempotent, malformed payloads are quarantined for inspection rather than dropped, and images are replaced by content hashes before leaving the app. The shared-hosting backend can't be reached from outside, so data flows strictly one way — and when the host turned out to lack the PHP primitive for post-response work, the fallback was a bounded flush with an accepted, measured ~1% loss rather than a redesign.

The eval loop

A keyboard-driven inbox for reviewing real traces; free-text open coding; axial coding into a small failure taxonomy — AI-suggested, human-created; then one LLM judge per failure mode. Judges are versioned append-only, return structured verdicts, and are validated against human labels with a confusion matrix, true-positive and true-negative rates, and Cohen's κ, disagreements listed first. A judge that doesn't agree with the human doesn't get to score.

Operating under real constraints

Scheduled scoring batches under the serverless 60-second ceiling and resumes where it stopped; an explicit error verdict keeps failed calls from re-queueing a run forever; every run carries its own cost accounting, with model prices verified against the live price list rather than memory.

Status, plainly

Ingestion and the trace viewer are live in production, with every fitness-platform AI surface instrumented across web, iOS, and Android. Annotation, the taxonomy tooling, and the judge stack are built and green and deploying now — the first validated judge runs over production traces are the current milestone.