Skip to content

Assist service

Purpose

backend/internal/platform/assist is the semantic layer behind every model-assisted feature, built on the platform/llm transport. It owns the task registry, the engine that runs a task, strict output validation, caching and the stored results. It imports no domain. See Assist layer for the system view.

Layout

Path Holds
assist/ Task, Spec, Handler, the Engine, schema validation, confidence, the ai_reviews store, the eval runner and the fake model
assist/tasks/app/ The member app's tasks, one file each, with their eval fixtures
assist/tasks/backstage/ Backstage's tasks, one file each, with their eval fixtures
assist/tasks/registry.go The single registry, listing the two groups separately
assist/assistdb/ sqlc queries for ai_reviews

Tasks

Task Audience Run from Model What it returns
location_resolve app The link-capture enricher (platform/places) small A clean search query, venue and kind for a captured post
location_select app The link-capture enricher small The index of the right place candidate, or none
report_triage backstage Job queued when a report is filed review Outcome (dismiss, action, review), kind, policy category, rationale, confidence and indicators
photo_review backstage Job queued when a cover photo is submitted review Accept, reject or review, with relevance, people, quality, duplicate, rationale and indicators
translate backstage Job queued when an announcement is saved review Title and message in each locale staff have not written

Every task is run at temperature 0. translate works in batches of five languages so a long message stays inside the output cap.

Gaps

  • There is no place review task. No member-created place or place change reaches a review queue today, so a task would have no trigger. Add one with the review path.
  • search_intent does not exist in the code, so it has no task.
  • photo_review cannot see pixels. It works from the place, the caption of the source moment, the platform checks and the member's earlier rejection count.

Running a task

A domain holds a Runner (one Do method) scoped to its audience and calls assist.Run(ctx, runner, task, subject, input). The engine then:

  1. Refuses when the assist switch is off (ErrOff) or the task belongs to the other audience.
  2. Returns the stored result when this subject already has one for the same input hash and prompt version, or the memoised one for a call with no subject.
  3. Applies the task's Pre rules. A rule can decide the result or withhold the item without a model call.
  4. Calls the model with the task's prompt, validates the JSON against the closed schema, and asks once more if it is invalid. A second failure stores no_suggestion.
  5. Applies the task's Constrain rules, which can override the model, then derives the confidence bucket.
  6. Stores the record on its subject (ai_reviews) or in the memo.

Confidence starts from the model's own number (0.8 and up is high, 0.5 and up is medium). A conflict with the rule signals lowers it one bucket, so does needing the retry, and a rule that overrode the model caps it at medium.

Results on items

ai_reviews keeps one row per run, keyed by subject_type, subject_id and task, newest first. Backstage reads them back as ai_review on the report group and cover submission detail, as ai_reviews by id on the report and cover queue pages and as suggested_texts on each announcement. A translation batch is stored under its own subject, derived from the announcement id, so batches never replace each other.

Triggers

Event Hook Job
A report is filed on a group (new report only) report.ReviewTrigger assist:review kind report_group
A cover photo is submitted geocover.ReviewTrigger kind geo_cover
An announcement is created or edited backstage.Service kind announcement

The two domain ports are satisfied by backstage.ReviewTriggers, which only enqueues and logs a failure. assist:review runs on the low queue, starts 20 seconds after the change that queued it so a burst of reports runs once after the last of them, reads the item as it is when it executes, and skips retry when assist is off, has no model or is out of budget. When an item's inputs change the next run sees a new input hash and calls the model again.

Configuration

Setting Meaning
assist.base_url, assist.model, assist.timeout, assist.allow_private, ASSIST_API_KEY The review model. Empty base URL or model leaves Backstage tasks without a model
assist.daily_token_budget, ASSIST_DAILY_TOKEN_BUDGET Review-model tokens per UTC day across all tasks, default 5,000,000
semantic.* The small model for the place resolution tasks, as before
semantic.daily_token_budget, SEMANTIC_DAILY_TOKEN_BUDGET Small-model tokens per UTC day, counted apart from the review model's budget so place resolution cannot starve staff review. Default 2,000,000
assist feature switch Turns every task off at once, from Backstage with a reason, audited

Metrics carry the audience. assist_runs_total counts runs by audience, task and outcome (model, cache, rule, invalid, off, budget, error), and the llm_* series label each call <audience>.<task>.

Retention

ai_reviews keeps results for 90 days and the newest three of each task on an item. A daily job (cron:assist_review_prune) applies both. Database triggers delete an item's results when the item is deleted and when the member it is about is erased, because the table has no foreign key. See Privacy.

Evals

Each task has a fixture of labelled cases next to its group (assist/tasks/<audience>/fixtures/). A case has an input, the labels the final output must carry after the rules ran, and a canned reply for the fake model. Cases flagged exemplar are baked into the prompt as few-shot examples.

  • In tests. go test ./internal/platform/assist/... runs every case through the real engine against the fake model. It covers parsing, closed vocabularies, rule overrides, confidence, caching and determinism. report_triage has more than forty cases including benign content, sarcasm and other languages, photo_review more than twenty, and translate checks placeholder and length preservation. task assist:eval runs the same check from the command line.
  • Against the real model. task assist:eval:real sends every case to the model named by ASSIST_EVAL_BASE_URL, ASSIST_EVAL_MODEL and ASSIST_EVAL_API_KEY and prints which labels differ. It asks for confirmation, costs one call per case and never runs in CI.

Adding a task

  1. Pick the audience. app for what a member's own use case calls. backstage for what staff review or translate. The audience sets the folder (assist/tasks/app/ or assist/tasks/backstage/), the runner a domain needs and who may trigger it.
  2. Write the input type and the closed schema. One file per task. Outputs are enums, closed lists, numbers from 0 to 1 and one short rationale. Anything else is rejected.
  3. Put rules first. Write Pre for what a rule can decide or withhold (child safety, banned terms, exact duplicates), Constrain for what the model may never do, and Agree for how rule signals corroborate or conflict. Render masks every member-written string with llm.Untrusted.
  4. Write the prompt and bump its version. Exemplars come from fixture cases flagged exemplar. The prompt lock test hashes the prompt, the fields and the behavior of every rule over the fixture cases, and fails until the version and hash are updated together.
  5. Add the fixture. Cases with labels and fake replies, including benign, ambiguous and rule-override cases. Backstage review tasks need enough cases to be meaningful (dozens, not a handful).
  6. Pick the model kind and budget class. small or review, and tiny, small, medium or large for the output cap.
  7. Register it in the group's All().
  8. Add the trigger. For an app task, the domain calls it behind its own port in deps.go. For a Backstage task, the domain gets a ReviewTrigger port, Backstage satisfies it by enqueuing assist:review, and the job body lives in backstage/reviews.go. Never add an endpoint that runs a task.
  9. Show it with the item. Return the stored result as part of the item's response.

Invariants

  • No endpoint runs a task. A test asserts no Backstage route mentions assist, suggest, draft or translate.
  • A Backstage task is imported only by Backstage code, an app task never by Backstage code, and the assist package imports no service or model. Tests walk the imports.
  • A child safety report reaches neither the model nor the input hash.
  • Staff text from a translation is never published by assist. Saved translations come from the editor.