ReviewData

System at a Glance

What the system does, and the shape of it in one screen.

ReviewData (RD) is backend integrator infrastructure. It has no customers of its own — other platforms (called integrators, chiefly Reputation Management / RM) call its API to get four jobs done, and RD reports back over webhooks.

The whole system on one diagram

Explicitly out of scope

✗ No customer SPA · ✗ No billing (that's RM) · ✗ No review content in the DB (reviews live in S3; the DB stores URLs + metadata) · ✗ No scraper/poster implementation here (that's the Workflow Team — Page 4).

The Technology Stack — a Roadmap

Read this top-to-bottom as a layered map. Each layer sits on the one below it; learn them in this order.

Auth & security, at a glance

Tooling & the frontend

Runtime Architecture & Processes

The stack (Page 2) is what the code is made of. This is which processes run and how a request travels across all of them.

The independent processes

Lifespan singletons — one client per process

Expensive clients are created once at startup in FastAPI's lifespan context manager and stored on app.state; dependencies pull from there. Creating a new DB engine / HTTP client / Redis pool per request would waste connections and defeat pooling.

@asynccontextmanager
async def lifespan(app: FastAPI):
    app.state.db_engine = create_async_engine("postgresql+asyncpg://...")
    app.state.redis     = Redis.from_url(...)          # one shared pool
    app.state.http      = httpx.AsyncClient(timeout=10.0)  # reuse TCP conns
    app.state.temporal  = await Client.connect(...)
    yield
    await app.state.http.aclose(); await app.state.redis.aclose()
    await app.state.db_engine.dispose()

How one request travels the whole system

Steps 1–5 happen in milliseconds inside the API service. Steps 6–10 happen later, durably, in the worker fleet. The integrator bridges the gap by either polling a retrieve-* endpoint or waiting for the webhook.

Two Teams, One Database

The single most important organizational fact. Internalize the boundary before you write code.

The three shared contracts

1. Database schema. Source of truth = this repo's Alembic migrations. The Workflow Team imports the same models but never writes migrations; schema changes route through Platform review.
2. Webhook events. Event names, payload shapes, and the FailureReason enum are Platform-owned. To fire a webhook, the Workflow Team calls a shared emit_webhook activity — never their own HTTP code.
3. Workflow IDs. task_id is generated by Platform at request time; the workflow is started with ID "{type}-{task_id}", so the Admin UI can deep-link into Temporal Web and the Workflow Team can look up job context by ID.

Who writes each row

The MCS Layering

Model → Controller → Service. Strict, and non-negotiable. It is the rule you will be reviewed against most often.

Why the layering earns its keep

The canonical shape

Controller — wiring only:

@router.post("/admin/users/{user_id}/disable", response_model=UserOut,
             dependencies=[require_permission("user.manage")])
async def disable_user(user_id: uuid.UUID,
                       session: AsyncSession = Depends(get_session),
                       auth: AuthContext = Depends(get_auth_context)) -> UserOut:
    return await UserService(session).disable_user(user_id, auth.user_id)

Service — logic + the transaction:

class UserService(BaseService):
    async def disable_user(self, user_id, actor_id) -> User:
        user = await self.db.get(User, user_id)
        if user is None:
            raise NotFoundError("User not found")   # domain exception
        user.is_active = False
        user.disabled_at = now_dt(); user.disabled_by = actor_id
        await self.commit()                          # transaction lives HERE
        await self.db.refresh(user)
        return user

Domain organization (not type organization)

Controllers, services, and schemas each split into the same seven domains: core, admin, qa, integrator, posting, scraping, internal. Only models stay flat (app/models/*.py, one file per table). External I/O (HTTP, S3, Redis, Temporal) goes through dedicated client classes injected at the service layer.

Design Patterns in RD

The recurring shapes you'll see everywhere. Recognize them and new code writes itself.

1 · Service object over a session

Every service extends BaseService and is constructed inline as UserService(session) — effectively zero-cost (one attribute set), stateless. Per-service Depends() factories were deliberately rejected for now: they double indirection without earning testability. The service owns the transaction; the controller owns nothing but the HTTP call.

2 · Exception hierarchy → single envelope

All errors subclass AppError (which carries an HTTP code, a stable error_code, and a message). Services raise them; a global handler renders the one true envelope. Clients branch on the machine-readable error_code, never on prose.

class AppError(Exception): code = 500; error_code = "INTERNAL_ERROR"
class NotFoundError(AppError):  code = 404; error_code = "NOT_FOUND"
class ConflictError(AppError):  code = 409; error_code = "CONFLICT"
# domain-specific extends the base:
class DuplicateLocationError(ConflictError): error_code = "DUPLICATE_LOCATION"

3 · Pydantic contract at the boundary

Request schema in, response schema out — and they are separate types. A request schema validates untrusted input; a response schema (with ConfigDict(from_attributes=True)) hydrates from an ORM row. Integrator responses inherit a StandardEnvelope (code + status) as their first two fields; pagination uses a generic Paginated[T] wrapper.

4 · Idempotency at every async boundary

Because Temporal retries and integrators retry, every retryable operation must be safe to run twice. The patterns:

5 · Correlation via contextvars

A middleware stamps request_id (and task_id/user_id when in scope) into a contextvar; the logger reads it automatically, so every log line across the async call stack is correlated without threading an argument everywhere.

6 · Lifespan singletons + explicit DI

Clients live on app.state (Page 3); the session is injected per-request via Depends(get_session) — one session per request, committed at the service boundary, never in the controller. Permission gates are declarative dependencies: dependencies=[require_permission("...")].

The Async & Concurrency Model

The whole stack runs on one event loop per process. A single blocking call stalls every concurrent request — treat sync I/O like a memory leak.

The tools by resource

Concurrency patterns you'll use

# Fan out in parallel
results = await asyncio.gather(fetch_a(), fetch_b(), fetch_c())

# Bounded concurrency (don't stampede a dependency)
sem = asyncio.Semaphore(10)
async def bounded(x):
    async with sem:
        return await process(x)
results = await asyncio.gather(*(bounded(x) for x in items))

# Deadline
async with asyncio.timeout(5):
    result = await slow_call()

# Unavoidable CPU-bound / sync code → offload the thread
result = await asyncio.to_thread(blocking_parse, blob)

Choosing where background work runs

The Data Model

The database is the shared backbone between both teams. These are the entities and identifiers you'll meet daily.

The location hierarchy

• Location — a business at one address, owned by an integrator, keyed by the integrator's external_location_id.
• Site — one Location on one Publisher. url_status: missing → pending_finder → pending_qa → active → invalid.
• PublisherAccount — one-to-one with a Site; stores how we authenticate to post (encrypted).

Configuration & identity

Jobs & cross-cutting tables

Job tables (Platform creates the row; Workflow Team writes results): scrape_jobs, posting_jobs, url_finder, ai_response_job, competitive_report_job, insight_report_job. Cross-cutting: inbound_request (every payload archived, keyed by request_id) · webhook endpoints/subscriptions/deliveries · users/roles/permissions (RBAC) · api_keys · qa_task/qa_task_event/qa_stream_config · jwt_token.

The identifiers (know the difference cold)

Temporal Orchestration

Temporal is how RD runs work that must survive crashes, retries, and 24-hour human waits. This is the engine room.

Workflows vs activities

Conventions the Platform team enforces

• Workflow ID = {type}-{task_id} (scrape-…, post-…, urlfinder-…, webhook-{delivery_root_id}). Deterministic & human-readable → the Admin UI computes the Temporal Web URL straight from a job row.
• One input dataclass, one output dataclass — no positional args; they live in a shared package both teams consume.
• Explicit retry policy on every activity — never rely on defaults (initial interval, backoff coefficient, max attempts, non-retryable types).
• DB commit happens before the workflow starts — so a failed start never leaves an orphan workflow. Starting an existing ID → catch WorkflowAlreadyStartedError and treat as success.
• Verb-resource activity names: upload_review_batch_to_s3, post_response_via_playwright, emit_webhook — never process/run/execute.

Human-in-the-loop: signals + wait conditions

When a workflow needs a human (a URL entered, cookies captured), it waits — durably, for up to 24h — on a signal. This is how the QA board plugs into a running workflow.

@workflow.signal
async def provide_url(self, payload: UrlProvidedPayload):
    self._provided_url = payload.url

# elsewhere in the workflow body — always with a timeout:
await workflow.wait_condition(
    lambda: self._provided_url is not None,
    timeout=timedelta(hours=24),
)

How Platform triggers a workflow (from a service)

job = await self.repo.create(...)
await self.session.commit()                       # commit FIRST
await self.temporal.start_workflow(
    "ScrapeWorkflow", ScrapeWorkflowInput(scrape_job_id=job.id, ...),
    id=f"scrape-{job.id}",
    task_queue=route_to_task_queue(payload.publisher_key),   # Page 10
)

Task Queues & Worker Topology

Temporal routes work by task queue. Different work has wildly different cost profiles, so it gets different queues and different concurrency limits.

The queues

Two concepts that trip people up

Worker fleet shape (prod)

Roughly ~30 task-queue workers run inside worker processes on ECS (EC2, c6i.2xlarge, bin-packed ~3 tasks/machine). The fleet scales 1 → 12 tasks across up to 4 machines. Crucially, it does not scale on CPU/memory (HTTP-light scraping sits at ~13% CPU) — it scales on queue backlog, which is the whole subject of the next page.

The Queue Engine: Dispatch, Fairness & Autoscaling

RD's most sophisticated Platform-owned subsystem. It decides how many workflows run, which jobs get a slot, and how big the worker fleet grows. Learn it — it's the interview-grade part of the codebase.

The problem it solves

Scrape submits are cheap and instant, but scrapes are slow and resource-bounded. If every submit started a live Temporal workflow immediately, a burst of thousands would flood one event loop, starve the pollers, and freeze the fleet (this actually happened in prod). So jobs land in Postgres as BUFFERED rows, and a single background queue manager meters them into Temporal.

The control loop

The five moving parts

1. Dynamic gate. The cap on concurrently-running workflows is recomputed each cycle as PER_TASK_HTTP_CONCURRENCY (50) × running_worker_count, plus a global browser cap of 10. So 1 worker → gate 50; 12 workers → gate 600 — each task always sees a safe per-task load regardless of fleet size. Worker count comes from Temporal's WorkerHeartbeat registry (presence-based, so a saturated worker is still counted — an activity-poller count under-reported the busy fleet by ~45%).
2. Backlog metric. ECS scales on OutstandingScrapeJobs = BUFFERED + PENDING_SCRAPE + QUEUED + IN_PROGRESS — all non-terminal work, not just waiting depth (scaling in while heavy workflows were still running once stranded ~150 of them).
3. Fair dispatch. Chairs are handed out round-robin across publishers that have buffered work — one ROW_NUMBER() OVER (PARTITION BY publisher_key) query — so a 1,000-job flood of one publisher can't head-of-line-block a single job of another.
4. Holding ceiling. Under contention (≥2 active publishers), no single publisher may occupy more than ~60% of the gate — reserving headroom for a newcomer. A lone publisher gets 100% (no capacity wasted).
5. Integrator priority tiers. Jobs carry a denormalized scrape_priority (lower = higher; production = 100, test/staging raised to 1000). Dispatch is priority-first: the whole production tier drains before any test-tier job gets a slot — but test load still counts toward autoscaling, so it's a first-class citizen for capacity, it just yields dispatch order.

The same idea, applied to LLM reports

Insight (C32) and Competitive (C34) reports reuse the exact pattern: a BUFFERED admission state + a ReportQueueManagerService, gating at the process-start boundary (not submit) — so even the "600 scrapes finish at once → 600 resume hooks fire" thundering herd gets metered. Two independent lanes (insight / competitive, ~40 concurrent each, since they run on separate worker fleets), with a shared OpenAI backpressure belt that honors 429/Retry-After above both.

Request & Webhook Flows

The external contract: how integrators submit work and how RD reports back.

The inbound REST surface

API key in the payload; every response wraps the standard envelope. Async endpoints return 202 Accepted + a task_id.

A scrape, end to end

1. RM calls POST /request-reviews; Platform authenticates the key and archives the payload in inbound_request (request_id).
2. A scrape_jobs row is created (BUFFERED) with a fresh task_id; the endpoint returns 202 + task_id.
3. The queue manager (Page 11) meters the job into a Temporal workflow scrape-<task_id> on the right queue.
4. The Workflow Team's scraper runs — internal (SAU) or external (LDE) per PublisherConfiguration — and writes reviews to S3 as .jl files (DB stores only CloudFront URLs + metadata).
5. On completion the workflow updates the row and fires EVENT.DATA_RESULT via the shared emit_webhook activity.
6. RM receives the webhook (or polls /retrieve-task).

Outbound webhook events

Delivery is itself a durable workflow

Webhook delivery isn't a fire-and-forget HTTP call — it's a Temporal workflow (Platform-owned, on the webhook-delivery queue) that loops an 8-attempt retry schedule (workflow.sleep() between attempts) over activities http_post_webhook → record_attempt → mark_dead_letter. After 8 failures it's dead-lettered and surfaced in the Admin UI for manual replay. Each attempt carries a fresh delivery_id; the body is HMAC-signed.

Posting, QA & Error Handling

The human-in-the-loop system, and how failures are shaped into a clean contract.

Posting: mode × auth type

The QA Task Board — five streams

1. URL — QA enters a publisher URL for a site missing one.
2. Responses — QA manually posts a reply (MANUAL_QA / escalations).
3. Poster Waiting — waiting on the client to accept a manager invite.
4. Poster Credentials — QA enters publisher login credentials.
5. Poster Cookies — QA captures cookies via the Chrome extension.

Tasks have lifecycle events (qa_task_event), are claimed/released by operators, scoped per user via stream configs, governed by qa.task.* permissions. An SLA sweeper flags stale tasks; reconcilers keep the board consistent with the underlying jobs. Completing a QA action signals the waiting Temporal workflow (Page 9) — that's the bridge between the board and the engine.

Error handling: one envelope, always

Services raise typed AppError subclasses; a global handler renders exactly this — raw FastAPI/Pydantic/SQLAlchemy errors never reach a client, and stack traces go to logs, not responses.

{
  "code": 409,
  "status": "failed",
  "error": "A location with this external_location_id already exists.",
  "details": { "external_location_id": ["already exists"] },
  "meta": {
    "request_id": "9f9d2b5a-...",
    "timestamp": "2026-04-28T14:23:11Z",
    "error_code": "DUPLICATE_LOCATION"
  }
}

• 4xx = terminal (don't accept duplicate task creation); 5xx = transient (may have partially succeeded — dedupe on the integrator's foreign_key).
• Log level tracks severity: 4xx → warning; 5xx → exception (with traceback). Expected validation failures are never logged at error level.
• error_code is SCREAMING_SNAKE_CASE, stable across versions, kept in a constants module so it's discoverable — clients branch on it.

Observability

Structured JSON logs (loguru) carry request_id/task_id/user_id on every line via contextvars; logs ship to S3 (daily partitions) + Datadog. Sentry catches exceptions — note a bare logger.error() does not reach Sentry; you must capture_exception(...). Prometheus exposes metrics.

Foundations to Master First

A week-by-week path. Each item says why it matters here and what "done" looks like.

Week 1 — Language & async fundamentals

1. Python async/await & the event loop (Page 7). Coroutines, gather, Semaphore, timeout, why blocking calls are banned.
   Done when: you can explain why one sync DB call stalls every concurrent request, and write a bounded-concurrency fan-out.
2. Modern typing. list[X], X | None, Annotated; from __future__ import annotations. RD runs mypy strict.
   Done when: you can read a fully-typed service method and know every shape without running it.
3. HTTP & REST. Methods, status codes (esp. 202 for async), idempotency, API-key vs bearer auth.
   Done when: you can say why /submit-response returns a task_id instead of the final result.

Week 2 — The frameworks

4. FastAPI — routers, Depends, response_model, declarative permission gates. Skim the live /docs.
   Done when: you can trace a request URL → controller → service → response and name each layer's job. Ref: backend/03-fastapi-conventions.md.
5. Pydantic v2 — BaseModel, Field, validators, BaseSettings, from_attributes. Understand why request and response schemas are separate types.
   Done when: you can write both and explain why models never cross HTTP. Ref: backend/04-pydantic-schemas.md.
6. SQLAlchemy 2.0 async + Alembic — Mapped, AsyncSession, select()/scalars(), relationships, migrations.
   Done when: you can read a model file, describe its table + FKs, and explain alembic upgrade head. Ref: backend/05-sqlalchemy-and-alembic.md; skills: alembic-migrations, run-migration.

Week 3 — What makes RD special

7. Temporal (Pages 9–10) — workflows vs activities, durability, retries, task queues, signals/wait-conditions, the {type}-{task_id} convention.
   Done when: you can explain what happens if a worker crashes mid-scrape and why the job survives. Ref: backend/06-temporal-conventions.md; skill: temporal-workflows.
8. The MCS layering + design patterns (Pages 5–6) — re-read 01-architecture/01-mcs-architecture.md until you spot a violation instantly.
   Done when: given a controller with a DB query in it, you name the broken rule and where the code moves.
9. The queue engine (Page 11) — read 01-architecture/06-worker-queue-driven-autoscaling.md. It's dense but it's the crown jewel.
   Done when: you can explain the dynamic gate, why it scales on backlog not CPU, and what fair dispatch prevents.
10. Errors, logging, testing (Page 13) — the envelope, the exception hierarchy, correlation IDs, why logger.error misses Sentry.
    Refs: backend/07-error-handling.md, 08-logging.md, 09-testing.md.

System Understanding & First Tasks

Being able to narrate these flows out loud is the real test that you understand RD.

Trace these end-to-end (on paper, then in the code)

1. A scrape — /request-reviews → BUFFERED → queue manager → workflow → S3 → EVENT.DATA_RESULT. Name where the job row is created, who owns each status, and how RM finds out.
2. A posting request — /submit-response for an AUTOMATED publisher (poster workflow) vs a MANUAL_QA one (spawns a QA task, workflow waits on a signal).
3. A cookie capture — QA logs in → extension captures cookies → internal endpoint stores them encrypted → EVENT.COOKIE_UPDATED → poster replays them.
4. A webhook retry — walk the 8-attempt schedule to dead-letter and the admin replay.
5. A burst — 2,000 scrapes submitted at once: why they don't flood Temporal, how the gate + autoscaling drain them, how fairness protects a small integrator.

Component specs to read, in order

• 02-components/01-rbac-and-users.md · 02-api-keys.md — auth.
• 04-locations-sites-accounts.md — the core hierarchy.
• 06-…request-reviews · 09-…submit-response — the two key async endpoints.
• 13-webhooks.md + 14-failure-reasons-enum.md — the outbound side.
• 16-manual-tasks-qa-board.md · 23-posting-workflows.md · 27-cookie-health.md — QA + posting internals.

Hands-on ramp (clear each with your mentor)

1. Green baseline. Bring up docker-compose; hit /docs; alembic upgrade head; run pytest.
2. Read one vertical slice. Pick a small admin endpoint; read controller → service → model → schema → test.
3. Write a test. Add one for an existing endpoint with the AsyncClient fixture + a factory_boy factory.
4. Ship a tiny spec-backed change. e.g. add a read-only field to a response schema (with the doc's blessing) + migration + test + changelog line — the full PR + doc-sync loop.
5. Shadow a QA flow. Watch a real task move through a stream in the Admin UI.

Habits that make you effective here

Where to look when stuck