SATO48

International-Judge

6 min read

International Judge

The International Judge process is a ranked-choice scoring system where invited judges each rank a curated set of films within a specific award category. Their rankings are aggregated to determine final standings per category.

This is distinct from the Selection Committee process (which uses 1–10 metrics per film per award) and the Academy process (which has its own page at /admin/results/academy).

Feature Flag

The judge-facing pages (/judge/*) are gated behind the international_judging OpenFeature flag, checked via middleware on every judge page. The admin results page (/admin/results/judge) uses role middleware and is always accessible to admins regardless of the flag.

Data Model

Key Tables

TablePurpose
film_groupsGroups of type "judge" — one group per award category, each containing the nominated films for that category
film_group_usersAssignment of a judge to a film group; also stores that judge’s rankings (jsonb) and submitted_at timestamp
awardsEach judge film group is linked to exactly one award via film_groups.award_id

film_group_users.rankings

Rankings are stored as a JSONB object keyed by team_id (as a string), with the value being the judge’s rank position (integer, 1 = best):

{
  "42": 1,
  "17": 2,
  "9":  3,
  ...
}

This is written on every drag interaction (auto-saved), and submitted_at is set separately when the judge explicitly marks their ballot complete.

Key Constraints

  • A judge is assigned to a film group via film_group_users — one row per (judge × film group)
  • The judge store always fetches users(limit: 1) — each judge only ever sees their own assignment row
  • Rankings are saved in-place on the same film_group_users row via update_film_group_users_by_pk

User Role

Judges are users with the role "[year] International Judge" (e.g. "2026 International Judge") stored in users.meta.roles. This role is used in the admin assignment UI to filter the eligible judge pool.

The Pinia Store (stores/judge.ts)

A single Pinia store powers all judge pages. It fetches eagerly on initialization (no lazy loading).

Data Fetched

JudgeFilmGroups query — fetches all film groups for the current festival (no type filter). This means the store loads every film group type, but the judge pages only render those relevant to the logged-in judge (via their film_group_users assignments).

For each group:

  • Group name and linked award (id, name, description)
  • users(limit: 1) — the current judge’s assignment row, including their rankings jsonb and submitted_at
  • All teams in the group (ordered by team number), each with film metadata, poster, Mux playback ID, studio, and award nominations

Mutations

MutationTriggerEffect
JudgeSaveRankingsEvery drag @end eventUpdates film_group_users.rankings with current order
JudgeSaveSubmittedAtJudge clicks the complete/incomplete toggleSets or clears submitted_at on the assignment row

Both mutations trigger a refetch() on completion.

Judge-Facing Pages

/judge — Group Dashboard

The landing page for judges. Gated by international_judging feature flag.

Shows one card per assigned film group. Each card displays:

  • The group/award name
  • A “Ranking Complete” (green) or “Ranking Incomplete” (warning) chip based on whether submitted_at is set
  • A horizontal scrollable poster strip of all films in the group, sorted by the judge’s current rankings (unranked films appear in random order)
  • Clicking a poster navigates to /judge/[filmGroupId]?tid=[film_group_team_id]

/judge/[filmGroupId] — Ranking Interface

The main judging page for a specific award group.

Top strip — draggable poster ranking:

  • All films in the group displayed as poster cards in a horizontal vuedraggable row
  • Judges drag posters left/right to re-order their ranking
  • Position 1 (leftmost) gets a gold badge; others get a numbered grey badge
  • Rankings are auto-saved on every drag end via JudgeSaveRankings
  • Dragging is disabled once submitted_at is set (ballot locked)

Complete/Incomplete toggle:

  • Upper right button — toggles submitted_at between the current timestamp and null
  • Acts as a two-way lock: clicking “Ranking Complete” locks the ballot; the tooltip reveals “Mark Incomplete” to undo

Left column (film detail):

  • Mux video embed for the currently selected film (selected via ?tid= query param or clicking a poster)
  • Film poster (click to expand), film name, genre, team number, studio
  • Nominee avatar thumbnails with tooltips (award name + nominee name), excluding “Song or Music”
  • Film synopsis

Right column (instructions):

  • Static text: “You have been assigned the Top Ten films in the [award name] category. Rank the films from best to worst using the following criteria:”
  • Followed by the award’s description field

Admin Assignment (/admin/film-groups/users)

Admins assign judges to film groups from this page.

  • The left nav lists all film groups grouped by type (excludes screening and fivefest types)
  • Selecting a group of type "judge" shows:
    • Existing assignments with the AdminFilmGroupJudgeAssignment component — displays a “Ranking Complete/Incomplete” chip and a relative timestamp of the last update
    • An autocomplete for adding a new judge, populated from users with the "[year] International Judge" role
    • Each user option in the autocomplete shows chips for any other groups that judge is already assigned to — useful for ensuring balanced workloads

Admin Results Page (/admin/results/judge)

Admin-only (role middleware). Aggregates all judge rankings into a final score per film per award category.

Score Aggregation

For each film group (award category):

  1. Each judge’s rankings jsonb is flattened into individual { team_id, award_id, user_id, rank } rows
  2. Per (team × award), ranks are averaged across all judges who ranked that film
  3. Final value = (11 - mean_rank) * 10 — rank 1 = 100 points, rank 10 = 10 points

This is the same formula used in the Selection Committee results page.

Results Table

  • Categories combobox — select which award columns to display
  • Filter field — text search across film names
  • Each cell shows a colored progress bar (green → yellow → red) with the mean rank to 2 decimal places
  • Films with no judge rankings for a category show no bar

Key Difference from Selection Committee Results

The Selection Committee results page requires ballot validation (checking that a SC member submitted enough metrics to count as a valid ballot). The Judge results page has no ballot validation step — all submitted rankings count. The submitted_at flag is informational only and does not gate inclusion in the results.

Round 2 Judging

In SATO48-2026, a second round of judging was run to address the impact of the Selection Committee Bug. Eleven additional film groups of type "judge" were created — one per award category, named "Round 2 - [Award Name]" — and populated with the top films from the corrected SC results. These groups follow the same judge process as Round 1.

Graph View

Backlinks