Design challenge · Physical AI

36 autonomous assets.One operator.No surprises after launch.

A search-and-rescue tasking console where the autonomy proposes, the operator decides, and every coverage gap is visible up front.

Screen 3, Tasking live: a mission console headed SAR-1102, Coastal Flood Response, at Charleston, South Carolina. A map of the flooded coastline is divided into four sectors — A East Cooper at 42%, B Peninsula at 18%, C Sea Islands at 36% and D Edisto at 33% — with individual asset markers across it, a few collapsed into groups such as HYD ×4, and one tagged RETURNING · BATTERY. The header reads Live 42 min, Behind plan Sector B −18 pts, and In contact 36 of 36. A Sector Status rail down the right states its own key — bar: searched so far, tick: where the plan expects to be now — then gives each sector a priority chip, a coverage bar with that tick, and its tasking rows. Sectors A, C and D read On plan; Sector B carries a warning reading 18 pts behind plan (36% expected by now), HYD-09 and HYD-11 returned for battery at 09:31, 5 of 7 Hydras still searching rooftops, offering Re-propose Sector B or Keep current plan. An event feed logs timestamped changes from 09:12 to 09:31, a legend names the six asset states and adds that a ring means needs attention and dimming means the position is not current, and a Hold all units button sits at the foot of the rail.

Available assets to task in one plan (2 Rocs in maintenance): 36Available assets to task in one plan (2 Rocs in maintenance)
Asset types with different range, endurance and sensors: 4Asset types with different range, endurance and sensors
km² of flooded coastal search area: 1,200km² of flooded coastal search area

Context

This was a design challenge set by an autonomy company.

The brief described a fictional disaster: a Category 4 hurricane made landfall 14 hours earlier near Charleston, South Carolina. Cell towers are down, roads are flooded, and people are stranded on rooftops across a 1,200 km² coastal region.

A two-person team runs the search. The mission operator makes every tasking decision; the supervisor watches the same view and can veto before launch. They work on a large desktop display, often in a dim room, under time pressure.

The fleet is 38 autonomous assets of four types, and they are not interchangeable:

Asset	Count	Strength	Limit
Roc (fixed-wing)	6	Covers large sectors fast	Low detail per pass
Hydra (quadrotor)	24	Close inspection	15 km range from a forward staging site
Griffin (VTOL)	4	Radar and comms relay	Few available
Kraken (surface drone)	4	Goes where crewed boats can’t	Slow, about 12 knots

Two Rocs are in maintenance, so 36 are available. Today the operator assigns them one at a time. That takes about 20 minutes and leaves coverage gaps nobody sees until the debrief.

My role: sole designer, responsible for the screens, edge-case states, component library, engineering annotations and the spec for one coded interaction. I worked with Claude through the Figma MCP for file production.

The core problem: the operator has to commit 36 different assets to four sectors in minutes, and trust that the plan actually covers the area before anything moves.

Autonomy can propose the plan, but the operator has to trust it before anything launches.

Objective

The autonomy proposes a full tasking plan, and the operator can verify it, change it and launch it in under five minutes, with nothing launching unseen.

Coverage at a glance: which sectors are covered, by what, and how well.
The weakest sector is obvious: the operator can name it without hunting.
Every autonomy choice is explained in one readable line.
The operator stays in control: accept the plan whole, change any assignment, or lock assets so a re-propose won’t move them.
Warnings inform, never block: a HIGH-priority sector below 60% coverage gets an in-place warning, and launch stays available.
Out of scope: live flight control, route planning (the autonomy owns it), and anything involving weapons or targeting.

Approach

1. A plan can’t be trusted if its own numbers don’t add up

The first draft of the review screen had the right structure, but the data broke it. The counts didn’t reconcile with the fleet. An operator who catches one wrong count stops trusting every other number on the screen.

I rebuilt the roster as a single source: 38 units, each with a real ID, type, sector and status.
ROC-05 and ROC-06 stay on screen as unavailable, with the reason and when they’re expected back.
“Zones” became “sectors” to match the brief, lettered A to D.

The asset roster beside the map. The header reads Available: 36 of 38. The Asset Roster rail, with columns for Unit, Status · site and Endurance · link, opens on the Roc · fixed-wing group, 4 available · 2 unavailable: ROC-01 to ROC-04 each read Ready · Airfield, 18 h · strong, and ROC-05 and ROC-06 read Maintenance, Unavailable, with ROC-06 dimmed. On the map a tooltip reads ROC-05 · Unavailable, In maintenance: sensor recalibration, Expected back 22:00. The Hydra · quadrotor group, 24 available, begins below.

2. Plan in groups, fly as individuals

On the review screen the operator is judging a plan, so the map groups assets per sector (HYD ×7). A group of one shows its ID instead of “×1,” which matches the rail and reads as intentional. Selecting a group fans it out into its units, which is where a single Hydra gets dragged to another sector. Individual positions appear only once assets are flying and position matters.

The same logic keeps the roster scannable before a plan exists: 24 Hydras with identical status, site and link collapse into one row, so 38 assets fit in a single view.

The map and roster before a plan exists. On the map, 24 Hydras sit as one marker at Staging A labelled HYD ×24, four Krakens as one marker at the Marina labelled KRK ×4, and two clusters of calls for help read 3 calls and 2 calls. In the rail, the Hydra · quadrotor group, 24 available, collapses into a single row — 24 units with the same status, site and link: HYD-01–24, Ready · Staging A, 45 min · strong — while the Griffin · VTOL relay group, 4 available, lists GRF-01 to GRF-04 one per row, each Ready · Airfield, 6 h · strong.

3. Coverage owns the fill, priority owns the chip

The brief asked how to show sector priority without competing with coverage color. The answer was to give each signal one channel.

Coverage fills the sector on the map and drives a thin bar on each card, with a tick at the 60% target.
Priority is a HIGH, MEDIUM or LOW chip beside the sector name, never a color.
Selection is a white outline, so fill always means coverage.
Color is earned: amber appears only where the operator should decide, and blue only on things the operator can act on.

Screen 2, Review proposed tasking. A stat bar reads Overall coverage 86%, Weakest Sector D · Edisto · 82% marked HIGH, and Unassigned 0 of 36. On the map the sector fill sits behind the label Sector A, East Cooper · 92%, with a ROC · 01 marker and a group marked HYD ×6. The Proposed Tasking rail, with Clear plan and Re-propose actions, reads Expected coverage, next 2 h · tick marks the 60% target. It lists GRF-01–04 as a comms relay over every sector, then two staged sector cards, each with a priority chip, a lock, a green coverage bar past its tick, the autonomy’s one-line reason and its tasking rows: Sector A · East Cooper, HIGH, at 92%, with HYD-01–06 on rooftops: densest areas, most help calls; and Sector B · Peninsula, MEDIUM, at 83%, with KRK-01–02 added: water too deep for ground crews.

4. Warn with the cost, never block

The first Sector D warning named the cause (Hydras at max range from staging) but not what it cost. The rewrite puts the cost first, in the operator’s terms:

◆ 44% coverage, 16 pts below target for a HIGH sector
Hydras at max range from Staging A: little time on station.
See options · Stage anyway

The mission header above the Sector D card. The header reads SAR-1102 · Coastal Flood Response, Charleston, SC, with Overall coverage 80%, Weakest Sector D · Edisto · 44% marked HIGH, and Unassigned 0 of 36. The Sector D · Edisto card, marked HIGH with a Stage sector button and a lock, shows an amber bar at 44% stopping short of the target tick, reading 44% · target 60%. Its warning reads 44% coverage, 16 pts below target for a HIGH sector, then Hydras at max range from Staging A: little time on station, with two actions, See options and Stage anyway. Below it, HYD-21–24 search rooftops until battery 50%, locked, and KRK-03–04 search streets with sonar on, marked by a teal bar reading Set by operator · moved from Sector B · 08:41.

“Stage anyway” makes the rule visible: the warning informs, and the operator can still stage and launch. A stat bar above the map names the weakest sector, so the operator never has to hunt for it.

5. Mark the exception, not the default

Most of the plan comes from the autonomy, so autonomy rows carry no marker, only their one-line reason (“KRK-01–02 added: water too deep for ground crews”). Operator changes are the exception, so they get the marker:

A teal left bar and a source line: “Set by operator · moved from Sector B · 08:41.”
Purple stays reserved for the Autonomous status, so the two never blur.
The status badge in each row became a lock control, since locking is the action the operator takes there. Locks and priority chips align in fixed columns, so the rail scans in one pass.

The Sector D · Edisto card, marked HIGH, with a Stage sector button and a lock, and a green coverage bar at 82% past its target tick. Under it the autonomy’s reason reads HYD-21–24 on rooftops: neighborhoods cut off by flooding. Two autonomy rows carry no marker: ROC-04, survey flood area at Alt 8,000 ft, and HYD-21–24, search rooftops until battery 50%, locked. The third row, KRK-03–04, search streets with sonar on, has a teal left bar and the line Set by operator · moved from Sector B · 08:41.

6. In this domain, the edge cases are the job

I designed five states, centered on one failure caught before launch and one handled during the mission.

Before launch: a sector the fleet can only partly reach. As soon as the operator draws Sector D, the console flags that Hydras can only partly reach it: most of the sector sits beyond their 15 km limit from Staging A. Selecting the sector shows exactly what can reach it — Roc from the airfield, Kraken from the marina, Griffin relay coverage, and Hydra only along the NE edge. The gap is visible before a plan is even proposed.

Screen 1, Area overview, with no coverage fill yet. The map shows the four sectors and their priority chips: Sector A East Cooper HIGH, Sector B Peninsula MEDIUM, Sector C Sea Islands MEDIUM and Sector D Edisto HIGH. Hydras wait as one marker at Staging A, HYD ×24, and Krakens as one at the Marina, KRK ×4; clusters of calls for help read 3 calls, 2 calls and 6 calls. Sector D is selected with a white outline, and its card reads Sector D · Edisto, HIGH priority · 6 calls for help. Under Can reach it lists Roc, from Airfield · 18 h; Kraken, from Marina · 30 h; and Griffin, relay covers sector, each with a green dot. Under Partial it lists Hydra, NE edge only · 15 km limit, with an amber dot.

During the mission: a sector falls behind plan. Two Hydras return for battery, and Sector B drops 18 points behind where the plan expected it to be. The card shows planned versus actual coverage, explains why in one line, and offers a choice: Re-propose Sector B or Keep current plan. The autonomy proposes; the operator decides.

Screen 3, Tasking live. The header reads SAR-1102 · Coastal Flood Response, Live 42 min, Behind plan: Sector B · −18 pts in amber, and In contact 36 of 36. On the map, Sector B Peninsula is labelled 18% in amber. A dashed line runs from the sector to a pair of Hydras marked HYD ×2 and tagged RETURNING · BATTERY, and the markers ROC · 02, HYD ×5 and KRK ×2 each carry a purple Autonomous status square. The Sector B · Peninsula card, marked MEDIUM and Behind, shows an amber bar at 18% short of its tick and a warning: 18 pts behind plan (36% expected by now). HYD-09 and HYD-11 returned for battery at 09:31. 5 of 7 Hydras still searching rooftops. It offers Re-propose Sector B or Keep current plan. The rows list ROC-02 surveying the flood area at Alt 8,000 ft, HYD-07–13 searching rooftops with 5 of 7 searching in amber, and KRK-01–02 searching streets with sonar on.

Also built: the empty state (no sectors drawn, so nothing to propose), a HIGH sector below its coverage target, and unavailable assets with the reason and expected return.

What I’d build next: a lost relay link, shown as one incident card with the root cause first; a supervisor veto; and a re-propose that conflicts with a locked asset.

Engineering Handoff

The review screen was the key decision, so its parts became the system: a sector card with 8 variants (staged, selected, low coverage), a tasking row with 4, and priority chip, lock, status and asset-marker components, all bound to 21 color tokens and reused across all three screens.

Each color token carries a USE WHEN / AVOID rule, written so engineers and AI agents apply color the same way the operator reads it.

The brief asked for one coded interaction: drag an asset between sectors, with coverage updating live. I wrote the handoff so the build could not drift from the design:

One data file: all 38 units and a simple coverage rule, tuned so the default plan reproduces the Figma screens exactly (A 92%, B 83%, C 87%, D 82%, 86% overall).
Tokens as a table: the 21 Figma color tokens mapped to code, with the visual rules (selection is white, never blue; amber only where the operator decides).
Two kinds of annotation: note cards beside each frame explain the design reasoning; native Figma annotations sit on the exact layers engineering will build.
A one-drag acceptance test: moving ROC-04 from Sector D to Sector B should drop D to 44% (B rises to 97%, overall falls from 86% to 80%), show the inline warning, name D as weakest in the stat bar, and mark ROC-04 as set by operator.

The handoff frame: four component panels laid over a dimmed Screen 2, whose header reads Overall coverage 86%, Weakest Sector D · Edisto · 82% HIGH, and Unassigned 0 of 36. Color tokens · rules for operators and AI agents lists four tokens, each with a USE WHEN and an AVOID rule: --status-needs-operator #FFB020, --status-operator-set #2EC4B5, --status-autonomous #CDB8FF and --interaction-primary #8EC5FF. Asset marker shows the Roc, Hydra, Griffin and Kraken marks. Tasking row shows four variants of ROC-01, survey flood area at Alt 8,000 ft: plain; locked; set by operator, with a teal bar and the line moved from Sector B · 08:41; and set by operator and locked. Sector card shows two variants: Sector B · Peninsula, MEDIUM and staged, with a green bar at 83% and the reason KRK-01–02 added: water too deep for ground crews; and Sector D · Edisto, HIGH, with a Stage sector button, an amber bar at 44% and the warning 44% coverage, 16 pts below target for a HIGH sector. — The tokens carry the rule, not just the value, so the next person building a screen knows when amber is allowed.

Outcomes & Impact

Three screens at desktop size, dark theme: Area overview, Review proposed tasking, Tasking live.
Five edge-case states built, from the empty state through a sector falling behind plan.
A component library of 8 component sets bound to 21 color tokens.
A launch summary that lists every unit before launch, so no assignment goes out unseen.
The brief’s open design questions answered on the canvas: priority versus coverage color, operator versus autonomy choices, warning strength, and a 38-row roster that stays scannable.

Target minutes to task 36 assets: <5Target minutes to task 36 assetsDown from about 20 today. A design target from the brief, not yet tested with operators.
Assets launched with an assignment the operator did not see: 0Assets launched with an assignment the operator did not seeEnforced by the launch summary.
Coverage floor for HIGH sectors: 60%Coverage floor for HIGH sectorsBelow it, an inline warning shows the cost; launch is never blocked.

Reflection

Cohesion came from taking elements away

Early versions looked like three products stitched together. Color and contrast draw the eye to what needs attention. The fastest way to make decisions is to build and get it in front of a few users; from there it’s a matter of refinement.

How I’d validate it

Time to task: run the review flow with SAR coordinators and measure time from proposal to launch against today’s ~20 minutes. Target: under 5.
The five-second glance: show the review screen briefly and ask, “Which sector is weakest?” If operators have to hunt, the coverage fill isn’t working.
Override rate, and why: track how often operators change the autonomy’s assignments. Constant overrides mean they don’t trust the plan; zero overrides might mean they’ve stopped checking. The goal is calibrated trust, not maximum trust.

The open product questions shape the screens

Can the supervisor edit, or only veto? Can the autonomy ever move a locked asset? What counts as good enough to launch? Each answer changes the design, and they’re the first conversations I’d want to have with a product lead.