The problemThe assignment
Build the coordinator server for a request-for-quote trading venue. Everything else (taker CLI, four stock makers, wire types, signing) was handed to you. The protocol has no spec document: common/ and the clients are the spec.
- You build only
server/: the coordinator of an RFQ venue, speaking the protocol defined bycommon/and the shipped clients. - A hidden black-box suite grades it, and graders check that your server follows the rules you wrote in SOLUTION.md (coherence, not a hidden right answer).
- Non-negotiable: no leverage, every trade fully backed, no negative balance, atomic settlement.
server/common, cli, maker, or the wireWhat you were given
| Crate | What it is | Why it matters to you |
|---|---|---|
common/ | Wire types (types.rs), signing (signing.rs), seed parser, HTTP/WS client | Ground truth for every endpoint, payload, signed field order, error code and status. The server depends on it directly. |
cli/ (rfq-cli) | Taker client: submit, subscribe, accept/decline, balances | Shows how a taker actually speaks: subscribes right after submit, counts down to accept_deadline on its own wall clock, declines when stdin closes. |
maker/ (rfq-maker) | Four strategies: tight, lazy, skew, sniper | Deliberately provocative. skew over-commits inventory; sniper cancels and re-quotes 20-200 ms before the close by its own clock. |
server/ | A stub that parsed flags and served /health | The only part you implement. |
seed.toml, keys/, prices.toml | Dev world: 6 assets, 1 taker, 4 makers | Graders use different seeds, so nothing can be hard-coded. |
The three primary challenges (from the README)
1. Recover the protocol
Signed requests with replay protection, the RFQ lifecycle with two timing windows, winner selection, two WebSocket feeds. Reverse-engineered from common/ and the clients.
2. Get the money right
When to check solvency, what to encumber and when to release, tie-breaks, which error wins. Every rule you choose must be documented. Hard constraints: no leverage, fully backed, never negative, atomic settlement.
3. Get the races right
Cancels racing the close, accepts racing the deadline, concurrent RFQs, slow WebSocket consumers. "Assume our tests actively try to break your atomicity."
How it is graded (roughly in order of weight)
- Protocol correctness: endpoints, schemas, error codes, and the error precedence you documented.
- Balance integrity: reservations, atomic settlement, zero-sum conservation under randomized and concurrent load.
- Signatures: canonical message, tampered / replayed / foreign envelopes, nonce monotonicity across action types.
- Timing: windows close on time, late orders and accepts refused, reservations released on every terminal path.
- Concurrency: many simultaneous RFQs, order/cancel storms, slow WS consumers, double-accept races.
Deliverables
- The workspace, building with
cargo build --release -p serveron stable Rust. Binary contract:--seed,--port,--accept-window-secs(default 5),/health200 when ready, fully offline. - Any scripts and tools that helped (you have
scripts/smoke.shand the black-box harness). - A human-written
SOLUTION.md: reconstructed protocol, state structure, atomic settlement, timing races, what you would do differently, how to run anything non-obvious, every policy rule, testing process. PROMPTS.md: every prompt to a coding agent, verbatim, in order (yours is appended automatically by a hook).
What you built, in one picture
| File | Lines | Role |
|---|---|---|
server/src/exchange.rs | 1203 | Pure state machine. No clock, no I/O, no awaits. Takes now as an argument, pushes events into a Vec. All money rules live here. |
server/src/app.rs | 176 | One mutex around the exchange; samples the clock after taking the lock; runs advance(now); dispatches events non-blocking while still holding the lock; per-RFQ wake-up timers. |
server/src/routes.rs | 158 | Thin HTTP: parse, pre-signature shape checks, verify signature, hand off. |
server/src/ws.rs | 277 | Maker broadcast feed and per-RFQ taker feed; slow consumers are closed, never silently skipped. |
server/src/clock.rs | 150 | Wall clock held monotonic (never decreases). |
server/src/error.rs | 155 | One ApiErr type rendering {"error":{"code","message"}} on the right status. |
server/src/lib.rs, main.rs | 120 | Router (incl. JSON 404 / 405 fallbacks) and the binary contract. |
| tests (unit, fuzz, ws, clock, integration, precedence, black-box) | ~13k | See Testing. |
The problemRFQ basics and units
Nothing rests on a book. A taker asks for a price on a specific size, makers answer with firm one-shot quotes that exist only for that request, and the taker trades on the best one or walks away.
- A taker asks for a price on a size; makers answer with firm one-shot quotes; the taker accepts the best one or walks away.
- Two windows: quoting
[created_at, closes_at), then accept[closes_at, closes_at + accept_window). - Order side = the maker's side. Accept side = the taker's side. All money and sizes are integers.
Why an RFQ venue
- Price competition on demand: makers price this exact size, now, head to head.
- Controlled information leakage: only makers see the request, and the taker may hide its side. Then makers quote two-way (both sides, blind) and the taker picks at the end.
- Every quote is firm: if the taker accepts in time, it trades. That is why the venue custodies balances and settles atomically.
The two windows
- Quoting window:
[created_at, closes_at), lengthttl_secsin 1..=29. Makers place and cancel orders. - Accept window:
[closes_at, closes_at + accept_window). The taker holds a firm quote and may accept one side or decline. - No quote, a lapse, a decline, or an unpayable accept all end the RFQ with no money moving.
closes_at is too late to place or cancel; at exactly accept_deadline it is too late to accept.Side conventions (easy to get backwards)
| Field | Whose side | Meaning |
|---|---|---|
SubmitRfqPayload.direction | Taker | Optional. Present: makers are told. Absent: two-way. |
PlaceOrderPayload.side | Maker | A maker sell fills a taker buy (it is the ask). A maker buy fills a taker sell (the bid). |
AcceptPayload.side | Taker | buy lifts the best ask, sell hits the best bid. Directed RFQ: must equal the direction. |
Quote.ask / Quote.bid | Taker view | ask = price the taker may buy at; bid = price the taker may sell at. |
Fill.maker_side, Trade.taker_side | as named | Settled events say whose side explicitly. |
Units given by common
- USD: integer micro-USD.
1_000_000= $1. - Asset sizes: integer lots. Each asset has
lot_decimals; 1 whole unit = 10lot_decimals lots (SOL has 3, so 2 SOL = 2,000 lots). - Prices: integer micro-USD per whole unit.
- Timestamps: unix milliseconds. IDs: UUID v4.
- Notional:
floor(size_lots * price / 10^lot_decimals), product in 128 bits. The same single number is debited from the buyer and credited to the seller, so settlement is exactly zero-sum. common/src/types.rs:88
Notional and reservation calculator
The protocolEndpoints and events
The protocol as reconstructed from common/src/types.rs, common/src/client.rs, and the clients. Router: server/src/lib.rs:31.
- 5 signed POSTs, 5 public GETs, 2 WebSockets (public maker broadcast, signed per-RFQ taker socket).
- Every RFQ produces exactly one
rfq_donefor makers and exactly one terminal event for the taker. - Every non-2xx body is
{"error":{"code","message"}}.
HTTP
| Route | Signed action | Signed fields (after rfq-v1:action:pubkey:nonce) | 200 response | Errors it can return |
|---|---|---|---|---|
GET /health | - | - | ok | - |
GET /v1/assets | - | - | {assets:[{symbol, lot_decimals}]} | - |
GET /v1/balances | - | - | {traders:[{pubkey, name?, usd:{total,reserved,available}, assets:{SYM:{...}}}]} in seed order, built under one lock | - |
GET /v1/balances/{pubkey} | - | - | one TraderBalances | 400 malformed (bad hex), 401 unknown_trader |
GET /v1/rfqs/{id} | - | - | RfqView: id, asset, size, state, created_at, closes_at, accept_deadline?. Never the direction. | 400 malformed (non-UUID), 404 rfq_not_found |
POST /v1/rfqs | submit_rfq | asset, direction or -, size, ttl_secs | {rfq_id, created_at, closes_at} | 400, 401s, 422 unknown_asset / invalid_size / invalid_ttl |
POST /v1/orders | place_order | rfq_id, side, price, size, client_order_id or - | {order_id} | 400 (incl. price 0), 401s, 404 rfq_not_found, 409 rfq_not_open, 422 self_quote_not_allowed / order_too_small / duplicate_client_order_id / invalid_size (notional overflow) / insufficient_balance |
POST /v1/orders/cancel | cancel_order | rfq_id, id:{uuid} or cid:{n} | {order_id} | 400 (not exactly one id), 401s, 404 rfq_not_found / order_not_found, 403 not_your_order, 409 rfq_not_open |
POST /v1/accept | accept | rfq_id, side | {trade} | 400, 401s, 404, 403 not_your_rfq, 409 not_awaiting_accept, 422 side_unavailable / insufficient_balance |
POST /v1/decline | decline | rfq_id | {rfq_id, state:"declined"} | 400, 401s, 404, 403, 409 not_awaiting_accept |
| anything else | - | - | - | 404 not_found; wrong method 405 method_not_allowed (both JSON ApiError bodies) |
Every signed body is an Envelope: {"pubkey": "<64 hex>", "nonce": n, "signature": "<128 hex>", "payload": {...}}. Every non-2xx body is {"error":{"code","message"}} on the status the code is grouped under in error_codes.
WebSockets
GET /ws/makers (unauthenticated broadcast)
rfq_opened{rfq_id, asset, size, direction? (iff taker gave one), created_at, closes_at}rfq_window_closed{rfq_id}rfq_done{rfq_id, outcome: settled | expired | declined | failed, fills: exactly one entry iff settled}
GET /ws/rfq/{rfq_id} (taker, signed subscribe)
- Client sends first frame:
Envelope<SubscribePayload{rfq_id}>, actionsubscribe. It consumes a nonce like any request. - Server:
subscribed{state, closes_at}, thenquote{bid?, ask?, size, accept_deadline} orno_quote, then one terminal:settled{trade} |expired|declined|failed. - Rejection:
error{message} then close 1008.
Event sequence per outcome
| Outcome | Maker feed | Taker feed | HTTP that caused it |
|---|---|---|---|
| settled | rfq_opened, rfq_window_closed, rfq_done(settled, 1 fill) | subscribed, quote, settled | accept 200 |
| expired no quotes | rfq_opened, rfq_window_closed, rfq_done(expired) | subscribed, no_quote, expired | none (time) |
| expired lapsed | rfq_opened, rfq_window_closed, rfq_done(expired) | subscribed, quote, expired | none (time) |
| declined | rfq_opened, rfq_window_closed, rfq_done(declined) | subscribed, quote, declined | decline 200 |
| failed | rfq_opened, rfq_window_closed, rfq_done(failed) | subscribed, quote, failed | accept 422 insufficient_balance |
No new event types were invented: "failed" reuses the existing rfq_done outcome and TakerEvent::Failed.
Things the code decides for you vs. what it leaves open
See Money policies for the full list, each tagged given or our choice.
The protocolSignatures and nonces
The signed bytes are not the JSON body. They are a domain-separated, colon-delimited ASCII string built from the payload's fields in a fixed order. common/src/signing.rs:1
- The signed bytes are
rfq-v1:action:pubkey:nonce:fields, not the JSON. - Nonces strictly increase per key, across every action including the WS subscribe.
- 400s and 401s never consume a nonce; anything that gets past authentication does.
rfq-v1:{action}:{pubkey_hex}:{nonce}:{field_1}:{field_2}:...
absent optional field -> "-"
cancel order ref -> "id:{uuid}" or "cid:{u64}"
Canonical message builder
Why a canonical string instead of signing JSON?
- JSON has no canonical form (key order, whitespace, number formats), so two valid serializations of the same body would have different signatures.
- The
rfq-v1prefix plus the action name is domain separation:declineandsubscribeboth sign exactly one field, the rfq_id, so without the action name a captured subscribe frame could be replayed as a decline of the taker's RFQ. With it, the bytes differ. - The pubkey and nonce are inside the signed bytes, so neither can be swapped.
Nonces strictly increasing per pubkey, across all action types
- Clients start at unix millis and
fetch_add(1)per request (common/src/client.rs:72), which survives restarts. - The server stores the last accepted nonce per pubkey and requires
nonce > last. server/src/exchange.rs:334 - The WS subscribe frame uses the same nonce sequence.
When is a nonce consumed? our choice
| Request stops at | Nonce consumed? | Why |
|---|---|---|
400 malformed_request (unparseable, price 0, cancel with 0 or 2 ids, subscribe rfq_id differs from path) | no | Decided from the request alone, before the signature is looked at. Nothing about the signer is trusted yet. |
401 invalid_signature / unknown_trader / stale_nonce | no | An unauthenticated request must not be able to change server state (otherwise anyone could burn a trader's nonce space with forged envelopes). |
| Any 403 / 404 / 409 / 422, or success | yes | Authentication happens first, under the lock, then business logic. A replay of a signed-but-rejected request is still a replay. server/src/app.rs:91 |
Overlapping requests from one key
Nonce check and nonce write happen in the same critical section as the business logic, so two concurrent requests with nonces 10 and 11 can never both pass a check against 9 and then both commit. A consequence that follows from the protocol: if 11 is processed first, 10 is answered stale_nonce. That is the expected cost of strict monotonicity with out-of-order arrival, and it was left as is on purpose.
The protocolWebSocket feeds
Two feeds with different trust and delivery needs. The core rule: a consumer may be disconnected, but it is never silently skipped. server/src/ws.rs:1
- A slow consumer is disconnected (close 1008). It is never silently skipped and it never blocks the exchange.
- A late taker subscriber gets the ack plus the current quote, or the terminal event.
- Events are sent in exactly the order the state changed.
Maker feed /ws/makers
- One
tokio::broadcastchannel, capacity 1024 (EVENT_QUEUE). - Receiver created before the upgrade completes, so no event falls between handshake and first read.
- Each socket has its own task; the exchange only does a non-blocking
send. Lagged(n)(fell 1024 behind): close 1008"too slow: missed n events".- A frame that cannot be written in 5 s (
SEND_TIMEOUT_SECS): connection dropped as wedged.
Taker feed /ws/rfq/{id}
- Path must be a UUID; first frame within 5 s must be a signed subscribe for that same rfq_id, from that RFQ's taker.
- Order of checks: path UUID, frame parses, rfq_id matches path (all 400-class, no nonce), signature, trader, nonce, rfq exists, ownership.
- Per-subscriber bounded
mpsc(1024). The dispatcher usestry_send; a full queue drops that sender, and the connection task closes with 1008. - After the terminal event: close 1000
"rfq finished".
Why disconnect instead of dropping events? our choice
A maker that silently misses rfq_done would never learn it was filled, and it would keep quoting on a wrong inventory picture. A closed socket is a loud, detectable failure (the stock maker exits with "maker feed closed by server"). Blocking the exchange until a slow reader catches up is not an option either: one slow consumer would stall every trader. So: never block, never skip, close the laggard.
a_maker_consumer_that_never_reads_cannot_stall_the_exchange, maker_feed_closes_a_consumer_a_full_queue_behind_instead_of_skipping, a_full_taker_queue_cuts_that_subscriber_off_and_nobody_else in server/src/ws_tests.rs (deterministic, paused time), and the black-box slow_consumers scenario (a stalled socket gets a gap-free prefix of what a reading socket got, then is closed).Late subscribers our choice
A taker that subscribes after things happened gets a replay snapshot: subscribed, then the current quote if one is held, or the terminal event if the RFQ is over (then close 1000). Authentication, registration and the snapshot happen in one critical section (server/src/app.rs:116), so no event can fall between "take snapshot" and "start listening", and none can be delivered twice.
Why replay the quote: the CLI subscribes right after submit, but a reconnecting or slow client would otherwise have no way to learn the prices or the deadline it is holding. no_quote is not replayed after the fact because the RFQ is already terminal; its terminal expired is.
Event ordering
Events are produced by the state machine into a Vec and dispatched while still holding the lock, using only non-blocking sends. Therefore the order every consumer sees equals the order of state transitions, across both feeds. Nothing awaits under the lock. server/src/app.rs:156
The designArchitecture
Core-first: a pure, clock-free state machine, wrapped by one thin locking layer, wrapped by thin HTTP and WebSocket adapters.
- exchange.rs is a pure state machine: no clock, no I/O,
nowpassed in. - app.rs holds one mutex, reads the clock after locking, applies overdue transitions, then dispatches events without blocking.
- routes.rs and ws.rs are thin; signatures are verified outside the lock.
with(): lock, sample now, advance(now), run the operation, check invariants (debug), dispatch events, unlock. signed() adds authenticate (trader + nonce) first. Timers per RFQ call tick() at close and deadline.One request, end to end: POST /v1/accept
Body<Envelope<AcceptPayload>>parses or 400. server/src/routes.rs:28env.verify()rebuilds the canonical string and checks ed25519, or 401. No lock held. server/src/routes.rs:138app.signed()takes the mutex, samplesnowafter the lock is held, runsadvance(now)(applies every overdue close/expiry in time order). server/src/app.rs:74authenticate(): trader exists, nonce fresh, store nonce. server/src/exchange.rs:334accept(): 404 / 403 / 409 / 422 checks, then compute all four resulting balances with checked arithmetic, then write them. server/src/exchange.rs:615- Events (taker
settled, makerrfq_done) dispatched non-blocking, still under the lock. Lock released; JSON response.
State
| Structure | Type | Notes |
|---|---|---|
Balance | {total, reserved, incoming} | available = total - reserved. incoming is room promised to winning quotes, only nonzero in overflow worlds. server/src/exchange.rs:61 |
traders | HashMap<Pubkey, Trader> | Trader = name, usd Balance, BTreeMap of asset Balances (every declared asset, default 0). |
rfqs | HashMap<Uuid, Rfq> | taker, asset, direction, size, times, state, order ids, best_ask, best_bid, trade. |
orders | HashMap<Uuid, Order> | Kept after they end (needed so a late cancel answers rfq_not_open, not order_not_found). Status: Resting, Winning, Cancelled, Released, Filled. fill_size and notional fixed at placement. |
cids | HashMap<(Pubkey,u64), Uuid> | Holds exactly the resting or winning orders that have a client_order_id. |
nonces | HashMap<Pubkey, u64> | Last accepted nonce. |
due | BTreeSet<(u64, Uuid)> | Pending time-driven transitions ordered by time: each open RFQ's close, each awaiting RFQ's deadline. |
seq | u64 | Arrival sequence for tie-breaks, assigned under the lock. |
initial_usd, initial_assets | u128 | Seed totals for the conservation check (u128 so seeds whose totals exceed u64 still load). |
Invariants checked after every operation (debug builds and tests)
server/src/exchange.rs:1065
- Conservation: sum of every trader's USD total and of each asset equals the seed (in u128).
- Reservations: per balance,
reserved <= totalandreservedequals the sum of what that trader's currently live orders hold (not the original amounts on historical orders). - Room:
incomingequals the room promised to its winners; zero where credits cannot overflow;total + incoming <= u64::MAX. - Structure: order status agrees with its RFQ's state and winners; the cid index holds exactly the live orders with a cid; the timer queue holds exactly the pending transitions.
Why one global mutex?
- Balances are shared across RFQs (one maker quotes many RFQs; a trader can be maker and taker), so per-RFQ locks would not protect them.
- A single serial order makes every rule easy to state: "the first to take the lock wins" decides every race.
- The expensive part (ed25519 verify, ~59% of worker CPU in the storm profile) runs outside the lock. Profile at c=64: about 9% of worker time waiting for the mutex, about 2% holding it. See Performance.
The designMoney policies
Every rule, tagged by whether the shipped code decides it or you did. Graders check that SOLUTION.md states each choice and that the server follows it. Click a card to expand.
- Makers reserve at place: the RFQ size (sell) or the notional (buy). The taker reserves nothing and is checked at accept.
- Taker can't pay at accept: RFQ fails, makers released. Wrong side: refused, RFQ stays open.
- Best price then earliest arrival wins; windows are exclusive; winners can't be cancelled.
The designError precedence
When several things are wrong with one request, which error wins? The README grades "the error precedence you documented". This server's order is pinned, fault combination by fault combination, in server/tests/precedence.rs (254 combinations plus nonce-consumption probes).
- 400 shape, then 401 auth, then 404 exists, 403 yours, 409 state, 422 validation, money last.
- Getting past the 401 tier consumes the nonce.
- Pinned for every combination in
precedence.rs.
- 400
malformed_request: decided from the request alone, before the signature is looked at. Includes rules the wire types state but give no code of their own: a cancel names exactly one order, a price is positive, a subscribe names the RFQ of its socket. - 401:
invalid_signature, thenunknown_trader, thenstale_nonce. Getting past this tier consumes the nonce. - Endpoint checks: find the thing (404), check it is yours (403), check its state (409), validate the request (422), money last.
Try it: pick an endpoint and break the request in several ways
Why this order?
- Cheap and stateless first. Shape errors need no lookup, and answering them first means a malformed request can never touch state or consume a nonce.
- Authenticate before revealing anything. 404/403/409 answers leak information about RFQs and orders, so they come only after the caller is proven to be a known trader with a fresh nonce.
- Existence, then ownership, then state. You cannot check who owns something that does not exist, and telling a stranger the state of your order would leak it. So
not_your_orderbeatsrfq_not_open. - Money last. An order that is invalid for structural reasons (too small, duplicate cid) is rejected as such even if the maker is also broke; insufficient balance is only meaningful for an otherwise valid request.
- Exception worth knowing: on place,
self_quote_not_allowedis a 422 but is checked right after the state check (409), beforeorder_too_small. On cancel, the order is resolved and authorized before the state is checked, so a late cancel of your own order getsrfq_not_open.
The designTiming and races
Every race is decided by one rule: operations are serialized by the exchange lock, and each one is judged at the time sampled after it acquired the lock, after all overdue transitions have been applied.
- One lock decides every race: whoever takes it first wins.
- Each request is judged at the time read after it holds the lock; boundaries are exclusive (
now < closes_at). - Timers only wake things up; every request applies overdue closes and expiries first.
Lazy time, eager wake-ups
The state machine never depends on a timer firing. Every entry point calls advance(now) first, which applies every due close and expiry in time order. Timers (one task per RFQ) only make event delivery prompt when no request comes in. Even GET /v1/balances advances first, so it never shows a reservation that should already be released.
Exclusive upper bounds
Place and cancel need now < closes_at. Accept and decline need now < accept_deadline. At exactly the boundary, the request's own advance(now) processes the transition first. One rule, no ties.
Now after the lock
If now were sampled before waiting for the mutex, a request that queued behind others could act on a stale timestamp and accept after the deadline. Sampling after acquiring the lock makes time monotonic in lock order.
Race explorer
Late wake-up: what if nothing happens until long after the close?
The deadline is anchored to the scheduled close (closes_at + accept_window), never to when the close was processed. A late wake-up cannot extend the accept window, and if both windows have passed, the RFQ goes through both transitions in one advance without ever offering a fresh window. server/src/exchange.rs:440 Test: late_advance_passes_through_both_windows_without_a_fresh_one.
The races the graders will throw
| Race | What decides it | Evidence |
|---|---|---|
| Cancel vs window close | Lock order. Before the close: cancelled and released. After: rfq_not_open and the order may already be a winner (firm). | exact_window_boundaries, cancel_storm_around_the_close, black-box orders_racing_the_close |
| Accept vs deadline | now < accept_deadline sampled under the lock. Settles or expires, never both, never partial. | accept_at_last_millisecond_settles, accepts_racing_the_deadline_settle_or_expire_cleanly, black-box accept_racing_the_deadline |
| Double accept (also accept vs decline) | The first to take the lock moves the RFQ to a terminal state; the second sees not_awaiting_accept. Not the nonce: two accepts with different fresh nonces both pass auth. | double_accept_is_rejected_by_state, concurrent_accepts_settle_exactly_once (fixture checks the RFQ is awaiting with funds and time first), black-box no_double_settlement (8 racing accepts) |
| Two RFQs competing for one taker's money | Taker reserves nothing; each accept checks available at that moment. First settles, later ones fail. | insolvent_taker_fails_the_rfq_and_releases_makers, black-box taker_funds_contention |
| Two RFQs competing for one maker's inventory | Reserve at place; the second order that cannot be backed is refused. | place_counts_reservations_across_rfqs, black-box maker_funds_contention |
| Overlapping requests, one key | Nonce check + write + business logic in one critical section. | overlapping_requests_from_one_key_are_atomic, black-box nonces_and_signatures |
Clock policy our choice
The server's now is the host wall clock (unix ms), held monotonic: it never returns less than it already has. server/src/clock.rs:1
Why wall clock: the clients compare server timestamps with their own wall clocks. The CLI counts down to accept_deadline - unix_millis(); the sniper wakes at closes_at - 20ms by its clock. If the server's clock drifts, a published deadline is not honoured when the client's clock reads it.
| Wall clock held monotonic (chosen) | Monotonic Instant anchored at startup (first version) | |
|---|---|---|
| Published timestamps | Honoured exactly: a window closes when the host wall clock reaches closes_at | Drift from wall time. Measured ~38 ppm on macOS: +33.6 ms after 885 s. Instant also stops while the machine sleeps. |
| Durations | Exact while the host slews (NTP normally slews) | Always exact |
| Host clock stepped back 5 min | Server clock holds still until wall time catches up: for 5 minutes nothing closes or expires, quotes stay firm, reservations stay held; new RFQs stamped with the held time wait too | Unaffected internally, but every published time is off by 5 minutes until restart |
| Host clock stepped forward 5 min | Every skipped close/expiry is overdue and applied at once, in time order; a taker can lose the rest of an accept window; nothing is extended | Unaffected internally, published times off by the step until restart |
| Money safety | Identical: the exchange only needs now never to decrease. Pinned on paused tokio time in server/src/clock_tests.rs with a scripted wall clock, side by side. | |
The designIncoming room (overflow worlds)
The subtlest money rule. It only switches on for a currency whose total across the whole seed exceeds u64::MAX, which no normal seed does, but the graders run "different seed files" and a firm quote must stay firm in every valid seed.
- Only matters when a currency's total across the seed exceeds u64::MAX (~$18.4T in micro-USD).
- Then each winner reserves room to receive its credit at close, so the quote stays firm.
- After that, only the taker side can make an in-time accept fail.
The problem
u64::MAX = 18,446,744,073,709,551,615. In micro-USD that is about $18.4 trillion. Each balance is a u64, and common's seed parser accepts any u64 per balance.
- If a currency's world total fits in u64, conservation means no single balance can ever exceed it, so no credit can overflow. Nothing to do.
- If the world total exceeds u64::MAX (say two makers each seeded with $10T), a balance can be pushed toward the limit by other trades. Then a quote published at close might be impossible to settle at accept because the maker cannot receive the credit. That is a maker-side failure of a firm quote, which the design forbids.
The earlier version checked room at close and again at accept, but another trade could consume the maker's room in between. The adversarial pass reproduced exactly that.
The fix
- At startup, mark each currency whose seed total exceeds u64::MAX (
room_usd,room_assets). server/src/exchange.rs:268 - At window close, an order is only eligible if both credits fit: the taker can receive what it gets and the maker can receive what it gets, without using room already promised to other quotes (
credits_fit). server/src/exchange.rs:529 - When it wins, in a tracked currency, the winner reserves its credit as
incomingon its receiving balance (promote). server/src/exchange.rs:478 - Every room check is
total + incoming + credit <= u64::MAX(has_room), so nobody else's credit can eat promised room. server/src/exchange.rs:76 - At accept: the maker side is covered by its reservation (debit) and promised room (credit). Only the taker can refuse: no funds, or no room to receive (for example because of room promised to its own winning quotes elsewhere). That fails the RFQ with
insufficient_balance. - Room is released on every terminal path without a fill, and consumed on fill.
Room calculator
Why not reserve room everywhere?
Where the world total fits in u64, room is guaranteed by conservation, so tracking it adds state and checks that can never fail for a valid reason. Limiting it to overflow currencies keeps the common path identical to the simple design. A related check you worked through: a 40-decimal asset loads and does not break the math, but always rounds to zero notional with u64 size and price, so a_paid_trade_settles_beyond_18_decimals covers a real paid trade at high precision separately.
Evidence
- Unit tests:
maker_usd_room_is_held_for_a_published_ask,maker_asset_room_is_held_for_a_published_bid,maker_cannot_spend_room_promised_to_its_quote_by_trading_as_taker,taker_room_used_up_by_its_own_trades_fails_like_insolvency,no_room_is_reserved_where_conservation_already_prevents_overflow,promised_room_is_released_on_every_path_that_ends_without_a_fill,credit_overflow_makes_an_order_ineligible_at_close. - Black-box: the
incoming_roomscenario andfuzz --overflow. Seeds 212 and 214 are regression seeds for a harness oracle bug (the server was right; see Testing).
Play with itLifecycle walkthroughs
Step through real flows. Each step runs a JavaScript port of the exchange rules, so balances, reservations and events are computed, not drawn. Use ← → to step.
- Pick a scenario, press Next (or the arrow keys).
- Watch the reserved columns and the two feeds: yellow marks what changed in this step.
Maker feed /ws/makers
Taker feed /ws/rfq/{id}
RFQs
Orders
Balances (total / reserved / available; highlighted cells changed in this step)
Play with itState machine
Six RFQ states from RfqState; two live, four terminal. Click a transition to see what triggers it, what it emits, and where funds go.
open->awaiting_accept-> one ofsettled/expired/declined/failed; oropen->expiredwhen nobody quotes.- Terminal states never change again.
Select a transition.
Order states (server-internal, not on the wire)
| Status | Holds reservation? | Holds cid? | Gets here by |
|---|---|---|---|
| Resting | yes | yes | place in an open window |
| Winning | yes (+ promised room in overflow worlds) | yes | best on its side at close |
| Cancelled | released | freed | maker cancel while open; stops resolving |
| Released | released | freed | lost at close, off-side of a directed RFQ, or winner of an RFQ that ended without filling it |
| Filled | consumed | freed | settled |
Play with itSandbox
Drive the simulated exchange yourself. Same rules and error codes as the server's business logic (no signatures). Time only moves when you move it, which is how the unit tests drive the core.
- Same business rules and error codes as the server's core; you control the clock.
Submit RFQ
Place order
Cancel order
Accept / decline
Responses
Events
RFQs
Orders
Balances
EvidenceTesting
Layered so that each layer catches what the one below cannot, ending in a harness that knows nothing about the implementation.
- Core tests at exact milliseconds, a fuzzer against an independent model, real-server integration tests, a precedence matrix.
- An independent black-box harness that only speaks HTTP/WS, plus a smoke run with the shipped makers and CLI.
| Layer | Where | Tests | Time | What it proves |
|---|---|---|---|---|
| Core unit tests | server/src/exchange_tests.rs | 37 | controlled | Every policy at exact millisecond boundaries: reservations, winners, every release path, rounding, room, cid scope, nonces, snapshots. |
| Differential fuzz | server/src/exchange_fuzz.rs | 1 (400 cases default) | controlled | An independently written reference model predicts the result or exact error code, the exact event stream and every balance after each op. Biased to awkward cases (dup cids, others' orders, extreme prices, steps landing exactly on / 1 ms either side of boundaries). FUZZ_SEED, FUZZ_CASES. |
| WebSocket units | server/src/ws_tests.rs | 7 | paused tokio | Deterministic slow-consumer coverage: queue overflow and stalled writer, using fake sinks (not huge messages filling loopback buffers). |
| Clock tests | server/src/clock_tests.rs, clock.rs | 5 | scripted wall clock | Both 5-minute steps, wall clock vs anchored clock side by side. |
| Integration | server/tests/integration.rs | 17 | real | Real server over HTTP/WS: happy paths, signatures and nonces, late subscribers, concurrent accepts, slow maker socket, cancel storms, randomized concurrent load, overlapping same-key requests, accepts racing the deadline. |
| Precedence | server/tests/precedence.rs | 2 (254 combos) | real | The documented error order for every endpoint: every fault alone, every compatible pair, every fault with all later compatible faults; plus whether each fault consumed the nonce. |
| Black-box harness | server/tests/blackbox/ | quick pass in cargo test | real binary | Spawns the release binary like the graders, uses only HTTP/WS + common. Scenarios, plan-based fuzzer with replay and shrinking, benchmark, demo, clock drift probe. |
| Smoke | scripts/smoke.sh | - | real | Part 1: scripted demo with exact numbers. Part 2: the four shipped makers and the shipped CLI (directed buy and sell, two-way, decline, expiry), then checks conservation and zero reserved. |
Why a black-box harness on top
Unit tests and the fuzz model were written alongside the implementation, so they could share a misunderstanding. The black-box harness derives expectations only from the README, common and the shipped clients, plus a short list of documented policies (check.rs policy), and never links server code.
What it checks
- Ledger: when quiet, every balance equals seed + reported trades using only
common's notional formula; nothing reserved. - Reservations at two quiet checkpoints per round equal exactly what live orders and winning quotes should hold.
- Firmness: an accept by a taker who could certainly pay and receive, whose response arrived before the deadline, must settle at exactly the quoted price.
- No double settlement: at most one of any set of racing accepts and declines succeeds, and the feeds agree.
- Feeds: exactly the right event sequence on both feeds, consistent with
GET /v1/rfqs/{id}, never before the time they announce. - Timing evidence: a response that arrived before a deadline proves the request was in time; one sent after it proves it was late; anything else may go either way.
- Conservation is only summed over a single
GET /v1/balancesresponse, never across separate reads taken during trading.
Found and fixed along the way
| Finding | How it surfaced | Fix |
|---|---|---|
| Quotes not strictly firm when a currency's world total exceeds u64::MAX: another trade could use up the maker's room to receive between close and accept | Known concern from the first implementation; reproduced in the adversarial pass | Reserve incoming room at close in overflow currencies only; taker-side room failure fails the RFQ |
Startup-anchored Instant clock drifted from wall time (~38 ppm, +33.6 ms after 885 s on macOS) | Black-box clock probe | Wall clock held monotonic; trade-off then pinned with deterministic clock-step tests |
Non-UUID /ws/rfq/{id} path did not answer like HTTP | Black-box protocol shapes | Answers malformed_request |
| WS subscribe path-mismatch was checked after the signature, breaking "every 400 before every 401" | Precedence review | Moved before the signature; precedence.rs added |
| Empty 405 body and plain-text WS upgrade rejection | Independent review | JSON ApiError bodies on the same statuses |
| Harness false positive: overflow-world oracle picked a winner that could not receive funds (seed 212), then over-corrected (seed 214) | Black-box fuzz --overflow | Oracle filters by receiving room using promised room as a lower bound; both seeds kept as regressions. The server was right. |
| Benchmark conclusions stronger than the evidence (p99s from too few samples) | Your review | Sample counts printed, unsupported percentiles hidden, profile added |
Run it yourself
cargo test -p server # everything incl. black-box quick pass
scripts/smoke.sh # demo + shipped makers and CLI, checks conservation
cargo test --release -p server --test blackbox -- scenarios
cargo test --release -p server --test blackbox -- fuzz --seeds 1..50
cargo test --release -p server --test blackbox -- fuzz --overflow --seeds 1..20 --traders 6
cargo test --release -p server --test blackbox -- bench --workload all --levels 1,4,16,64,256 --secs 8
FUZZ_CASES=5000 cargo test -p server fuzz_core_against_reference_model
# by hand, three terminals
just server
just makers --offline
just submit --asset SOL --size 2 --direction buy
EvidencePerformance
Measured with the black-box bench on an Apple M1 (8 cores, 4 performance), 8 GiB, macOS, release build, harness and server on the same machine, accept window 1 s, 8 s per level. Rates count successes only, and every workload had 0 rejected requests, so fast rejections are not inflating anything.
- Storm load plateaus at ~44k ok place+cancel requests/s from 16 concurrent clients.
- The lock is held ~2% of the time; ed25519 verification (outside the lock) is the main cost.
How to read it
- Normal workload (whole RFQs: submit, subscribe, 4 quotes, accept or expire) scales linearly with sessions because each session spends most of its time waiting through the 1 s windows. It measures correctness under load, not capacity. Server CPU stays at 21% of one core at c=256.
- Storm (place+cancel on 32 open RFQs, 300 RFQs/s churning, 2 maker sockets that cannot keep up) plateaus around 44k ok requests/s from c=16, with the server at about 4.75 cores.
- Where the time goes (10 ms sample profile, storm c=64): ~59% of worker time in ed25519 verification, ~9% waiting on the exchange mutex, ~2% holding it. Signature checks run outside the lock, so the mutex is not the limit at this load. A 1 ms profile inflated lock waits to 33%, so that one is not trusted.
- Notification lateness (how long after
closes_atthe quote orrfq_window_closedarrives): p50 around 1 ms at every level; storm c=256 p99 13 ms for window_closed. - Slow consumers: the never-reading maker socket was closed with 1008 "too slow" after about 5,100 events at every level, and no request was delayed by it.
EvidenceHow it was built
From PROMPTS.md (29 prompts so far, including task notifications). Useful when asked how you used AI and where your own judgment came in.
- You set the policies; agents implemented them, attacked them, and an agent that did not build it reviewed them.
UserPromptSubmit hook appends every prompt to PROMPTS.md automatically.PrepareLimits and next steps
Things to be honest about, and directions to discuss when asked "what would you do differently with more time". These are prompts for your own thinking, not conclusions.
- In memory only, history never pruned, one global lock, strict nonces, wall-clock steps.
| Limit | Detail | Possible direction |
|---|---|---|
| In-memory only | No persistence: a restart loses balances, RFQs and the nonce table. Clients' nonces start at unix millis, so they survive a restart, but replays of old envelopes would be accepted after one. | Append-only event log or WAL of committed transitions; snapshot + replay; persist last nonce per key. |
| History grows forever | Terminal RFQs and ended orders are never pruned (kept so late cancels and late subscribers get precise answers). | Evict terminal RFQs after a retention period; keep a compact tombstone for error answers. |
| One global lock | Measured not to be the bottleneck (~2% holding, ~9% waiting at storm c=64), but it caps a single venue at one core of business logic. | Single-threaded actor with a command queue (same semantics, no lock contention); or per-trader locks taken in a fixed order; batch signature verification. |
| Strict nonces with concurrent clients | A client sending two requests at once can have the lower nonce rejected as stale if it arrives second. Required by the protocol. | Document it; clients should serialize per key or use separate keys. |
| Wall-clock steps | Back step stalls everything for the step; forward step expires skipped windows at once. | Run with NTP slewing; alert on large steps; optionally a hybrid clock (monotonic durations, wall-clock anchor re-synced). |
| Taker reserves nothing | A taker can hold quotes it cannot all pay for; the extras fail at accept and makers were locked until then. | Optional reservation at quote time for directed RFQs; per-taker limits on open RFQs. |
Snapshot semantics of GET /v1/balances | Built under one lock, so it is a consistent snapshot, but the protocol does not promise that. | Document as a guarantee or not. |
GET /v1/balances/{unknown} | Answers 401 unknown_trader (reusing the closest code in error_codes), not 404. | A choice to state in SOLUTION.md. |
| Benchmark realism | Same machine, closed loop, loopback. | Separate load-generator host, open-loop arrival rates. |
PrepareInterview questions
Likely questions with the facts and code you need to answer them. Try answering out loud before opening each one.
- Answer out loud first, then open the card.
PrepareSOLUTION.md checklist
What the README requires the write-up to contain, plus every policy that must be stated so it is not "graded as an accident". Ticks are saved in this browser. Write the text yourself.
- Everything SOLUTION.md must state. Write it in your own words.