Skip to main content

Runbooks

Full desk readiness checklist and integration certification tests.

Desk Readiness Checklist​

Before going live, verify:

API Connectivity​

  • Production endpoints configured
  • Health check endpoint responding
  • WebSocket connection stable

Authentication​

  • EIP-712 signing working
  • Agent authorization configured (if applicable)
  • Nonce management implemented
  • Signature verification tested

Quoting System​

  • Bulk order placement tested
  • Order lifecycle reconciliation verified
  • Fill tracking implemented
  • Cancel functionality tested

Risk Controls​

  • MMP configured and tested
  • Margin model understood
  • Tier restrictions verified
  • Portfolio monitoring implemented

Monitoring​

  • WebSocket feeds subscribed
  • Order/fill reconciliation automated
  • Health checks configured
  • Alerting set up

Kill Switch​

  • Ability to cancel all orders
  • Ability to disconnect immediately
  • Emergency procedures documented

Hedging​

  • Integration with Hyperliquid perps verified
  • Delta hedging strategy implemented
  • Perp margin calculation understood

Settlement​

  • Expiry handling tested
  • Position settlement verified
  • Reference price source understood

Integration Certification Tests​

Test 1: Order Placement​

Steps:

  1. Place single order via POST /order
  2. Verify the order response status is ACKED or OPEN
  3. Check order appears in GET /orders?wallet=...
  4. Verify WebSocket order_updates received

Expected: Order placed successfully, status updates received

Test 2: Bulk Order Placement​

Steps:

  1. Place 10 orders via POST /bulk_order
  2. Verify all orders accepted
  3. Check order statuses via REST and WS
  4. Verify no signature/auth errors

Expected: All orders placed successfully

Test 3: Fill Tracking​

Steps:

  1. Place order that will fill
  2. Monitor WebSocket fills channel
  3. Verify fill received with correct order_id
  4. Check fill appears in GET /fills?wallet=...
  5. Verify order status updates to FILLED

Expected: Fills tracked correctly, order status updated

Test 4: Partial Fill Tracking​

Steps:

  1. Place order that will partially fill
  2. Monitor the WebSocket fills and order_updates channels
  3. Verify the partial fill arrives on fills
  4. Verify order_updates reports PARTIALLY_FILLED with a cumulative filled_size
  5. Aggregate fills by order_id and check the total equals filled_size

Expected: Per-fill quantities on fills, cumulative filled size on order_updates

Test 5: Order Cancellation​

Steps:

  1. Place order via POST /order
  2. Cancel order via DELETE /order
  3. Verify order status: CANCELED
  4. Check WebSocket order_updates received

Expected: Order canceled successfully, status updates received

Test 6: MMP Trigger​

Steps:

  1. Configure MMP via signed POST /mmp-config. Send X-Hypercall-Fence-Wait: 1 when the client will immediately read the projection, and retain the returned X-Hypercall-Fence value.
  2. Place MMP-enabled orders
  3. Trigger MMP by exceeding limits
  4. Verify orders canceled with MMP reason
  5. Check other MMP-enabled orders also canceled

Expected: MMP triggers correctly, orders canceled

MMP configuration reads also require authentication. Sign GetMmpConfig and send its expiry and signature as X-Hypercall-Expires-At-Ms and X-Hypercall-Signature headers. Echo the write response's X-Hypercall-Fence value on this GET to wait for that exact committed configuration. Writes without X-Hypercall-Fence-Wait: 1 intentionally do not return a fence header.

Test 7: Margin Rejection​

Steps:

  1. Place order that exceeds margin
  2. Verify order rejected with "Insufficient margin" reason
  3. Check rejection reason includes required, equity, and shortfall values
  4. Verify order status: REJECTED

Expected: Order rejected with clear reason

Test 8: Tier Restriction​

Steps:

  1. Use a wallet whose tier permits only covered sells (check GET /risk/limits?wallet=...)
  2. Attempt to place sell order without long position
  3. Verify order rejected with tier reason
  4. Place buy order (should succeed)
  5. Place sell order to close (should succeed)

Expected: Tier restrictions enforced correctly

Test 9: WebSocket Reconnection​

Steps:

  1. Connect to WebSocket
  2. Subscribe to channels
  3. Disconnect WebSocket
  4. Reconnect with exponential backoff
  5. Resubscribe to channels
  6. Poll REST to catch up on missed updates

Expected: Reconnection works, state recovered

Test 10: Portfolio Reconciliation​

Steps:

  1. Place and fill orders
  2. Check portfolio via GET /portfolio?wallet=...
  3. Verify positions match fills
  4. Check margin usage correct
  5. Reconcile with WebSocket portfolio updates

Expected: Portfolio state accurate, reconciliation successful

Production Readiness​

Pre-Production Checklist​

  • All integration tests passed
  • Monitoring and alerting configured
  • Kill switch tested
  • Emergency procedures documented
  • Team trained on incident response
  • Support contacts identified

Go-Live Checklist​

  • Certification tests passed with small size in production
  • Production endpoints verified
  • Monitoring dashboards ready
  • Alerting configured
  • Team on-call schedule established
  • Documentation reviewed

References​

Last updated on