Order routing latency: a checklist you can run this week

If the venue is fast but the customer sees a slow acknowledgement, the venue is only one part of the problem. Order routing latency is the time across a chain of systems, queues and network hops, and a single average hides where that time accumulates. This week, measure the chain before changing it.

Start with one order type, one route and one account class. Record the path from client ingress to venue acknowledgement, then split that interval into stages. The aim is not to chase the smallest possible number everywhere. It is to find delay that is both material and under your control, without weakening risk checks or making failure handling less safe.

Set the measurement boundary first

Define the timestamps

Write down what each timestamp means. For a FIX route, a useful sequence is client message received, order parsed, risk check started and completed, route selected, outbound message queued, socket write completed, venue acknowledgement received, and client acknowledgement sent. For a REST or WebSocket API, record equivalent ingress, validation, dispatch and response events. Keep the event names consistent across gateways.

Use a monotonic clock for intervals within a process. Use Precision Time Protocol (PTP) where you need to compare hosts, and monitor offset and clock steps. Wall-clock timestamps that jump during a time adjustment can produce negative or misleading intervals.

Keep both the raw event time and the host clock offset in your telemetry. Do not infer that a message was sent when an application placed it in a queue: record the actual dispatch or socket-write event as well.

Separate acknowledgement from execution

An acknowledgement says the venue accepted or rejected the order, not that it was filled. Track the acknowledgement path separately from time to first fill and time to completion.

Those execution measures depend on market conditions, queue position, order type and price, while the acknowledgement path is more directly useful for diagnosing infrastructure. Include cancels and replaces: they can use different code paths and compete for the same gateway capacity.

Choose a fixed measurement population and report p50, p95 and p99, plus message count and rejection rate. An average can look stable while a small set of orders waits behind a congested queue.

Compare like with like: venue, session, order type, account tier and time of day. A 100-microsecond p99 change is material in one low-latency route and negligible in a public internet API path.

Map the path and locate the waits

Use ranges as a starting point, not an SLA

The figures below are typical market ranges for common electronic trading paths, not audited results or service guarantees. They vary with distance, hardware, protocol, venue load and the way a system defines each boundary. A gateway in the same data centre as a venue will not have the same network profile as a broker serving clients over the public internet.

Use the table to frame questions, not to label a component slow without evidence. An acknowledgement interval that includes both network travel and venue processing cannot identify which side contributed the delay. If the protocol does not provide a trustworthy venue receive time, compare routes and message types, and treat the result as a combined interval.

Inspect queues and shared dependencies

For each internal queue, capture depth, oldest-message age, enqueue-to-dequeue time, drops and worker utilisation. Queue depth alone can mislead: a queue of five items may be harmless if it drains immediately, while a shallow queue with an ageing head item can indicate a blocked worker. Check whether new orders, cancels, market data handling and account updates share a thread pool, lock, database connection pool or network interface.

Trace one order using a stable correlation ID across the client session, order manager, risk service, gateway and venue session. When a system fans out to multiple routes, preserve the parent ID and record each child attempt. The operational material published by Trading Technology can help teams compare the platform and connectivity components in their stack, but your own timestamped trace remains the evidence for this route.

Run a five-day diagnostic

Keep the scope narrow enough to finish. Do not change routing policy, risk controls and hardware at the same time. A workable sequence for this week is:

1. Day one: choose a route and order population. Confirm timestamp definitions, clock health, correlation IDs and event coverage. Check whether application logs capture queue entry and dispatch, not just request receipt and response.
2. Day two: collect a full trading session, including the open and a busy period. Record at least p50, p95 and p99 for each stage, split by order type and venue. Keep rejected orders in the dataset, but report them separately from accepted orders.
3. Day three: align latency spikes with queue age, CPU run queue, garbage collection pauses, database pool wait, network retransmits and venue session events. Check that metrics use compatible time windows; a five-minute CPU average will not explain a 200-millisecond burst.
4. Day four: test one change in a controlled environment or on a limited route. Examples include removing a non-essential synchronous lookup, reducing lock contention, increasing a bounded worker pool, or correcting a poor network path. Capture the same metrics before and after.
5. Day five: review the tail as well as the median. Confirm that rejection, duplicate, timeout and cancel rates did not worsen. Document the change, rollback condition and owner, then decide whether it is safe to expand.

For every proposed fix, state the measured bottleneck, expected effect and trade-off. Moving a risk check off the synchronous path may reduce ingress-to-dispatch time, but it can also permit an order to reach the venue before a required control completes.

Batching writes can improve throughput and worsen single-order latency. A faster path is not an improvement if it compromises the control model or makes behaviour under load less predictable.

Plot latency against offered order rate and queue age. If delay rises sharply only near peak load, capacity or back-pressure is likely part of the issue. If it is consistently higher for one venue, inspect route distance, session behaviour and venue response time. If one account class differs, check for different risk rules, entitlement lookups or isolation settings before assuming a network problem.

Operations teams should also agree what happens when a route degrades. Set thresholds from observed distributions and service requirements, rather than copying a generic microsecond target.

Define whether to alert, stop new orders, switch routes or leave orders in place. A failover route can have different fees, liquidity and execution quality, so automatic switching needs explicit controls and testing. For broader infrastructure coverage, see Trading Technology's platform reporting.

Change only what the trace supports

Once the slow stage is clear, make the smallest change that tests the cause. If the application spends time waiting on a database call, measure the call and its pool wait before adding capacity.

If the network interval is high, compare route traces and retransmits before moving hosts. If venue acknowledgement time varies with message volume, ask the venue for session-level diagnostics and compare against a second route only where the order and account rules permit it.

Protect the baseline. Keep a dashboard that shows stage percentiles, queue age, throughput, rejects and timeouts for the same population.

Alert on sustained tail deterioration and clock-health failures. Retain enough detail to investigate a short burst without logging sensitive order data unnecessarily. Review the dashboard after software releases, venue session changes, data-centre work and market-structure changes that could alter message volume.

Stop doing these things:

• Stop using one end-to-end average as the latency diagnosis.
• Stop treating venue acknowledgement as evidence of a fill.
• Stop tuning from unsynchronised clocks or incomplete event logs.
• Stop changing several stages at once without a rollback test.

#order routing #trading latency #FIX gateway #broker infrastructure #order management system #venue connectivity #execution gateway #latency monitoring