Clock synchronisation: a checklist for trading systems

Server rack with network equipment, illustrating clock synchronisation across trading systems
Photo by panumas nikhomkhai on Pexels

A 20 basis point execution difference on a £250,000 fill is £500. If clocks are wrong, a firm may spend more than that establishing whether the difference came from routing, market movement or a stale feed. Clock synchronisation does not improve an execution by itself. It makes the event record credible enough to investigate.

Put a price on ambiguity

The direct cost is often staff time. Engineers, operations staff and compliance teams may need to reconstruct an order across market data, the order management system, gateways and venue acknowledgements. Where timestamps disagree, the investigation can stall even when every system is otherwise healthy.

These are planning estimates, not audited results for any named firm. Actual costs depend on staffing, order size and infrastructure.

Use the ranges below to estimate exposure, not to forecast a particular incident. The order-impact example is arithmetic on a hypothetical fill, not a claim that clock error caused that slippage.

Typical market planning ranges, not audited company results:

Cost areaTypical rangeWhat drives it
Incident investigation£1,000 to £8,000 per incident10 to 40 staff hours at an assumed loaded cost of £100 to £200 per hour
Execution difference to review£100 to £500 on a £1 million order1 to 5 basis points; not necessarily a loss caused by the clock
Redundant site time infrastructure£5,000 to £30,000 initial spendTypical planning range for grandmasters, timing interfaces and network changes; scope varies

Separate the cost of controls from the cost of uncertainty. A cheaper NTP design may be sufficient for some systems. A venue-facing gateway with tight timing requirements may need PTP, hardware timestamping and tested holdover. The right choice depends on applicable rules, the trading strategy and how much timing error your investigation can tolerate.

Define what each timestamp means

Start by listing the timestamps used to explain an order. A timestamp without a defined event is only a number. For each field, record whether it marks a packet arriving at the network interface, a message becoming available to an application, a decision being made, an order leaving the host or an acknowledgement being received.

  • Event time: when the source says an event happened, such as an exchange message timestamp.
  • Receive time: when your network interface or application observed it. Keep these separate if both are available.
  • Processing time: when a service parsed, validated or acted on the event.
  • Send and response time: when an order left your system and when a response arrived, ideally captured at the interface as well as by the application.

Store the clock domain and timestamp resolution alongside the value. Unix epoch time in nanoseconds does not mean nanosecond accuracy.

A field may have nanosecond representation while its clock is only disciplined to tens of microseconds. Preserve the original source timestamp and add a normalised UTC field; do not overwrite the raw value during reconciliation.

Also distinguish wall-clock time from monotonic time. UTC wall clocks can be stepped or slewed to correct an offset.

Monotonic clocks are suitable for measuring elapsed time within a host, but do not provide a shared timeline between machines. Record both where latency measurement needs elapsed time and cross-system ordering needs UTC.

Map the clock path

Trace time from the reference source to the process that writes the audit record. A common design uses GNSS-referenced PTP grandmasters, redundant paths and PTP-aware switches. The host's NIC captures packet timestamps against a hardware clock, or PHC.

The operating system then disciplines its system clock, and applications read either that clock or the NIC timestamp. Every conversion or hand-off is a potential offset.

NTP remains a sensible option for less timing-sensitive services, management systems and environments where microsecond-level alignment is not required. It is simpler to deploy, but network asymmetry and software timestamping can add variable error. PTP can improve precision, but only if switches, NICs, drivers and host configuration support the chosen mode. One PTP-capable grandmaster does not make an entire path accurate.

Check which clock the application actually reads. On Linux, compare the system clock with the NIC PHC and verify that the process uses the intended source. Confirm that chrony, ptp4l and phc2sys, or their equivalents, do not compete to steer the same clock. Check whether the network uses boundary or transparent clocks, and whether packet timestamping is hardware or software.

Test loss of the primary reference. A grandmaster can enter holdover when GNSS is unavailable; its error then grows with oscillator quality and time without a reference.

Check how the system selects a secondary source, how it flags degraded accuracy, and whether a source switch creates a step. Treat leap-second handling and vendor smear policies as configuration to document, not a detail to discover during reconciliation.

Reconcile before tuning

Pick one order and trace it through market data receipt, strategy decision, risk checks, order gateway, network transmission and venue response. Use identifiers such as client order ID, exchange order ID and sequence number to join records. Timestamps help establish chronology; they should not be the only join key.

Compare timestamps only after accounting for their clock sources and capture points. If a venue timestamp precedes your local receive timestamp, that may be normal when the venue clock is ahead, or when one value marks event creation and the other marks packet arrival.

Estimate the offset using repeated observations, not one message. Where possible, compare hardware receive time with an independent UTC-disciplined reference and track the distribution, not just the average.

Define a reconciliation tolerance for each path. Include expected clock offset, timestamp granularity, capture delay and network asymmetry. If two clocks are each within 50 microseconds of UTC, their readings can still differ by close to 100 microseconds in the worst direction, before capture and software delays. Do not label that difference as processing latency without separating the components.

For firms within scope of EU RTS 25, HFT system clocks have a 100 microsecond maximum divergence from UTC and 1 microsecond timestamp granularity. Other categories have different limits. Confirm the current requirements for the firm, activity and systems with compliance. For broader operational coverage of trading infrastructure, see Trading Technology's infrastructure reporting.

Run this week's checklist

Run these checks on one representative production path first, then extend them to other venues, regions and failover sites. Record evidence, not just green dashboard states.

  1. Inventory clocks. List time sources, grandmasters, NTP servers, PTP domains, host clocks and NIC PHCs for the gateway, market data and order-management paths. Name an owner for each.
  2. Write down timestamp semantics. For every critical field, record the event being timed, clock source, capture point, resolution and whether it is hardware- or software-generated.
  3. Measure offset and stability. Capture offset from UTC, frequency adjustment and source state over a normal trading session. Save minimum, median, 99th percentile and maximum, plus the measurement interval.
  4. Compare host and NIC clocks. Measure PHC-to-system-clock offset on the trading host. Check whether the application timestamps packets using the NIC clock or a later software event.
  5. Exercise failover. Remove or simulate loss of the preferred source in a controlled environment. Verify source selection, holdover alarms, clock steps or slews, and whether the audit log marks degraded timing.
  6. Reconstruct one order. Join events by identifiers and sequence numbers. Calculate feed-to-decision, decision-to-send and send-to-response intervals using timestamps with known clock domains.
  7. Check for impossible sequences. Flag negative durations, response-before-send records, duplicate timestamps and unexplained jumps. Decide whether each is a clock issue, event-definition mismatch or logging defect.
  8. Set alert thresholds. Alert on loss of reference, offset beyond the approved tolerance, holdover age and disagreement between independent sources. Route alerts to an owner who can act during the trading session.

Keep the control measurable

Do not reduce the control to a single offset number. A clock can report a small current offset while its reference is unstable, its holdover is ageing or a downstream service is reading another clock.

Monitor offset, frequency correction, source identity, reference state and timestamp gaps. Keep logs long enough to cover the firm's trade reconstruction and record-retention needs.

Set different thresholds for different systems. A reporting service and a venue-facing order gateway do not necessarily need the same tolerance.

Document why each threshold is appropriate, what happens when it is breached and which timestamps remain usable during degraded operation. Re-test after kernel, NIC firmware, switch configuration, time-service or network changes.

Make the weekly review produce three outputs: a measured offset distribution, one reconstructed order path and a list of unresolved clock-domain assumptions. Assign owners and dates to any gaps. Trading Technology has covered clock controls as an operational concern before; this checklist makes the work testable on a live path rather than leaving it as a configuration claim.

This week, record your gateway's 99.9th-percentile UTC offset and compare it with 100 microseconds.

Preguntas frecuentes

What is the difference between NTP and PTP for trading systems?

NTP is simpler and often adequate for systems that do not need tight microsecond alignment. PTP can provide more precise timing, but it depends on compatible NICs, switches and host configuration. Measure the complete path before choosing.

How should a broker reconcile timestamps from different systems?

Keep each raw timestamp, its clock source, event meaning and capture point. Join records using order identifiers and sequence numbers, then compare times after accounting for clock offset, granularity and capture delay.

What clock accuracy does a trading system need?

The required accuracy depends on the activity, applicable rules and operational use. For firms within scope of EU RTS 25, HFT system clocks have a 100 microsecond maximum divergence from UTC and 1 microsecond timestamp granularity. Confirm the requirements that apply to the firm with compliance.