Precise one-way frame delay measurements to UNIs at the Metro Edge are critical for service providers providing mobile backhaul. To accurately measure one-way frame delay it is necessary that there be a very close agreement of the Time of Day (ToD) at the UNIs at both ends of the service. In the past GPS was used for this purpose, but today IEEE 1588v2 PTP  is available for accurate synchronization, without the complexity of GPS.
Choosing Metro Edge equipment with IEEE 1588v2 support is important, but it is not enough. The Metro Edge must be properly architected to support 1588v2. To do so requires an understanding of how measurement accuracy degrades when the backhaul network exhibits different delays in each direction.
This paper shows how asymmetric delays degrade one-way frame delay measurement accuracy and explains how service providers can architect their mobile backhaul networks to provide highly accurate customer SLA reports and reduce the costs of synchronizing their network.
For more information on Carrier Ethernet and measuring one-way frame delay for Mobile Backhaul, please see the description of PM-1 and PM-2 in section 8 of MEF 35 .
RTD Measurements Need Precision, but not Synchronization
Before we dig into One-Way Frame Delay, let’s take a look at its predecessor, round trip delay (RTD). A measurement of RTD is simple and popular because it requires no synchronization between the endpoints. As noted in section 8.2 of Y.1731 .
However, one-way frame delay measurement requires that the clocks at
the transmitting MEP and the receiving MEPs are synchronized. For the
purposes of frame delay variation measurement, which is based on the difference
between subsequent frame delay measurements, the requirement for the clock
synchronizations can be relaxed since the out-of-phase period can be eliminated
in the difference of subsequent frame delay measurements.
Frame delay = RxTimeb – TxTimeStampf
The MEP can also make two-way frame delay variation measurements based on its ability to calculate the difference between two subsequent two-way frame delay measurements.
As described above, a timer is started when a packet is sent from one end, and the elapsed time is measured when a response is received. An early example of this method is the RFC792 echo/echo reply (usually referred to by the “ping” utility). The accuracy of a RTD measurement can be improved by adding timestamps at the responder to remove its processing time from the measurement, as defined in the DMM/DMR messages define in Y.1731  and in IEEE 1588v2 . Figure 1 below (reproduced from Figure 12 in IEEE 1588v2 ) shows how the 4-timestamp method works. As shown, the processing time at the receiver may be calculated as t3-t2.
After the Delay_Req message is received at the master and the Delay_Resp message is received at the slave, each side has four timestamps: t1, t2, t3, and t4. The master and slave can then calculate the RTD as:
RTD = (elapsed time at master) – (processing time at slave) = (t4-t1) – (t3-t2)
Note that the absolute values of the timestamps at the master (t4 and t1) do not need to be correlated to the absolute values of the timestamps at the slave (t3 and t2). The accuracy of RTD measurement will be limited by how close the master and slave clocks are in frequency, but it is usually straightforward and economical to achieve the needed accuracy.
The Key to Measuring One Way Frame Delay is Time of Day (ToD)
In contrast to an RTD measurement, a measurement of One Way Frame Delay requires that the two endpoints be synchronized with respect to ToD. There are two basic ways to achieve ToD synchronization:
The advantage of an out-of-band method is that it decouples the measurement
system from the measuring system. This adds additional accuracy and resiliency.
However, an out-of-band method adds cost and complexity.
NTP and PTP Assume Symmetry in the Physical Data Path
The limitation of NTP and PTP in detecting asymmetry is due to the way that
they determine the offset from their master. When the slave receives a timestamp
form the master the time contained is now offset by the time it took to travel
from the master to the slave. The slave can make a calculation of the one way
propagation delay by measuring the RTD as described above and then dividing the
RTD by 2.
Offset = [ ( t2 - t1 ) – ( t4 - t3 ) ] / 2
Again, the division by 2 reflects the assumption of symmetry. The offset is used to correct the timestamp sent by the master.
For those of you who want to understand the details of why asymmetry causes an offset, I have included a section below called “Example Calculations” containing details for the symmetric and asymmetric cases, along with the results of a real-world test.
The Utility of 1DM
We see now that One Way Frame Delay cannot detect differences in the physical data path when a packet timing protocol is used and the path is shared. So, what is the value of One Way Frame Delay?
The answer lies in the difference in the treatment of the timing packets versus the 1DM measurement packets.
Figure 2 below shows how One Way Frame Delay can indicate the type of treatment that user packets are receiving. Note that the slave timing function would typically be implemented in a Network Interface Device (NID), as would the delay measurement function.
The upper red path shows the path taken by timing packets. Queuing delays are minimized for these packets, and known internal delays are corrected using transparent clock functionality.
In contrast, the lower green path shows the path taken by 1DM packets for measuring One Way Frame Delay. They experience the same queuing as other packets in the same COS, giving an accurate indication of the delay seen by the user of the service.
Key Takeaways for Using One Way Frame Delay and Packet Timing in the Metro Edge
Takeaway #1: Asymmetry in the Physical Data Path Can’t Be Detected Using One Way Frame Delay
As described above, packet-based timing protocols assume symmetry in the physical data path. Because they assume symmetry, they can’t detect asymmetry, and any asymmetry in the physical data path will introduce a bias error in the slave clock. This is the nature of the packet timing protocols, and can’t be overcome by any particular equipment or implementation of the timing. This limitation applies to part of the physical data path shared by clock messages and measurement messages. Figure 3 below shows where One Way Frame Delay provides useful info on physical asymmetry and where it is simply RTD/2.
As shown, end-to-end One Way Frame Delay measurements made on EVC12 between
UNI1 and UNI2 will show any differences in the physical data path between the
two masters because the timing packets do not share the same path as the data.
However, One Way Frame Delay measurements made on EVC13 between UN1 and UNI3
will show RTD/2 when there is no congestion, regardless of any physical
asymmetry. This is because the timing packets and the measurement packets travel
along the same path.
As shown, the timing packets and the measurement packets share a common physical path through part of the MEN nearest to the UNI at the tower. Physical asymmetry in this part of the network is not detectable using One Way Frame Delay due to the limitations of packet-based timing.
Takeaway #2: Symmetry in the Physical Data Path Must be Designed For and Built In
If you need symmetry in the physical data path it must designed in. Above all, this means using media with symmetric behavior, such as the following:
Media with asymmetric properties will introduce ToD bias that is not detectable using One Way Frame Delay. Example 6 below shows how a typical VDSL2 deployment can introduce a ToD bias of 9 us. This bias is much larger than the vestigial asymmetry typical in symmetric media. While the bias may not be material in all cases, it should be considered during network design. Other examples of asymmetric media include:
The asymmetry can be verified using an external test set with an external timing source such as GPS, but not by equipment that relies on timing packets for synchronization.
Takeaway #3: The Clock Master Should be as Close as Possible to the UNI
The issues identified in the paper affect the part of the path shared by clock messages and measurement messages. This is why we don’t use a single timing source in the network and distribute the timing packets from there. As noted in ,
Since centrally served NTP timing packets experience the
same network delays as the payload data, they are fundamentally unsuitable as a
one-way delay measurement reference.
Here is an example of calculating offset and One Way Frame Delay when the
assumption of physical symmetry is valid (4 ms in each direction) as well as
when the physical delays are asymmetric (1 ms downstream, 7 ms upstream). The
values that differ in the asymmetric case are highlighted. Please refer back to
Figure 1 above to see the various times graphically.
Example 1: Measure RTD and Estimate ToD Offset
This example shows the initial measurement of the offset of the slave clock
from that of the master. It is assumed that the timing packets are given
priority through the system, and that a number of calculations are performed to
come up with a measurement of the offset.
In the symmetric case, the clock at the slave is adjusted by the measured offset
of 50 ms to achieve synchronization with the master.
Example 2: Measure RTD and Verify ToD Offset
This example uses the same networks shown in Example 1 above. Note that the calculations in this example are repeated periodically to verify the clock alignment. Again, the timing packets are assumed to be given priority treatment through the system.
The measured offset of 0 means that the clocks at the master and slave are still measured to be in synch. However, there is an undetected bias of 3 ms at the slave. This is not detectable because the 4 timestamps (t1, t2, t3, and t4) are the same in both cases!
Example 3: Measure One Way Frame Delay – Physical Data Path
Once the master and slave clocks are synchronized with respect to ToD it is
possible to use these clocks to measure One Way Frame Delay in each direction.
However, this is a bad assumption in the asymmetric case. Table 3 below
contrasts measurements using 1DM frames as defined in Y.1731  in two
situations with the same RTD (8ms): a symmetric case (4ms in each direction),
and an asymmetric case (1 ms downstream and 7 ms upstream). As shown in Table 3
below, the asymmetric case will have an error of 3 ms in the One Way Frame Delay
As with examples 1 and 2, the 4 timestamps (t1, t2, t3, and t4) are the same in both the symmetric and asymmetric cases. While the 1DM measurement is accurate in the symmetric case, it is in error by 3 ms in each direction for the asymmetric case.
Example 4: Measure One Way Frame Delay – Logical Path
This example illustrates the true utility of One Way Frame Delay – measurement of delay variation due to congestion (1 ms downstream, 2 ms upstream) with the physical data path being symmetric (4 ms each way). In this case we look at the difference between the timing packets and the measurement packets due to their different treatment. See Figure 2 for an example application.
Because the physical path is symmetric, both ends are synchronized with respect to ToD. However, the One Way Frame Delay packets are given the same priority treatment as other packets in the class. Any incremental delay will be indicated by the One Way Frame Delay packets.
Example 5: Measure One Way Frame Delay – Logical Path – Impact of Asymmetry
This example illustrates the impact of an asymmetric physical data path on One Way Frame Delay. In this case, there is the same congestion as in the previous example (1 ms downstream, 2 ms upstream). This congestion affects only the data and measurement packets, but it does not affect the timing packets. However, in one case the path is symmetric (4 ms in each direction), and asymmetric in the other (3 ms downstream, 5 ms upstream). The physical asymmetry will affect the timing packets, so slave in the asymmetric case will have a 1 ms offset in its ToD value. See Figure 2 for an example application.
As shown, the physical asymmetry introduces a 1 ms error in each One Way Frame Delay measurement. This is due to the offset of the clock at the slave.
Example 6: Impact of Asymmetric Bandwidth
The previous examples have shown the impact of physical asymmetry on One Way
Frame Delay results. In this section we look at the impact of asymmetric
bandwidth on latency.
Error in ToD = ½ * [ upstream latency – downstream
We can relate the upstream and downstream rates with a ratio:
Error in ToD = ½ * [ (down/up ratio ) (packet size /
downstream rate ) – ( packet size / downstream rate ) ]
For this example we will look at the impact of a VDSL2 link with 40 Mbps downstream and 20 Mbps upstream on a 90 byte / 720 bit SYNC packet.
Error in ToD = ½ * [(down/up ratio – 1) * packet size ]/
Many applications call for a ToD accuracy of 1 us or better, and the ITU is currently discussing a +/- 50 ns requirement. In the example above the VDSL2 media imposes asymmetry of 9 us that does not meet these requirements. Table 6 below shows the impact of various rates and ratios on ToD error. Even at Gigabit rates the errors are still too large (200 ns) to meet the ITU target of 50 ns.
It is possible for a 1588v2 boundary clock node to correct for known delays. If the delay asymmetry were calculated each time the upstream or downstream bandwidth changed, then a correction could be applied. However, this correction is only valid for packets that are the same size as SYNC packets. Other sizes of packets would experience an asymmetric delay that would not be corrected. The bottom line is that physical asymmetry introduces differences in packet delay that can’t be detected and which can affect ToD and/or One Way Frame Delay.
Real World Measurement
We set up two of our 1GigE Ethernet Access Devices (EADs) back to back with a
delay generator between them. They were using NTP for timing with one being the
master and the other being the slave. Synchronization using 1588v2 will yield
the same results.
 IEEE 1588v2, “IEEE Standard for a Precision Clock
Synchronization Protocol for Networked Measurement and
|Last Updated on Monday, 16 September 2013 20:47|