GPS World Editor-in-Chief Matteo Luccio discussed the role and challenges of GNSS timing with Farrokh Farrokhi, founder and president of etherWhere; Beacham Still, director of business development and operations lead at Syncworks; Paul Skoog and Eric Colard, senior technical engineers of product marketing, Microchip, frequency and time systems business unit; and Jeff Gao, GM of communications, enterprise and data centers, SiTime.
etherWhere
Farrokh Farrokhi, founder and president
How is GNSS timing used?
In addition to providing location, GNSS receivers have a built-in time synchronization mechanism. GNSS timing is used to provide the reference for many applications, such as cellular system base stations, server farms, power grid, and financial networks. It is also a source of timing for other systems that require synchronized operations. Examples of those systems are weather radars, distributed sensors and instruments that require synchronous measurement, In networks, we have PTP and other synchronization protocols, but everything goes back to reference GNSS receivers, because everything must synchronize to a global clock. Signal delays are becoming important because they create inaccuracy in the timing at the destination. The beauty of GNSS is that all these systems are synchronized to cesium clocks in the satellites and on Earth.
What are the key challenges for accurate time?
GPS signals have unavoidable imperfections. There are ionospheric and tropospheric delays that must be taken care of at the receiver to provide very accurate timing. The strength and quality of the received signal is also very important. To provide very accurate and robust timing, algorithms that run on these receivers must be very robust and must cope with all the sources of error. The timing pulse coming out of these receivers must be very accurate, so it has implications in both hardware design, algorithm design and how the system works.
Are there error sources specific to timing that are not relevant for positioning?
In timing applications, normally, we need to have much more accurate timing compared to position applications. In GPS, let’s say that position accuracy is one meter with a clear view of the sky. That translates to a few ns of error. To achieve that over, say, a 24-hour cycle, requires much tighter jitter on the receiver. So, the challenge for a timing application is to do a much better job in removing sources of errors compared to positioning.
In the past, a requirement of 20 ns jitter in timing may have been enough for many applications. However, as cellular systems increase in bandwidth and throughput time, TDD systems require tighter timing, and the requirement for jitter becomes more and more stringent. We must come up with new algorithms and new architectures to reduce that jitter.
Another difference is that timing receivers, in general, are stationary. They’re going to be sitting in a cabinet or on a rack somewhere with a fixed antenna.
In timing applications, — for example in a cellular base station — GPS antenna has a good view of the sky at a fixed location, and so there are methods to extract and remove most of those uncertainties and inaccuracies.
Can you group the main critical applications by the accuracy they require? Would you say, for example, that financial transactions have a higher requirement than power distribution?
Yes. For example, for cellular systems up to 30 ns jitter used to be enough. As we move to 5G and 6G, this spec becomes tighter and tighter. We now must be below 5 ns for 6G, as we increase the bandwidth and need to support higher throughput, we are more sensitive to timing inaccuracies. For financial transactions, of course, the requirement is much tighter, so we have to be in the lower ns range, and that has its own challenges. But it’s still doable, especially because we’re going to have more and more GNSS satellites, LEO satellites, enhancements to existing GPS satellites and systems and more commercial satellite systems, so they all can contribute to the improvements in the accuracy of the GNSS time.
When you have any kind of network — let’s say, a power grid — that you need to synchronize, is the solution nowadays to install many timing receivers, or would you be more likely to rely on fiberoptic cables?
It depends on how much error the application can tolerate. For example, for power systems, you need fewer reference receivers that are locked to the satellites; then you can distribute timing using fiber or copper. When it comes to financial transactions, maybe that’s not enough, due to the fiber delays. So, different applications have different requirements.
On what variables do timing receiver manufacturers compete?
The most important parameter is accuracy and jitter. We must cover all different applications. Timing jitter must be reduced below 5 ns and even below 1 ns for the applications that can utilize external corrections to the timing pulse. These are the key differentiators amongst vendors.
One more challenge that we have been dealing with recently is resilience in timing. We must make sure that the timing systems are more robust , for examplein presence of GNSS jammer. During jamming events, we can resort to other resources for holdover. For example, one solution is to use Iridium satellites or other timing systems. In addition, there are LEO satellites that are being launched that could provide secure and resilient timing for more sensitive applications. In generla, redundancy is an important factor. In the future, vendors need to integrate redundant systems as part of their solutions.
What’s the typical drift for holdover during GNSS outages?
The holdover depends on the accuracy of the reference clock. Reference clock accuracies range from a tiny fraction of a ppb to few ppm. In more common commercial applications Rubidium, OCXO and MEMS clocks can provide clock accuracy in the order of 1ppb or lower. That amounts to few micro seconds of drift over a day. On lower cost solutions where standard TCXOs in the range of 1ppm are utilized, this drift can go up to few milliseconds., So, it all depends on how much cost we can tolerate for each solution.
How much did the completion of Galileo and BeiDou improve timing?
The availability, accuracy and reliability of those systems has contributed to the improvement in timing performance. Specifically, Galileo has shown a better performance.. In addition, commercial PNT satellites are also going up — for example, Xona Space Systems — that could also improve timing and provide redundancy.
If Galileo is better for timing than GPS, why not use only Galileo so as not to dilute your timing accuracy?
That is correct. That is the responsibility of the timing algorithm to selectively use a mix of GNSS constellations that provide more accurate timing. When there are more choices available, we always include that as part of our selection algorithms. As you mentioned, Galileo, and GPS are given a higher priority compared to less accurate ones.
What is your company’s niche?
We have one of the lowest jitter solutions in the market. We also have a low power solution that is being used for low power tracking applications. In addition, we offer a cloud solution that can be utilized to reduce the power consumption and improve performance for IOT tracking devices.
Which applications or markets do you focus on the most?
Cellular base stations and server farms are the markets we focus on for our timing solutions. In addition, our low power geolocation solution has been used for IOT tracking solutions.
Syncworks
Beacham Still, director of business development and operations lead
What does your company do?
We are a 25-year partner of Microchip Frequency and Time Systems (formerly Microsemi / SYMMETRICOM), a long-time dominant player in the synchronization and timing space. So, we’re a leading value add reseller, engineering partner and implementation service team. We work actively with telcos, power utilities, transportation networks, and cable companies, as well as some different enterprise applications, to help address synchronization and timing issues, address some of the security and resiliency components that have become a focus in recent years, and ultimately help transition companies from the TDM frequency BITS synchronization that we’ve used for many years, to the gradual implementation of precision packet-based timing.
I work directly with our customers to help understand their applicational vision, where their services might take them, how those requirements might be addressed, and ultimately, how we can ensure the critical need for timing is addressed and stable.
How do the technical challenges differ between GNSS positioning and GNSS timing?
Often the T in PNT is an afterthought. More generally, timing has been somewhat of an afterthought, particularly for something as mission-critical as it is. Positioning is often a very important application, and it can be detrimental for the user if something were to go wrong. But ultimately, timing is a highly critical component for network operation and stability of networks. The static nature of most timing receivers vs the mobile nature of positioning is a contrast. However, it’s a dynamic environment for navigation as you move through different regions and sectors. There certainly are evolving considerations towards where you’re located with static GPS, and particularly some of the risks I think we’ve seen from interference, both unintentional and intentional. They do have some susceptibility just because of their location. So, we can talk through how that evolves, or more over how that actually matriculates, is with your interaction with the general public, as well as with foreign state actors. But with the timing side of it, obviously the critical nature of it. additionally, I think the nature of that makes it more susceptible to prolonged jamming, spoofing, particularly through what we see in the United States, maybe being different than you would find in Eastern Europe or something where there are a lot more state-based actors and spoofing and jamming are a primary concern for us.
To what challenge are you referring in the United States?
We’re designing networks and understanding that, whether it be war zone or state-based actors, there’s very complex spoofing and manipulation that can occur in most frequently power environments, but really in the United States, one challenge for us is the proliferation of personal GPS jammers. You see this through people with corporate vehicles and a fear of being tracked — similar to the rise of VPNs. It’s federally illegal to operate a GPS jammer, but naturally you can buy them off Amazon and eBay, and so it’s one of those situations where it is legal to purchase and own but illegal to use. We see a rising proliferation of that, and ultimately how that impacts static GPS, as you can see, a consistent or maybe repeated incident.
Our power distribution systems, our substations, our telco central offices are in the communities they serve. They’re static and they’re prone to what could be a repeated incident. So, we find this at substations located next to truck stops, night clubs, all the different places that you know, folks might not want to pop up on their corporately tracked vehicles. So, the rise in data has shined the light on people a little bit more, and they take different approaches to try to get around that. So, I think one part of static GPS is that it is susceptible, in a consistent manner, to its neighbors.
How do you deal with that?
The first challenge is understanding what is impacting the system. We tell people that we know a lot about GPS, but it’s not always a precise science, and it can be impacted by such things as solar flares or weird ionospheric conditions that may impact reception and understanding where the jamming is coming from. So, it’s not always straightforward. This can occur with RF interference from neighboring stray signals, anything that falls in the 1.5 MHz range, so often a challenge for operators when they see anomalies on their GNSS-based timing systems is understanding where that interference is coming from.
You can naturally go to the site and try to do audits, and there are tools to try to measure and monitor this. What is more common and what is more practical for network operators is designing and deploying their GNSS networks with the expectation that they’re going to encounter some form of interference, whether that be intentional or unintentional, and so particularly in the modern geopolitical climate in which we live we’re working to design the networks with that intention.
Meaning coupling them with oscillators for holdover?
That’s how traditionally it’s been in the timing space. We’ve always had clocks with onboard oscillators that have a GNSS receiver, and ultimately, if they were to lose that reference to their satellite constellation, they would rely on their onboard hardware to maintain that time as long as they can. So, depending on the quality and the age of that oscillator, they’re going to be able to hold that for longer intervals. But even with a rubidium oscillator, typically the highest quality available, and in the time servers, you’re still maybe looking at three days of serviceable time. So, as the applications as the applications evolve and the requirements for accuracy become more stringent, it becomes more difficult to hold that time.
With the advent of precision packet timing, we’re now able to do some reference redundancies, and we can talk through some of the resiliency architectures and strategies that people are putting in place. However, ultimately, we’re working to augment, supplement and provide redundant and resilient references through not only PTP-based inputs, but also the rise of what looks like to be a return of eLoran and some terrestrial-based signals, low Earth orbit satellites — anything that can be taken to reduce reliance on GNSS satellites to synchronize our systems.
Is the multipath challenge any different for timing than it is for positioning?
Ultimately, we’re relying on the same visibility and reception, and naturally that’s impacted by the deployment location and parameters of what you’re working to achieve. So, I won’t pretend to be a navigation expert, my proficiency is in the timing aspect of that, but I would say that we’re still largely affected by the same conditions. With static antennas, that doesn’t change for the most part. Urban canyons and certain other deployment scenarios do present a challenge. A tough thing for a lot of folks with static antennas is understanding that that sky view changes over the course of the year, and our visibility throughout the seasons and the winter solstice will be different than in the summer.
For most of our operators, everything is always very much black and white as it comes to speeds and different kind of network architectures that they’re deploying, but ultimately GNSS is constantly evolving and changing based on things as obscure as solar flares and ionospheric conditions. It’s a challenge when you’re relying on such a weak signal coming from 12,000 miles above Earth’s surface and things are constantly evolving up there. So, that’s a challenge for operators. Not only using precision packet timing now as just a backup to GPS, it also allows us to deploy exclusively via GPS.
Tell me more about the transition to precision packets.
There’s significant change underway in how we time and synchronize networks. Throughout the telecom industry, we have the general transition from TDM to IP and packet-based networks. Particularly in TDM networks, the idea of UTC-traceable time of day was not really a focus until the advent of NTP, but ultimately it was all frequency synchronization. The idea was that as long as your network was in a frequency lock, and the phased alignment was good, your network would all drift together. So, TDM networks were also inherently synchronous, in a Synchronous Optical Networking (SONET) environment; you can distribute that frequency again throughout your network and pull it down from the overhead to be able to access that frequency. By comparison, packet networks are inherently asynchronous, so it breaks the frequency chains that we’ve long relied on to distribute and synchronize our networks, and ultimately requires a new approach.
So, modern networks and applications are increasingly reliant on precision time from GNSS-derived sources — high speed, low latency, high throughput, all being deployed to meet current and future needs. Not, only is it a departure from the way we’ve distributed that sync, but we’re also requiring new sources of time, like UTC-traceable time of day. So, when we look at both the interconnectivity of global networks, and, moreover, what we see in edge applications, or the idea of distributing away from the core and hosting closer to the customer premise, all of those are going to be relying on time of day.
So, we have a general migration from purely frequency synchronization to UTC-traceable time of day. That is the general evolution of this. Timing is critical, but it’s often an afterthought. It’s considered a means to an end, and often addressed either reactively or retroactively in a kind of break-fix mode. Maybe we take for granted how easily we distributed and accessed frequency and what it took to not only distribute that, but to maintain a serviceable level of accuracy on that. So, now, as we move to precision timing and time of day, we move into the ns range of accuracy. Not only are you trying to keep all your own networks synchronized, you must also have a relative accuracy to the rest of the world. So, some significant changes are taking place, particularly for engineers who have spent their whole career on TDM or SONET networks. It is a certainly a new approach.
How do the timing requirements differ for different end user applications and industry sectors — for example, banking vs. power distribution?
PTP (IEEE 1588) started as an industrial manufacturing protocol to synchronize high-speed presses and drills so that they would fire at the same time. It then came to telecom, with Telecom 2008 being the most common profile. So, you’ll see a PTP power profile, which is designed and intended specifically for smart grid and substation automation and virtualization. You have broadcast profiles. Depending on the medium or the transport technology that you’re using to distribute this time, there are different profiles inside of telecom to address them. In high frequency trading, there are competitive advantages for folks to have high levels of accuracy.
For more critical infrastructure, it’s very much driven by not only new applicational evolutions — for instance, in substations, moving from an old fuse and relay to a virtualized protection scheme, their fault recognition and reconstruction of those things is becoming increasingly more advanced.
All these things — again, whether through a competitive perspective or from a focus on security and resiliency — are requiring an increasingly stringent time reference. Through part of that, people have different motivations, obviously, from the critical infrastructure space, and now we would consider cable and home Internet, with the rise of people working from home, that that may be, it used to be considered just kind of power and light. Ultimately, you know, critical communications such as 911, or now e911 services, now it’s almost evolved to companies like Comcast or Charter Communications, some of the largest consolidated cable providers, are now a critical communication link for people.
So, there are different motivations that drive different industries to invest. Transportation, for example, is a mix where, if you look at things like positive train control (PTC), there are some safety factors that play in there, but there also if you’re a metro or a customer-based rail line, you want to bring new broadband services and Wi Fi to improve your customer experience. So, it can be a mix of both a push and a pull. From the desire to increase your competitive offerings to increasing the resiliency and security with which critical services are offered.
Would you say that the requirements are increasing and converging between these different industries? Are all of them transitioning to PTP? If so, will they all have the same accuracy?
I would say yes, due to the flexibility of PTP, how it’s deployed for different applications, and your ability to translate it to different profiles. To use the example of a power company, they might be using telecom profiles in their transport network and PTP power profiles in their grid and their substations. So, the ability to move that telecom stuff around and translate it to PTP.
That goes on in many industries, where you can begin to see a collapse of the product base into what would be considered previously just purely enterprise or low quality versus a big iron telecom box used for carrier services or critical applications. I do think you see a convergence of the technology and the solutions used.
It depends on the criticality, or the motivation. If it’s a competitive-type thing that’s driven by the motivation of the companies, that will be a bit more push than the converse the inverse side of that being government-regulated utilities and communication companies, where they’re driven by many Homeland Security mandates.
The technology — as far as the hardware, the protocols and the strategies we use to deliver high-precision timing — are converging. The question is, what are the motivations of each industry or enterprise? So, some of that is converging. Ultimately, timing is still somewhat of an afterthought, unless there’s a primary motivation. One challenge of that is people being reactive to those things. Ultimately, you do see some convergence of the technology, but some of the motivations and the thought process of the industries can differ based on what the driving factor for the investment and timing is.
So, a table showing the different timing accuracy requirements of different sectors would be a bit misleading.
Yes. Ultimately, each industry or vertical has different levels of legacy services. For instance, traditional wireline companies are making the transition, but they have a lot of legacy service to migrate. You’ll see this in a pseudo wire-type application or frequency reconstruction, doing your TDM services in an Ethernet environment. Many of these operators are trying to evolve their network. They’re having their eye on the horizon and trying to plan and achieve deployment of new technologies, but they still have to account for the evolution of their old stuff and their installed base.
By contrast, if you look at the emergence of precision timing and high frequency trading and finance — outside of some kind of enterprise NTP servers that they’ve used to roughly sync their domain controllers and their different IT applications — the introduction of PTP into finance is a rather new foray for them. So, they’re more financially motivated to try to get an edge. In comparison to a power utility, they move very slowly, because regardless of whether your phone and internet works, keeping the lights on is an absolute requirement. People have different obligations that they must account for when designing and implementing their networks, so it can vary.
What is Syncwork’s niche in the industry?
We are somewhat unique. Timing is a small niche corner of the industry. We specialize in helping our customers understand what could be considered an afterthought. We’re very good at helping customers analyze, understand and augment their systems. That goes with a lot of consultation and engineering services. This is a technology that does immense interop with other network elements. So, the testing of that and guaranteeing that these services will work as intended is often a drawn-out process. I like to think of timing as a network utility, no different than power and grounding, because if you were to lose timing there may as well be no power to the box.
That is part of our process, and really being an ally to the customer as from the value-add perspective of not only the consultative side, but also assisting with the migration of old services from BITS clocks onto to the new precision timing platform. So, we’re well positioned as a trusted resource and experts and what may be a small, forgotten or undervalued part of the network.
In which sectors do you specialize?
Telecom, utilities, transportation, cable and enterprise, which could be anything from industrial manufacturing to finance or even military development and some of the labs and testing. We go into many labs as well.
How do data centers fit in there?
Data centers is an interesting area. You’re seeing a rise in timing-as-a-service in data centers, where data center companies — most notably folks such as Equinix — have invested heavily in their own timing. Previously, customers had their cage and their roof space where they hung their own antennas and run their own time servers. When you see all these antennas strung out next to each other on the roof, there certainly is a risk of an antenna shorting out and interfering with or jamming its peers. You have increased exposure to the elements and to lightning strikes. So, ultimately, many data center companies are following this same path where, not only are they beginning to host their PTP services and sell that as a service to their customers, rather than their customers implementing their own time servers, they’re also heavily invested in the backup and the kind of resiliency architectures that you might see in other critical infrastructure, such as power utilities. That often incorporates cesium atomic clocks, which is a mature, self-sufficient frequency reference that’s been long Used in TDM networks. It’s interesting, as you begin to incorporate cesium atomic clocks into PTP or precision packet networks and Ethernet deployments, you’re able to really distribute or almost share that. You can think of a cesium atomic clock as like a super-accurate oscillator. You would have a rubidium oscillator in your GPS clocks, they’re not actually maintaining time, per se. They’re working to hold a frequency lock that can then be used to discipline that time of day.
So, a cesium therein can be used to create what we would call an enhanced primary reference time clock (EPRTC) architecture. The spec for that — and this is big in power utilities — is to be able to limit time error over three weeks to 100 nanoseconds. So, that’s a consideration now when moving from frequency to time. The question is not only whether you can keep everything in a phase lock and drifting together, but it’s interesting how you can use a mature technology in combination with emerging technologies, such as IEEE 1588, to really bring a new level of stability to the network.
PTP can be used to distribute and share that extended holdover. So, power utilities, which are heavily invested in these EPRTC architectures, are preparing for “dark sky” incidents. So, they’re trying to prepare for worst case scenarios. Ultimately, through those investments, some of these utilities would be able to maintain 100 nanoseconds of time over a month-long, widespread GPS outage. Localized GPS interference is far more common, and we’re very fortunate that we haven’t seen a whole lot of malicious spoofing, or state-based acts like you would find in some of the Eastern European countries. For instance, Finland no longer really relies on GPS. They’ve moved to a terrestrial, PTP-based network.
Now, with the proliferation in the improvements of PTP and packet networks, GPS is often a backup. We can be more accurate using PTP on the edge than we can be with GPS. Time accuracy in these high-precision networks, and our ability to distribute and maintain that time, can be more accurate than the time difference between two GPS receivers at disparate locations. So, often GPS now is often an option. We keep those in place, distributed throughout the network, in case of bi-directional fiber cuts or losing some of the transport that we use to distribute precision timing, but you can be more accurate, more secure and more stable by using PTP than we can by relying on GPS.
We know that GPS changed the world and proliferated so widely because it was such a revolutionary technology, but ultimately, now we’ve become very complacent to its security and its reliability. Now, we’re forced to shore that up and we can drive many business cases and additional benefits through the implementation of packets.
Microchip
Paul Skoog and Eric Colard, senior technical engineers of product marketing
What is your role?
PS: I’m a product manager. I manage network time servers and instrumentation products that output timing signals such as NTP, PTP, IRIG, 1PPS, and 10 megahertz. The products I manage have very broad applications.
EC: I manage similar products, but I’m more focused on the telecom or communication side, while Paul is more focused on finance, power and other markets.
What are the main differences in the technical challenges between GNSS positioning and GNSS timing?
PS: The main difference is that a timing receiver, by and large, doesn’t move. Since it’s not moving, you can do an awful lot. You can have a lot of satellites that feed into the equations that help you solve the math to get you very accurate timing.
Multipath these days is not as big a deal as it used to be, because these GPS receivers have so many correlators in them that they can figure it out correctly. Multipath might be a problem in a GPS antenna for timing, which usually sits on the roof. If you can keep this signal from reflecting up to the antenna in the first place with an adequate ground plane, that’s probably the singularly most effective thing you can do. I’ve been in GPS a long time. It used to be a very big deal. I never get asked about it anymore. It’s an old problem that’s sort of been solved.
How do the timing requirements differ depending on the application or market?
EC: Many of these markets have a lot of commonalities, because they have communication networks. For example, train and subway networks have communication networks very similar to those of telecoms. In the power industry, you have a communication network and a substation network. In the defense sector, you have confidential communication networks that are very similar to those from AT&T or Verizon. So, all these markets have the same requirements and the same features and challenges. Some markets are more focused on some protocols because of their needs. Some markets are more focused on NTP, for example, such as Paul’s products, which are probably the best in the world for NTP servers. Some other markets, such as telecoms, are more focused on Precision Time Protocol (PTP), which is used across all these segments. The accuracy requirements are also very different from market to market. So, in the telecom market, you can go now in the ns and ps while in other markets, you know, ms is enough. So, it’s really also a big, big difference there in terms of accuracy requirements.
What is the common telecom aspect of these different networks?
EC: For example, the same products can be deployed in communications for transportation and telecom, but they don’t focus on the same aspects. In the train or metro environment, they don’t necessarily use PTP to synchronize their environment, but they introduce SyncE. They used to have legacy signals and then upgraded to focus on SyncE. SyncE in telecom however is a no brainer. It’s been deployed for a long time, but in some of these other markets, it’s a big thing to upgrade their network to SyncE. They use grandmasters, the same as the telecom environment, but they focus on the SyncE need that they have.
These grandmasters nowadays can be deployed for various needs, depending on the segment, but they use the same product. The benefits include resiliency and redundancy, which are common across critical infrastructure. If you’re an infrastructure that needs to be in service 24/7, you need redundancy and resiliency. How do you provide that in your grandmasters? It is a key aspect of the products that we build, for example. So, it’s not necessarily about the protocol, but it’s really about the box and the environment into which you’re deploying and making sure that they help in getting the service 24/7, even if GPS goes down, even if one unit goes down, even if a link goes down.
So, a lot of that goes into the definition and the design of products that can be deployed across these segments, because all of them are critical infrastructure. You don’t want a train or your power or your cell phone to stop operating. So, it’s the same challenge, even if maybe the accuracy is different.
PS: Your question also alluded to an application and a related timing requirement, right? If you’re a bank, you need to be this good, if you’re a power utility, you need to be that good. To a certain extent, some of that time accuracy requirement exists. But, you know, we move timing around in a couple of ways. We run it over a network. We also provide a one pulse per second (PPS) edge, or an extremely good 10 MHz sine wave for radar systems or satellite uplinks. However, more often time now moves over the network, and the irony is, the faster that network gets, the better the time accuracy you can achieve in moving time, because your asymmetric delays that cause time errors get smaller and smaller because your network is faster and faster. You did nothing for it from a clock standpoint.
So, if someone says, “I just need NTP,” well, it’s pretty easy to get 1 microsecond to 10 microsecond time accuracy between an NTP server and an NTP client. They may not even need 1 microsecond to 10 microseconds, but they’re going to take it if they get it, because log file correlation is very useful. Then when you get to PTP, it brings in a lot of hardware, time stamping and on-path assistance to get rid of some of that asymmetric delay. Now you’re down to sub-microseconds, and even approaching low nanoseconds. Then, if you must be down to 1 nanosecond or something smaller, you’re into a one 1PPS application.
One of Eric’s new clocks can get to sub-ns on PTP. You can say, “Who needs it?” Well, the Financial Industry Regulatory Authority (FINRA) requires you to be 50 milliseconds to NIST. That’s a hole so big you can drive an aircraft carrier through it. However, if you want to trade on a stock exchange in Europe, you’re down to 100 microseconds. People typically will get a time server that will get them down to where they’re doing all their time stamping at better than a microsecond, but they put in a rubidium oscillator, so that if GPS goes away — I mean, the cable could be disconnected — they can still finish that trading day and be better than 100 microseconds to UTC.
So, they get a very accurate clock, and they put a rubidium oscillator in it, and they know they’ll get through the day if lightning hits the antenna, or whatever the case may be. So, it is a little hard to say, “Oh, you’re in this industry, therefore you need this particular timing.” Everyone gets what they get for their purposes. We just talked about accuracy, but there are three things that are important to everybody. That’s number two. Guess what number one is? Security.
If I talk to any customer, accuracy, reliability or security will come up. Depending on who you are, it’ll be a combination of those. Today, these clocks are so accurate for most people — sure you’re going to get some high frequency trader would like to get picoseconds or even femtoseconds, right? There are a few people out there that care about that. In general, though, the clocks are pretty accurate, but they connect to the network. All the network guys — the people who manage these networks — cannot plug this clock in until the security people give their stamp of approval, period. That’s just the way it is. They go by the notion of zero trust and if you’re going to plug something into the network they first do penetration tests on it. They’re ruthless. You would think that they want to know how accurate it was. No, they believe you. It’s about security.
What role do you play in that?
PS: I do a lot of listening and learning from huge corporations on planet Earth that purchase our product. They beat the daylights out of it when it comes to penetration testing and security. Then they give us feedback and we make changes. Why don’t we publish that? You never want to publish something that someone found, because then all the bad guys will try to exploit it, right? So, with all our releases, we’re mitigating any common vulnerabilities and exposures (CVEs), and we keep hardening these products, because they have experts that beat on them from the outside. Then they log in as a legitimate admin and start beating on it from the inside to try to hack it. They’re brutal.
EC: Security is a very broad concept. There’s the concept of attacking a particular device, like a timing device, but there’s also the topic of GPS itself. Is it a signal as a time reference that can be trusted? So, you know, anti-spoofing, anti-jamming. Because, basically, time into a grand master reference is coming from GPS most of the time. If it’s not trustable, then you need to find alternatives. So that’s a big topic of discussion and innovation in finding alternatives, you know, to GPS.
Is it better to have GPS everywhere or to distribute time over optical networks?
PS: That’s also part of the security discussion, because people are more and more aware that GPS sometimes is attacked as a signal.
So, a table showing the accuracy requirements of different economic sectors would be a bit misleading, right?
PS: A little bit. Probably the number one reason why people put in a Stratum 1 NTP time server is to make sure that their log file time stamps are accurate, because that makes their network management systems more accurate and reliable, so that they can resolve forensic diagnostic problems faster, because all those timestamps need to correlate. If they don’t correlate correctly, you’re not solving your problem. The government wants you to report all cyber security incidents. You turn your logs over to the government, and they want accurate timestamps, so they can sift through this stuff fast.
On the positioning side of PNT, there is an increasing concern about jamming and spoofing. Is it the same for timing?
EC: The world is becoming more and more unsafe. There are many wars out there, and in all those regions we’ve had much interest in distribution of timing over optical networks. Close to Russia, China, Israel, any many of the conflicts in the world, there have been attacks on these networks every day. Operators in the Middle East, for example, have cell phone networks that work only a few hours every day because they get jammed, so they need to find alternatives. This is also true in Northern Europe, Eastern Europe, and parts of Asia. We face it less in North America and that’s why here there’s less of that urgency.
What does spoofing of timing look like? Has it begun to take place?
EC: Spoofing is more important to operators now than jamming, because we know how to deal with jamming and it is easy to locate the source. Spoofing is the main concern that I’ve seen. For example, I was in Taiwan in May of this year, and the number one thing they want to know is how to detect spoofing. One form of spoofing is somebody who is acting as if it were GPS, but it’s not GPS. They want countermeasures to that and alternatives, because anti-spoofing or anti-jamming are not enough. You need to find alternate time references for when GPS fails for any reason, so it’s an architecture discussion. For example, assisted partial timing support (APTS) has been used for years. It connects to other PTP grandmasters in the network and provides PTP input while GPS is down. Another alternative is to rely less and less on GNSS in general, right? In North America, for example, one of the top three mobile operators was deploying GPS everywhere and using PTP as a backup. That was kind of the architecture when 5G came around. A few years ago, that whole strategy got reversed: PTP became the primary and GPS was used only in some situations.
The alternative to using GPS receivers everywhere is to limit them to very specific strategic points and distribute time over optical networks. There are segments of hundreds of kilometers in many countries without any GPS receivers.
There are also enhanced primary reference time clocks (ePRTCs), which are usually connected to GPS and cesium clocks for resiliency. Many times, carriers now are not even using GPS there. They’re using metrology labs and the national time coming from NIST or similar national time agencies as the time reference, instead of GPS, to limit the use of GPS as much as possible across the network. So, time distribution over networks is becoming more and more of a topic, especially when you own the network. That’s where there’s a big difference between North America and many other countries, in which the operators own their wireline network and control the distribution of time over the whole country. In the United States, there are many third-party backhaul operators. So, Verizon, ATT, and T Mobile don’t necessarily own the backhaul and cannot control how time goes from A to B and guarantee the accuracy across the country.
Alternate forms of solutions include common view. In telecom and power, many of our customers are moving toward time distribution over optical networks to diminish reliance on GPS as a time reference.
What are some of the challenges with that?
EC: It’s a little bit tricky sometimes to deploy, because you may have to consider repeaters, amplifiers, filters or other types of devices. You must deal with calibration and with deployment considerations. It’s a network engineering exercise. However, the standards — such as the one for telecoms — are helping. They define which protocols must be used. About 30% of the units that we sell for grandmasters are sold into these optical distribution environments.
PS: Have you heard of the jammer test in Norway?
Yes.
PS: We have technology that looks at the signals from space, analyzes their integrity and compares them to what you expect from the satellites to detect spoofing. Not everybody can back up their system with a terrestrial network. The people who know that they could be vulnerable buy that option from us. It’s either a hardware basedGNSS firewall or it’s a software option, like a clock that analyzes the GPS signal. If it detects GPS spoofing, It stops using GPS and switches to another time source, such as a holdover oscillator, or getting time terrestrially from another clock over the network.
What is your company’s niche, compared to other timing receiver manufacturers?
EC: Well, you know, some of the considerations include resilience. You know, building grandmasters with no fan, for example, because in timing, if you have a fan, it’s going to interfere with your oscillator — because certain oscillators speed up and slow down as a function of temperature change — and the oscillator is a big aspect of the holdover capabilities and the functioning of your grandmaster. So, having a fan or not having a fan is a big deal for resiliency and for accuracy and performance. And you know, fans are known to be causes of failures, right? So, we have a big architecture design exercise to avoid having fans in some of the grandmasters that we sell, especially in telecom. The second secret sauce, I would say, is our algorithms to meet accuracy in the telecom space, to go into ns and sub-ns that’s really very important in the in the evolution of these networks.
We build these algorithms and our oscillators in-house. We also have external systems, such as cesiums and masers. So, we can provide the whole end-to-end solution without relying on other vendors. We know the quality of everything we build. The other piece is that we rely on Microchip devices inside our own products. That may seem like a detail, but the supply chain crisis is not too far in the in the past, and we face some of these issues using third party components. Now, we maximize the use of our own components inside our products to be more in control of what we can ship.
PS: We sell a very accurate clock, called the SyncServer, to government, military, aerospace and enterprise entities. DOD buys many of these time servers because they are trying to solve the same problems as commercial enterprises. They have servers, and the time must be kept the same. The clocks also sell into radar systems, satellite uplinks, test ranges, and so forth. So, probably the three biggest attributes for the SyncServer product I manage is that they’re very accurate, very secure, and very flexible.
We make it very easy for our customers to adapt this hardware product to their needs, so I suppose our niche is software configurable hardware. We are very software modular. The advantage is that we have a very competitive price for the product. If down the road, you go, “Oh, I want PTP,” it’s just a software option, because every SyncServer ever built has PTP-equipped hardware.
SiTime
Jeff Gao, GM of communications, enterprise and data centers
What is your background and what do you do at SiTime?
I’ve been with SiTime for 16 years. I started in 2008 as Director of Product Marketing for communications products. That was a time when SiTime was just coming out of the initial startup stage with the first generation of commercial products, so I was one of many people who came on board to take those products to market. It was very challenging. I showed our device to a customer who did not believe it was an oscillator because it was black and oscillators were usually in a shiny ceramic package.
The performance of the oscillator back then was mostly for consumer devices or low-end CPU clocks. Sixteen years later, the performance of our devices has improved dramatically, to the point where it’s all about ultra stability. It’s all about position for applications such as GNSS, for location and timing as well as for time distribution and synchronization. My role now is to manage the data center, communication and enterprise business.
Today, what’s driving the growth of our business is all in data centers and AI. That ranges from traditional hyper-scalers, their front-end traditional server infrastructure and back-end AI workloads, to edge data centers. There’s also definitely a portion of the 5G, as well as broadcasting and the power grid, but from a timing point of view, even for GNSS timing, the datacenter is becoming just as critical as the rest of the markets.
What are the timing requirements for data centers? How do they differ from those for other applications?
Conceptually, it comes down to a couple of things. One is the level of accuracy that they need to achieve. The second is reliability or redundancy, if you lose GNSS, how do I ensure the time accuracy until I recover my GNSS signa?. Then, how do I distribute GNSS timing into different locations? Not all locations can have an antenna on the roof. So, it’s actually a very interesting, multi-dimensional problem.
5G probably has the clearest requirement because you need 130 ns of relative time accuracy from one tower to another. It’s used mostly for handoff because, in most of the world, everything is running off TDD systems, so you need that to manage the channels and do the handoffs right. The accuracy requirements increase as you start to provide different services. For example, if you want to do inter-carrier aggregation — meaning that, for example, Verizon and ATT want to use the same tower and the same radio and want to aggregate some services — you start moving from 130 ns down to 65 ns, maybe even down to something more accurate, and that’s all defined as part of 3GPP. This is a world where the need for [???] time accuracy is clearly spelled out by 3GPP or other industry standards.
For financial services, it is defined by the SEC and by ESMA in Europe. More specifically, Government regulations for financial/banking enterprises have driven PTP adoption in the USA (via FINRA Rule 4590 and SEC Regulatory Notice 16-23) and Europe (via ESMA MiFID II requirements).There are two requirements to be legal: one is that the timing must have an audit trail all the way back to UTC. The second is a maximum divergence of 100 μs from UTC. Now, that’s at the transaction level — the servers and the routers. To guarantee that maximum error of 100 μs, what can the maximum time error be? Because everything adds up along the way, right?
For the bigger data centers it is a little bit different, there are no industry-wide standards. Hyperscalers for example, Google, Meta, and Amazoncan each define their own requirements, in terms of the time sensitive services that they want to provide to their end customers. So, the thing they care about is the window of time uncertainty. If at the server level, I have a 5 ms error, or a 1 ms error. I can go to 1 μs of error, or I can go down to 10 ns of error, each of which will enable you to provide a set of services. So, at 100 μs, for example, 99% of all your services are running. At 5 ms, I may have to start shutting down some services. This is where it gets tricky, because each of them is defined a bit differently in terms of what services are available for which accuracy.
The classic example has always been this: you’re in New York and you’re searching for an airline ticket from New York to Hong Kong, but there’s another person in Hong Kong searching for the same ticket. Those services must be synchronized so that you don’t have a contagion problem. You must guarantee a window of time uncertainty to avoid these kinds of problems.
The other reason for that requirement is to avoid having to send a lot of transactions back and forth between servers. More accurate time on the server enables you to minimize the network traffic. So, conceptually, you can look at data center requirements anywhere from 5 ms all the way down to hundreds of ns, or even more accurate.
For all these accuracies, the key mechanism is GNSS timing. In a typical data center, you’re going to start with two grandmasters. That’s probably going to provide 5 ns to 10 ns of accuracy. More importantly, in addition to that, they have extremely good local oscillators that could be, you know, OCXOs, even some atomic clocks, that enable them to hold over if they lose GNSS timing for four, five hours, or ten hours, for whatever reason — up to 24 hours or 48 hours for a huge facility with many AI clusters.
Financial services is very similar. Many financial services units don’t have GNSS antennas for every server, every router, every network card. It just gets tremendously expensive, Plus, beyond the cost of the hardware, distributing the signal all the way down to each server, because most of them are housed in huge warehouses that don’t have access to an antenna. They typically have a grandmaster clock. It’s a box that combines GNSS timing with locally accurate timing to provide the timing to the rest of the facility.
When you must use fiber optics to distribute time, what error does that introduce?
There are a couple of different ways to do this, and we see all of them. One is to use a physical medium, such as fiber, to distribute time from the grandmaster all the way down to the server level. The challenge is not thedelay itself, because you can calibrate for that. It’s the variability of the delay, because of such factors as temperature. So, that could go to in the ms range.
In that scenario, we are now seeing two things. First, you have two or three GPS receivers per facility, and you pair them with highly accurate local clocks. So, you have fail over and you get your 5ns or even better accuracy. Then, you must design a physical distribution network. So, it’s not a single fiber all the way down to the server. You may have two or three layers of distribution, where the first chunk of the fiber goes into a time switch, where you have a very accurate local reference and try to calibrate out some of the delays. Then you have another layer do the same thing. Then maybe it’s two to three layers all the way down to the server level.
Some people do it in a purely optical way. Often it is done optically, then back to electrical, then back to optical, meaning that you literally try to recover the timestamp, because it’s basically sending over a 1 PPS signal. You recover it, then convert into electrical, you do some adjustments to it, then you send it out again over optical. With that mechanism you can probably deliver it down to ns accuracy.
Over what distances? Does it matter?
A few kilometers, but it’s a cable distance. Now, how do you get an accuracy of 200 ns, 5 ns, or even 1 ns? This is where the electronics will come into play, where you must recalibrate the 1 PPS that’s coming in from your grandmaster or from your receiver, and take out, essentially the variations in temperature and distance that are caused by a physical media.
Does GNSS timing pose different challenges from GNSS positioning? Many factors, such as tropospheric delay, apply equally to both.
The challenges are exactly the same. Multipath is extremely relevant to timing. Let’s say, to give an extreme example, that you’re locking onto a single satellite. Depending on whether you have an unimpeded line of sight and no multipath or the signals are bouncing off a building, the difference could be 100 ns to 500 ns.
Can you take care of that upfront, when you install the antenna?
It’s easier when you have a huge data center under the open sky in the middle of Arkansas or Texas but significantly more challenging if you are in the middle of San Francisco. One use case we’ve seen is in India, where they try to provide fixed wireless access using 5G in combination with WiFi because it is more economical than digging up the road and put in the fiber. So, the easiest way for them to do that is to put the aggregation box somewhere in the building where they have many client nodes and the connectivity is basically a 5G sub 6G or mmwave. To do that effectively, they will need to have accurate timing on the aggregation box, which, unfortunately, sometimes sits on a cell tower, which has clearance. Sometimes it’s just sitting on a building with higher buildings around it.
So, multipath is a problem. We’re seeing exactly the same problems for timing. The one difference is that GNSS positioning applications for the most part don’t require holdover. Autonomous driving is still not at level four or five. It’s really just for navigation. For GNSS timing, on the other hand, holdover is a universal requirement, ranging from four hours, for an edge data center or a small facility, all the way to 24 hours for a hyperscaler, large cluster of servers, or in, some extreme cases, even 48 hours to 72 hours for deployment in or near a hostile environment, where you expect jamming and all those bad things to happen. So, the difference is that the holdover requirement is an absolute must in time.
There are multiple redundancies as well, and they don’t necessarily get deployed all at the same time. If I look at a classic core network system, essentially you have three sources for timing. The advantage of GNSS is that over a long period of time it is extremely accurate. The accuracy of an oscillator depends on how much holdover time you require. GNSS has a natural resolution of roughly 20 ns. At 5 ns, you start to rely on your local oscillator to do the next level filtering. For a base station or a core router, you need to get to 5 ns or better. So, you have GNSS native, you have an oscillator to do filtering to get a better accuracy and holdover, then you have network-based timing in a time scale of some sort.
Network time protocol (NTP), which is typically used in notebooks and traditional data centers, only gives you an accuracy of a few ms. In the telecom and data center worlds, everyone is moving over to precision time protocol (PTP), which follows IEEE standard 1588. It’s a network-based timing protocol, where the end node exchanges several protocols back and forth with an upstream node. In the exchange process, you figure out what the time is. It is traceable all the way to a grand master somewhere. So, from the point of view of a data center, a core network, or an edge network you never rely on a single source for timing.
GNSS is always viewed as extremely stable timing that everybody needs when you have access to the receiver and the antenna. Then you rely on the local oscillators and 1588 network timing as complementary technologies to ensure that you will always have timing all the time at a given accuracy. My primary timing is always GNSS. If I lose my GNSS signal, I don’t go into holdover in the router. I go to my 1588. A sequence of events will happen in the system that allow you to move from one time source to another.
What is your company’s niche in the market?
SiTime is the only company in the world dedicated to all aspects of timing. Competitors don’t focus solely in this market. SiTime is also the only player that has a complete portfolio of precision timing components you need to build a complete system. If you look at a complete GNSS timing system, you will need ultra-stable oscillators, some sort of clock, and some software on top of it.
What else distinguishes your company from the competition?
Everything works great in the lab, right? Everyone is saying, “Hey, my solution delivers 5 ns with a 24-hour holdover,” until you put their box in a data center, with a lot of airflow and the heat being generated. Then, suddenly, those 5 ns become 1 ms and 24 hours shrink to 20 minutes.
Our claim to fame is that we have designed our solution to ensure that we actually deliver the performance we claim we deliver in the real-world environment — whether the system is being subject to heat, airflow, rapid temperature changes, shocks, or vibrations, such as when a train goes by a cell tower. We want to ensure that our solution delivers the accuracy and the holdover in the conditions to which the systems is actually being subjected in real world operations.
We can do that because the consistency of the MEMS behavior enables us to compensate with electronics that allow us to be much more resilient to environmental factors.
