From Megabits to Milliseconds

White Paper

White Paper Provided by Connected Nation

Introduction: This Time Is Different

It is tempting to think that the places where artificial intelligence is “made” will be the places where its economic benefits accrue. That intuition has historical precedent. Detroit’s rise as the center of automobile manufacturing reshaped the American Midwest for decades. Silicon Valley’s role in the semiconductor and personal computing revolutions produced one of the most durable concentrations of wealth and innovation in modern history. As AI investment surges and ever-larger data centers are announced, it is natural for regions to hope that hosting AI infrastructure will make them the next great center of technological gravity.

But this time is different. Hosting a data center will not turn a region into Silicon Valley. Today, infrastructure matters in a different way.

AI is not a single technology but a stack, and different layers of that stack have different infrastructure needs. AI training, which produces new models, is increasingly constrained by power supply and transmission capacity. AI deployment—the inference phase where models are actually used—depends far less on power and far more on communication network performance. Especially important is latency: the “reflexes” of the internet, as distinct from bandwidth or throughput, which is often, somewhat misleadingly, called “speed.”

This distinction will become more important as AI moves out of screens and into the physical world. As AI systems begin to augment human perception in real time, or guide machines through messy environments, milliseconds matter. Network performance requirements are poised to escalate, reshaping the geography of digital advantage and disadvantage and redefining what they mean. Some areas rich in last-mile fiber risk lagging behind because of a lack of local interconnection—though, fortunately, this problem is likely cheaper to solve than the tens of billions spent closing the last digital divide.

That earlier wave of investment has been remarkably successful. Decades of private and public effort have dramatically increased available bandwidth and closed most coverage gaps, even if some remote areas will remain dependent on satellite. Fiber deployment has accelerated to over ten million passings per year (see The Broadband Competitive Landscape on the Eve of BEAD). Even where fiber is absent, cable upgrades have made gigabit service widely accessible. Gigabit download speeds—far beyond the needs of most current applications—are now available to a majority of broadband service locations. The central constraint of the 2010s — insufficient throughput for streaming video - was not solved by accident. Carriers and ISPs competed their way past it, deploying capacity that now far exceeds what any mainstream application demands.

But success has changed the problem rather than solved it. The goalposts have moved.

As bandwidth has become abundant, other aspects of network performance have emerged as binding constraints. The most important of these is latency—the time it takes for data to travel across the network— and the interconnection structures that shape that journey. Latency is a “time tax” on every interaction that gigabit bandwidth alone does not address. Indeed, as bandwidth increases, latency accounts for a larger share of total delay. While measurement remains imperfect, most places are likely already past the point where latency, not bandwidth, determines the effective “speed” of most internet use. As Jensen Huang highlighted in his recent keynote address at the GPU Technology Conference, agentic AI, which often does human-like web search tasks without being constrained by the speed of human thought processes, depends on ultra-low latency for high performance.

So far, this shift has been easy to overlook. As long as a human is in the loop, human perception imposes a natural ceiling: even mediocre latency feels instantaneous. But that constraint is disappearing. As AI systems begin to act in the world—guiding robots, aligning augmented reality overlays, coordinating machines— computer rather than human reflexes become the limiting factor. Latency becomes the binding constraint on syncing a local actuator with a remote “brain.”

“AI infrastructure” is beginning to loom in policymakers’ thinking as a key driver of competitiveness. It is— but distinctions matter. The infrastructure constraints on AI training are primarily about power. But training capacity, while economically meaningful, is not the main determinant of broad regional advantage. AI inference—where AI is actually used—depends far more on communication networks: local peering, interconnection density, and the efficiency of data routing. The emerging domain of physical AI will depend especially on these factors, requiring ultra-low latency and highly reliable connectivity.

At some point, a physical AI “killer app” will emerge—something that people expect to work seamlessly, everywhere. When it does, some regions will find that their networks cannot support it—not because they lack fiber to the home, but because their data must travel too far, through too many hops, to stay within its latency requirements. The frontier is shifting from megabits to milliseconds—from scarcity of throughput to the barrier of reflex speed. The telecommunications infrastructure ecosystem must evolve.