Karl Bode: After Decades of Talk, Palo Alto Drives Forward on Municipal Fiber Build

For over 20 years, the city of Palo Alto, the “Birthplace of Silicon Valley,” has flirted with the idea of building a city-owned municipal fiber network. Now after years of debate, numerous studies, several false starts, and many unfulfilled RFPs, city officials say they’re finally moving forward with a city-owned fiber network they hope will transform affordable broadband connectivity citywide.

Palo Alto officials tell ILSR that the project will be spearheaded by the city-utility, and deployed in coordination with a major upgrade of the city’s electrical systems. Phase One of the city’s planned fiber deployment should begin later this year, delivering fiber access to around 20 percent of the city–or 6,500 homes and businesses.

Phase One will be funded entirely from the utility’s existing cash reserves. Profits from that deployment will then be used to expand affordable, multi-gigabit fiber access to all of the city’s 63,210 residents. Though no shortage of challenges remain.

A long time coming

That Palo Alto residents have been clamoring for better, more affordable alternatives to regional telecom monopolies for 25 straight years speaks for itself. The high costs, slow speeds, and abysmal customer service of regional telecom giants AT&T and Comcast have long driven the public’s unflagging interest in better, cheaper connectivity options.

Palo Alto officials have been talking about building such a network since at least 1998. In 2001, the city conducted a small network trial that was not expanded due to cost. In 2006, the city issued an RFP that eyed a public-private partnership (P3) to expand fiber access, but says it terminated this effort “due to the lack of financial resources of the private firms.”

In the years’ since, the city has issued several additional RFPs with an eye on P3s, but consistently found that providers weren’t keen on spending the kind of money necessary to ensure even, equitable access to affordable fiber. Under the P3 model, cities generally don’t have much control over the contours, details, and pricing of the finished network.

“[Major ISPs] were looking to pick and choose where they wanted to go within the city,” Palo Alto Director of Utilities Dean Batchelor told ILSR. “They want to go to a lot of the MDUs, multiple dwelling units, large complexes, things like that. And the goal from the council was that they wanted to build the entire city out. There was not going to be a partnership that would agree to build 100 percent of the last mile.”

Officials determined that a PPP wouldn’t provide the kind of uniform affordability the city was aiming for, while an immediate citywide build would be too cost prohibitive.

Ultimately, last winter city leaders decided that an incremental build–starting with the least competitive and most vocally underserved parts of the city–would make the most financial sense.

“We gave the city council three options,” David Yuan, strategic planning manager at the Utilities Department, told ILSR. “The first was a citywide build starting right away, but we would have to bond finance like a hundred million dollars. And then the second option, the one that they chose, was to do an incremental build with the twenty million dollars that we have available.”

COVID, needed electrical utility upgrades change the calculus

Palo Alto is currently the only major city in California that owns and operates its own municipal utility providing residents with electricity, natural gas, water, sewer and limited fiber optic services. As part of the city’s pledge to become carbon neutral by 2030, Palo Alto is conducting a major upgrade of the city’s electrical grid.

That upgrade is expected to cost somewhere between $220 million and $306 million, and take the better part of seven years to complete. A key component of those upgrades is the $26 million expansion of the city’s existing fiber backbone; to be used primarily for utility maintenance, including the management of city smart meters and municipal services.

Last December, city officials voted to approve plans to spend another $20 million to extend access to that network to local residents and businesses. Phase One of the deployment is expected to provide affordable access to around 7,160 homes and 875 businesses, paid for by the city utility’s existing fiber and electric reserves.

“Areas where we were going to go into the first phase of the fiber is pretty much the first phase of the rebuild on the electric side, so we think that there might be some cost savings and some sharing of the electric utility funds to be able to replace these poles, probably at a much quicker pace than what we would do if we were just doing them on the fiber side,” Batchelor said.

While electrical upgrades were a motivating factor, officials say Palo Alto’s newfound urgency was also driven by the widespread frustration with broadband access many locals experienced during the COVID home education and telecommuting boom.

“COVID definitely played a major role,” Yuan said. “I think during that time span, everyone had to work remotely in both school and for work…Internet basically became an essential service. And then people were having problems at home with their capacity or getting dropped off calls. So I think at that time the Council realized that it should be considered a public utility as well.”

Palo Alto generally sees more broadband competition than many municipalities that eye a municipal network. According to city surveys, Comcast provides service to around 70 percent of the city, while AT&T provides either fiber or aging DSL to around 30 percent. Independent ISP Sonic provides gigabit fiber access to roughly four percent of the city.

But the same survey indicates that 36 percent of city residents are frustrated by substandard customer service from their ISP, and more than 46 percent of locals are dissatisfied or somewhat dissatisfied with the cost of broadband service.

A quest for lower prices was cited as the number one reason residents would be likely to switch to the Palo Alto fiber network.

Data consistently indicates that affordability remains among the top obstacles to widespread broadband adoption. Darren Numoto, director of IT for Palo Alto, told ILSR that while it’s too early to detail tier pricing, officials are keenly interested in leveraging the city’s new fiber build to drive down costs for marginalized populations.

Challenges abound, including potential utility pole standoff with AT&T

Last May, the City Council approved a contract amendment with Magellan for program management, organizational change management, network operations and technical support, as well as the utility pole electric make-ready engineering necessary for Phase One.

But expanding the city’s existing fiber network to every resident and business still faces no shortage of hurdles. Including the need to negotiate pole attachment rates with what will ultimately be a direct competitor for the network in some neighborhoods: AT&T.

Palo Alto and AT&T co-own a significant number of the city’s utilities poles under an agreement that dates back to 1918. Some competitors, like Google Fiber, have faced significant delays due to the difficulty of utility pole negotiations with AT&T, a company that has spent the better part of three decades attempting to undermine the development of community broadband alternatives.

“We definitely have concerns,” Yuan said. “They have been slow to respond to some of our requests. So, we are trying to see what kind of cooperation they’re giving us. And we’re also looking into joining the North California Joint Pole Association (NCJPA) to see if there’s more enforcement on that end.”

The city’s plan requires running 48 miles of new underground line and 35 miles of overhead cable requiring the use of 6,000 existing utility poles. Phase One alone requires passing 1,750 existing poles, the attachment of cable to 1,300 poles, the replacement of 100 poles, and working with third-parties to lower hardware on another 325 poles.

Between ongoing environmental review and negotiations with AT&T, the city has its hands full for the remainder of the year.

“I think the environmental review is expected to take about six months,” Yuan said. “AT&T will be an ongoing discussion for a while, but hopefully we can get them to the table soon.”

Once the environmental review and pole attachment negotiations with AT&T are completed, officials say future deployment phases will be dictated by resident interest gauged by community surveys, the need to minimize disruption, alignment with the city’s electrical system upgrades, and neighborhood proximity to existing fiber huts.

Locals can receive updates via the project website and accompanying blog. Unsurprisingly, city officials are thrilled to see meaningful progress on a project a quarter-century in the making.

“We’re very excited about the project. It’s been 20 plus years and waiting,” Yuan said.

Editor’s Note: This piece was originally published on MuniNetworks.org on August 15, 2023, and is reprinted with permission.