BroadbandCensus.com: Starting the Ball Rolling on Crowdsourcing

WASHINGTON, September 22, 2009 – Public and transparent broadband data has now been elevated to the level of a fundamental principle, at least in the Monday speech by Federal Communications Commission Chairman Julius Genachowski.

But it’s worth reflecting on the time – not so long ago – when the quest to collect this kind of broadband data was an unrealized vision at the losing end of a Freedom of Information Act lawsuit.

On Monday, I recounted the history and aftermath of this FOIA request and lawsuit that the Center for Public Integrity filed against Kevin Martin’s FCC. In many ways, that defeat directly set the stage for the launch of BroadbandCensus.com in the fall of 2007.

All this week, during One Web Week, I’m speaking about the history of BroadbandCensus.com from a personal perspective. In this series of blog posts, I’m going to speak about what we’ve been through, who we have worked with to advance the principles of public and transparent broadband data, and what we ultimately aim to achieve at BroadbandCensus.com.

• Part 1: The debate begins with the Freedom of Information Act lawsuit in 2006.
• Part 2, on One Web Day: The founding of BroadbandCensus.com in the fall of 2007.
• Part 3: The Broadband Census for America Conference in September 2008, and our work with the academic community to foster public and transparent broadband data-collection efforts.
• Part 4: BroadbandCensus.com’s involvement with the National Broadband Plan in 2009.
• The Final Part: The role BroadbandCensus.com and broadband users have to play in the creation of a robust and reliable National Broadband Data Warehouse.

BroadbandCensus.com is Born: An Attempt to Go Around the Incumbents

With the loss of the Freedom of Information Act lawsuit – which attempted to obtain carrier-level data about the broadband availability that the FCC holds in its Form 477 database – round one in the battle over broadband data went to the incumbents.

For round two, we decided to go after the broadband data using self-reported broadband data on a web site with a catchy name, like BroadbandCensus.com. In essence, BroadbandCensus.com is an effort to marry the data about the quality of broadband connections that only consumers have, with publicly discoverable data about the state of broadband connections on a geographic area.

All of this began to come together in late September 2007 – at the annual Telecommunications Policy Research Conference at George Mason University School of Law – and in early October of 2007 at the Berkman Center for Internet & Society at Harvard Law School. In a blog post at the time, I wrote:

Last week was a whirlwind of activity for the telecommunications, media and technology project with which I had been engaged since August 2006.The folks at the Berkman Center for Internet and Society at Harvard were kind enough to invite me to speak in their luncheon series on Tuesday, October 9. I discussed “Media Tracker, FCC Watch, and the Politics of Telecom, Media and Technology.” I’m happy to report that the event is now archived on Media Berkman as a webcast.

…

David Weinberger (blog: Joho the Blog) was particularly interested in broadband tracking, and how more detailed information about how to obtain information about the availability of broadband services. (See David’s post.) One of the key efforts of the project, under my direction, was the quest to obtain information from the FCC about the names of the companies that provide broadband service in each particular ZIP code. We filed a lawsuit in federal district court in Washington to obtain the information, under the Freedom of Information Act. The FCC denied our request. Right now the matter is pending before Judge Ellen Huvelle.

Say Doc Searls and John Palfrey, “Drew’s work links in obvious fashion toLawrence Lessig’s next 10 years of work on corruption.”

As I noted in the post, all of this made the Center for Public Integrity’s decision to scale back its “Well Connected” telecommunications and media ownership project particularly untimely. My last day at the Center was on Friday, October 12, 2007.

The active work on BroadbandCensus.com began on October 15, 2007. Together with Andrew MacRae, who had worked with me at the Center for Public Integrity – and now serves as Chief Operating Officer at BroadbandCensus.com – we began to sketch out the model for “crowdsourcing” broadband data collection efforts. On the business side, after an initial period of outreach, Broadband Census LLC was organized as a Limited Liability Company in the Commonwealth of Virginia on December 7, 2007.

BroadbandCensus.com Began Crowdsourcing Internet Data Collection Efforts

To get started, BroadbandCensus.com received some modest seed funding from the Pew Internet and American Life Project, and from the Benton Foundation. We’ve also been blessed by wonderful collaborators of technical and outreach matters: Virginia Tech’s eCorridors Program (I’ll speak more about eCorridors later in the week), Internet2, the Network Policy Council of EDUCAUSE, the National Association of Telecommunications Officers and Advisors, and others.

Working with our web designers and data architects, we built the data-collection mechanism on BroadbandCensus.com, and launched the site live on January 31, 2008. Here are links to some of the early press we received from New Scientist and Telephony Online.

The questions in the “Take the Broadband Census” are basic: (1) Where are you taking the Census, (2) What is your ZIP code, (3) Which carrier do you use? (we require individuals to select from a drop-down menu, rather than a free form box, to ensure standardization), (4) What type of service?, (5) What are your promised speeds, (6) How do you rate the service? (on a scale of 1-5 stars), and (7) Comments?

Home users are required to pick select from among the carriers; office and university users are not. Everyone taking the Broadband Census is required to include their ZIP code, or their ZIP+4 code, and to rate the service quality of their connection.

Very soon after we launched the Take the Broadband Census page, we launched Step 2, the Beta Speed Test, in February 2008. We use the open-source NDT test, or the Network Diagnostic Tool, developed by Internet2. Virginia Tech’s eCorridors Program pioneered the use of NDT for public speed tests.

We do not host any NDT servers. Rather, we direct our internet traffic to eight computers around the country on which they may test their speeds. Using the programming language Java, the applet we deploy collects the results of the NDT test, copies them over to BroadbandCensus.com, and publicly displays the results of the upstream and downstream speeds on BroadbandCensus.com.

All of the content and data-sets on BroadbandCensus.com are published under a Creative Commons Attribution Noncommercial License, allowing state/local governments, and universities, to freely take and republish all of the data-sets, provided that they attribute them to BroadbandCensus.com.

The combination of the Broadband Census questionnaire with the NDT speed test allows important observations to be realized. Are users getting the speeds that they are promises? Is there a correlation between promised and delivered speeds, and the rankings that consumers give to their service quality? Which carriers are the fastest, and are they faster in some parts of the country than in others?

A further dimension of BroadbandCensus.com’s activities is to help consumers monitor how well broadband providers live up to their promised terms of service. See this article about Comcast’s Terms of Service for an early example of this.

I gave an interim report about the progress and use of BroadbandCensus.com in July 2008 at the Joint Techs Conference in Lincoln, Neb.

Spreading the Word About BroadbandCensus.com and the Broadband SPARC

Building sufficient momentum behind BroadbandCensus.com has always been our biggest challenge, particularly in the those early months of 2008. This, remember, was before the intense focus that the presidential campaign, and the broadband stimulus package, placed on a data-driven approach to broadband policy.

Our marketing has been built upon word-of-mouth efforts, cross-promotion by our partners, and through the speeches and articles that I’ve written about the need for public and transparent broadband data. Among these efforts were speeches at Freedom to Connect, Internet2, NATOA, the National Conference for Media Reform, the Institute for Politics, Democracy and the Internet’s Politics Online conference, and in other venues.

One of the ways that BroadbandCensus.com has encapsulated our efforts, in a short-hand way, is through a simple acronym: Broadband SPARC. This stands for the Speeds, Prices, Availability, Reliability and Competition within a particular area.

We’ve pioneered this concept of collecting and aggregating broadband data from a variety of sources – from carriers that voluntarily provide data, from consumers and the speed tests they take, and from publicly available sources. SPARC is an effort to represent the panoply of broadband options, and not simply to focus on whether “broadband,” at any particular pre-defined speed, is available or not.

One Web Day 2008 marked a turning point in our outreach efforts.

We joined together with One Web Day to help promote a conference that we organized in September 2008 – the Broadband Census for America Conference – and to urge people to Take the Broadband Census. One Web Day was one of the non-profit sponsors of the Broadband Census for America Conference, which I’ll discuss in greater detail on Wednesday.

We urge you to also Get Involved in our efforts. You can:

•Take the Broadband Census and Speed Test

•Grab a Button for Your Blog

•Join one of BroadbandCensus.com’s Committees

About BroadbandCensus.com

BroadbandCensus.com was launched in January 2008, and uses “crowdsourcing” to collect the Broadband SPARC: Speeds, Prices, Availability, Reliability and Competition. The news on BroadbandCensus.com is produced by Broadband Census News LLC, a subsidiary of Broadband Census LLC that was created in July 2009.

A recent split of operations helps to clarify the mission of BroadbandCensus.com. Broadband Census Data LLC offers commercial broadband verification services to cities, states, carriers and broadband users. Created in July 2009, Broadband Census Data LLC produced a joint application in the NTIA’s Broadband Technology Opportunities Program with Virginia Tech’s eCorridors Program. In August 2009, BroadbandCensus.com released a beta map of Columbia, South Carolina, in partnership with Benedict-Allen Community Development Corporation.

Broadband Census News LLC offers daily and weekly reporting, as well as the Broadband Breakfast Club. The Broadband Breakfast Club has been inviting top experts and policy-makers to share breakfast and perspectives on broadband technology and internet policy since October 2008. Both Broadband Census News LLC and Broadband Census Data LLC are subsidiaries of Broadband Census LLC, and are organized in the Commonwealth of Virginia. About BroadbandCensus.com.