WASHINGTON, January 21, 2010 – Today’s edition of The New York Times includes a story about internet speed tests, including BroadbandCensus.com, and the various approaches that each of the major providers take in offering speed tests.
The piece, “How Fast is Your Web Connection?” by Peter Wayner, includes ISPGeeks.com, Toast.net, VisualWare, Pingtest.net, and DSLReports.com, along with BroadbandCensus.com.
The article discusses what BroadbandCensus.com aims to achieve in allowing users to “crowdsource” and share data about their internet connections:
Some users are also setting up Web sites to share test results so others can see the general performance of an Internet service provider. DSLReports.com and BroadbandCensus.com, for instance, both offer tests and collect their results to produce reports to help consumers decide whether problems are sporadic or part of a larger pattern.
Among the speed tests mentioned in the article, BroadbandCensus.com is the only one that uses the open source Network Diagnostic Tool of Internet2.
If you’d like to participate in the Broadband Census, and take the BroadbandCensus.com speed test, click here.
We appreciate companies like BroadbandCensus that are actively involved in making the Internet a better place for consumers, and helping consumers evaluate their broadband performance and encouraging “truth in advertising” for service providers regarding the speeds they deliver. However, what is often over looked is “truth in Speedtest advertising”. Just like an access provider should disclose their network management practices, so should a speed test provider, so that Consumer know the truth about what a speed test site is really testing. Sharing one’s speed test experience with the world without sharing the details and variables of the test that was provided can be “slanderous” and “unethical” because it can publicise inaccurate data and consumers can be misled to false conclusion regarding the provider. The fact is, The Internet is complicated, and there are millions of consumers that connect to millions of places, and its impossible for Access Providers to guarantee the path to all locations for everybody. Most importantly, its to expensive for a Speedtest site provider to interconnect with all Access provider under equal comparable conditions, because its simply to expensive for them to do that. I’ll go as far as saying that MOST speed tests only concern themselves with engineering adequate capacity to interconnect with the Major National Access providers of Last mile service. As well, some speed tests sites use extortion tactics to forces high fees from Access provider to engineer connectivity that will result in faster speed test results for them over other providers. Sometimes a fee to large for small providers to pay. The question I pose is, are Third party speed test sites really Neutral? To enforce “truth in speedtests” a Speed test provider should publish in a place impossible not to find easilly, the following details….
1) What providers do they have direct peer or transit connections with?
2) If connecting through public peering switches, which colocation center or peering platform are they connecting through?
3) What speed connections do they have?
4) What is the cost for an Access Provider to purchase a direct connection per MB to the Speed test site?
5) What Access Providers are currently paying fees or contributing funding to the speedtest site?
6) What is the IP Address of the test server.
7) real time graph showing estimated Average capacity availabilty on connections, and average CPU usage of test server, at the time of a consumers’ speed tests.
8) Are they running IPv4 or IPv6?
At the end of the day all that really matter is that consumers experience the performance that they perceive to feel uninhibiting during communication to the sites that they go to most frequently. If speed testing is not testing to those sites, whats the point? The test wont be testing the user’s real world experience or usage patterns. For example, an Access provider specializing in finance companies may have great connectivity to NewYork Stock Exchange, where as an ISP specializing in home users may have better connectivty to UTube and Facebook.
And lastly, BGP routing can vastly differ for IP space dependent on Access Providers interconnecting, and impossible to optimize the BGP path for all. This considers both inbound and outbound paths, half of which is not controlled by the Speedtest provider themselves. At MINIMUM speedtest sites should operate on more than 1 disimilar IP block space or providers, so End Users can select the alternate IP space that could enable a better BGP path across the Internet to them.
The question that comes up is, Is a Speedtest really testing the end user’s speed? Or is it really testing the Speedtest site’s Access Provider and the quality of their peers?
And lastly, Why should speed be the defining factor in judging an access provider? Most people have connections way faster than they ever use. Aren’t other factors more important like, Good tech support, Uptime SLAs, Repair time SLAs, fair billing practices?
I believe using a combination of several third party speed tests can be a good tool that can help consumer identify the “possibilty” of performance issues. But consumers are more often misled than they are helped, if there is not “truth in speedtets” and they are not knowledgeable on the complex issues that contribute to throughput over the Internet. The results really need to be interpreted by a Network Specialist, and End Users should ask for help, before they come to conclusion. Reputable SpeedTest Providers should be able to offer that help, and Most Reputable Access Providers will also. The reality is, most access providers are reputable, and most want their consumers to have the best experience possible. A consumer’s Access Provder should be the first place they go to ask for help to indentify possible performance issues.
An End User should also honestly ask themselves if they are experienced enough to evaluate their connection.
This is a check list of questions they should have the answer to….
1) What is mbps, do they know what their’s is, and do they have a perception of what that specific MBPS should be capable of delivering to them from a user experience perspective?
2) What is packet size, buffer size, and window size? Do they know the window size that their PC negotiates can drastically influence their max possible speed?
3) What Max speed is their PC capable of delivering? Have they verified that? Many laptop chipsets, even with 100mbp ports, cant actually transfer speed tests faster than 10mbps, and the capabilty may not be the same dependant on inbound or outbound
4) Do they understand “packetloss” and the TCPIP congestion avoidance algorythyms, that every PCs uses designed to slow down peak speed to improve quality and latency?
5) Do they know what latency is?
6) Are they making a direct Ethernet connection, or connecting through a Home Wifi connection?
7) Do they understand that 54mbps on their PC’s Wifi software panel does not mean they are not connecting at that speed but instead are just using a modulation more susceptable to packet loss and noise? Do they understand that that value does not measure quality of the connection? HAve they performed speed tests bypassing their inhome Wifi which typically is full of pauses, retranmissions, and self-interference.
Consumers may want their connection broken down to a simple world of grade A-F, but unfortunteately technology isn’t that simple. And any technician that tries to break it down to such a simple factor is ignoring the realities of our technical environment. Acces PRoviders are NOT against End Users testing their speeds. Access PRoviders are against End USers comming to false conclusions because they rely on third party test facilities that do not paint an accurate picture of all the factors involved, nor properly isolate issues that could be contributing to a problem.
Speedtesting is only one step, the second step is troubleshooting to find out why the results are what they are,
It is an unfair conclusion to assume the fault lies with the Access Provider, when the Connectivty is only one small tiny factor contributing to performance inbetween the End USer’s fingers and the remote Speedtest application.
Respectfully,
Tom DeReggi
RapidDSL & Wireless, Inc