WASHINGTON, May 23, 2009 - The American Recovery and Reinvestment Act of 2009 charges government agencies not only with choosing grant recipients and setting performance bench marks, but also with measuring their results, which only a carefully pre-planned evaluation strategy will enable them to do.
That was the message of a recent paper, “Measuring the Effectiveness of the Broadband Stimulus Plan,” by Scott Wallsten, vice president for research at the Technology Policy Institute. Wallsten played a role in coordinating the “71 Concerned Economists” comment to the Commerce Department’s National Telecommunications and Information Administration. The comment is listed in the BroadbandCensus.com List of NTIA Comments.
The broadband stimulus is unique in that it is a one-time strategy that has not yet been implemented and therefore less of a pre-organized constituency with a stake in a pre-determined result, said Wallsten.
The broadband stimulus has two distinct goals: to improve broadband and to create jobs; the former is the primary goal. The “gold standard” in measuring result is randomization, a sort of “double blind” procedure similar to that used to determine the effectiveness of some medicines. However, due to political pressure, the “gold standard” is not plausible; rather, a “silver standard” may be the answer.
Under such a “silver standard,” Wallsten wrote, agencies would track the build-out of both grant recipients and those who have been denied grants; the latter would serve as an imperfect control group. In addition, he said, the NTIA should fund high-ranked projects and continue funding low-ranked projects until its funds have been exhausted and then collect the data for both.
