Artificial Intelligence
Labeling and Rating Potentially Harmful AI Systems Is Inherently Complex, Say Brookings Panelists

October 5, 2020 — Germany has attempted to improve awareness of the potential degree of consumer harm that artificial intelligence systems cause by developing categories about the degree of environmental, economic, and societal impact it could cause.
To hear expert opinions on whether policymakers and stakeholders in the United States should adopt similar models and utilize AI certifications, ratings and labeling, the Brookings Institute hosted a panel on Thursday to consider potential “blind spots.”
“We need some sort of testing or auditing mechanism,” said Elham Tabassi, chief of staff of the information technology laboratory at the National Institute of Standards and Technology.
“Standards for AI are lacking right now,” Tabassi said, and experts “don’t know how to test AI systems for bias.”
Mark MacCarthy, adjunct faculty of communication, culture, and technology at Georgetown University, detailed what has traditionally occurred when industry applied ethics systems in an attempt to manage new forms of media or technology.
The Platform for Privacy Preferences Project, or P3P, was “a system developed when the public was worried about safety online, which allowed users to opt out of online tracking,” said MacMarthy.
The idea was that websites would post their privacy policies in P3P format and web browsers would download them automatically and compare them with each user’s privacy settings.
In the event that a privacy policy did not match the user’s settings, the browser could alert the user, block cookies, or take other actions automatically. Yet, the system “did not come to anything.”
“A vast number of systems have been proposed for ethics of artificial intelligence,” said MacMarthy. “The rating system is not well developed, but I hope this conversation can help it,” he said.
MacCarthy maintained that he held two reservations. First, he noted that most non-mandatory ethics regulations have failed because of a lack of industry buy-in. It “would need to be mandatory,” as “industry has no reason to use it,” said MacCarthy.
Second, “even with coercion the objective of an AI rating system is to give consumers more information about these systems, and to be honest, that may not be enough,” he concluded.
“There are so many complications in the AI process, it makes it extremely difficult,” to pinpoint where systems go wrong, said John Villasenor, nonresident senior fellow of governance studies at the Center for Technology.
One issue, according to Villasenor, is that AI systems are “produced by commercial entities, to make a profit.”
“Many companies don’t put their source code on the internet because they want to remain competitive,” he said. Further, there are over “500,000 lines of code, it may not be easy to figure out exactly what the algorithm is doing.”
The panelists did not rally around the German approach of looking at high risk algorithms and labeling them.
“How to decide if something is high risk?,” questioned Elham. He also wondered if labeling and categorizing AI would create a false sense of complacency.
Artificial Intelligence
Still Learning About Artificial Intelligence, Legislators Say Congress Must Act
Markey also urged Meta CEO Mark Zuckerberg to halt the release of an AI-powered chatbot.

WASHINGTON, September 30, 2023 – Although Congress is still learning key aspects of artificial intelligence, senators and representatives speaking at an AI summit on Wednesday said they believed the urgency of the moment required the passage of “some narrow pieces” of legislation.
The same day that Sen. Ed Markey, D-Mass., sent a letter to Meta CEO Mark Zuckerberg urging him to halt the release of AI-powered chatbots that the social media giant plans to integrate within its platforms, Markey urged the Federal Trade Commission to protect minors from AI-powered software.
Markey, speaking at Politico’s AI and Tech Summit, cited suicide rates amongst minors using social media and a recent warning from the Surgeon General about social media and adolescent mental health.
“We’re not going to be able to handle devices talking to young people in our society without understanding what the safeguards are going to be,” Markey said.
His message to Big Tech was: “Don’t deploy it until we get the answers to what the safeguards are going to be for the young people in our society.”
Similarly, Sen. Todd Young, R-Indiana, said he believed it was “very likely” that Congress would pass “some narrow pieces” of a regime regulating AI.
“I hope we go wider and consider a host of different legislative proposals because our innovators, our entrepreneurs, our researchers, our national security committee, they all say that we need to act in this space and we continue to lead the way of the world and manage the many risks that are out there around the financial markets,” Young said.
Other legislators proposed other specific facets of AI regulation.
Congressman Ted Lieu, D-Calif., proposed a law to prevent AI from autonomously using nuclear weapons. He also suggested a national AI commission.
Such a commission would help create a public record about how and why AI should be regulated. Doing so would be preferable to the approach in which Senate Majority Leader Chuck Schumer, D-N.Y., has been hosting closed-door briefings with tech giants on the topic.
“AI is innovating so quickly that I think it’s important that we have the national AI commission experts,” Lieu said. “There’s quite a lot of legislation to work on that, that can make recommendations from Congress asking what kind of AI we might want to regulate, how we might want to do about doing so and also provide some time for AI to be developed.”
Rep. Jay Obernolte, R-Calif., vice chair of the Congressional Artificial Intelligence Caucus, said that Congress is doing a “great job” educating themselves on AI but that creating legislation that has a human centric framework needs to be properly defined.
“By framework, I don’t mean a bunch of buzzwords flying in close formation, right?” Obernolte said. “What does it mean for AI to be human centered? What role does government have in making sure that they are human centered?”
Artificial Intelligence
Companies Must Be Transparent About Their Use of Artificial Intelligence
Making the use of AI known is key to addressing any pitfalls, researchers said.

WASHINGTON, September 20, 2023 – Researchers at an artificial intelligence workshop Tuesday said companies should be transparent about their use of algorithmic AI in things like hiring processes and content writing.
Andrew Bell, a fellow at the New York University Center for Responsible AI, said that making the use of AI known is key to addressing any pitfalls AI might have.
Algorithmic AI is behind systems like chatbots which can generate texts and answers to questions. It is used in hiring processes to quickly screen resumes or in journalism to write articles.
According to Bell, ‘algorithmic transparency’ is the idea that “information about decisions made by algorithms should be visible to those who use, regulate, and are affected by the systems that employ those algorithms.”
The need for this kind of transparency comes after events like Amazons’ old AI recruiting tool showed bias toward women in the hiring process, or when OpenAI, the company that created ChatGPT, was probed by the FTC for generating misinformation.
Incidents like these have brought the topic of regulating AI and making sure it is transparent to the forefront of Senate conversations.
Senate committee hears need for AI regulation
The Senate’s subcommittee on consumer protection on September 12 heard about proposals to make AI use more transparent, including disclaiming when AI is being used and developing tools to predict and understand risk associated with different AI models.
Similar transparency methods were mentioned by Bell and his supervisor Julia Stoyanovich, the Director of the Center for Responsible AI at New York University, a research center that explores how AI can be made safe and accessible as the technology evolves.
According to Bell, a transparency label on algorithmic AI would “[provide] insight into ingredients of an algorithm.” Similar to a nutrition label, a transparency label would identify all the factors that go into algorithmic decision making.
Data visualization was another option suggested by Bell, which would require a company to put up a public-facing document that explains the way their AI works, and how it generates the decisions it spits out.
Adding in those disclaimers creates a better ecosystem between AI and AI users, increasing levels of trust between all stakeholders involved, explained Bell.
Bell and his supervisor built their workshop around an Algorithm Transparency Playbook, a document they published that has straightforward guidelines on why transparency is important and ways companies can go about it.
Tech lobbying groups like the Computer and Communications Industry Association, which represent Big Tech companies, however, have spoken out in the past against the Senate regulating AI, claiming that it could stifle innovation.
Artificial Intelligence
Congress Should Mandate AI Guidelines for Transparency and Labeling, Say Witnesses
Transparency around data collection and risk assessments should be mandated by law, especially in high-risk applications of AI.

WASHINGTON, September 12, 2023 – The United States should enact legislation mandating transparency from companies making and using artificial intelligence models, experts told the Senate Commerce Subcommittee on Consumer Protection, Product Safety, and Data Security on Tuesday.
It was one of two AI policy hearings on the hill Tuesday, with a Senate Judiciary Committee hearing, as well as an executive branch meeting created under the National AI Advisory Committee.
The Senate Commerce subcommittee asked witnesses how AI-specific regulations should be implemented and what lawmakers should keep in mind when drafting potential legislation.
“The unwillingness of leading vendors to disclose the attributes and provenance of the data they’ve used to train models needs to be urgently addressed,” said Ramayya Krishnan, dean of Carnegie Mellon University’s college of information systems and public policy.
Addressing problems with transparency of AI systems
Addressing the lack of transparency might look like standardized documentation outlining data sources and bias assessments, Krishnan said. That documentation could be verified by auditors and function “like a nutrition label” for users.
Witnesses from both private industry and human rights advocacy agreed legally binding guidelines – both for transparency and risk management – will be necessary.
Victoria Espinel, CEO of the Business Software Alliance, a trade group representing software companies, said the AI risk management framework developed in March by the National Institute of Standards and Technology was important, “but we do not think it is sufficient.”
“We think it would be best if legislation required companies in high-risk situations to be doing impact assessments and have internal risk management programs,” she said.
Those mandates – along with other transparency requirements discussed by the panel – should look different for companies that develop AI models and those that use them, and should only apply in the most high-risk applications, panelists said.
That last suggestion is in line with legislation being discussed in the European Union, which would apply differently depending on the assessed risk of a model’s use.
“High-risk” uses of AI, according to the witnesses, are situations in which an AI model is making consequential decisions, like in healthcare, hiring processes, and driving. Less consequential machine-learning models like those powering voice assistants and autocorrect would be subject to less government scrutiny under this framework.
Labeling AI-generated content
The panel also discussed the need to label AI-generated content.
“It is unreasonable to expect consumers to spot deceptive yet realistic imagery and voices,” said Sam Gregory, director of human right advocacy group WITNESS. “Guidance to look for a six fingered hand or spot virtual errors in a puffer jacket do not help in the long run.”
With elections in the U.S. approaching, panelists agreed mandating labels on AI-generated images and videos will be essential. They said those labels will have to be more comprehensive than visual watermarks, which can be easily removed, and might take the form of cryptographically bound metadata.
Labeling content as being AI-generated will also be important for developers, Krishnan noted, as generative AI models become much less effective when trained on writing or images made by other AIs.
Privacy around these content labels was a concern for panelists. Some protocols for verifying the origins of a piece of content with metadata require the personal information of human creators.
“This is absolutely critical,” said Gregory. “We have to start from the principle that these approaches do not oblige personal information or identity to be a part of them.”
Separately, the executive branch committee that met Tuesday was established under the National AI Initiative Act of 2020, is tasked with advising the president on AI-related matters. The NAIAC gathers representatives from the Departments of State, Defense, Energy and Commerce, together with the Attorney General, Director of National Intelligence, and Director of Science and Technology Policy.
-
Funding1 week ago
BEAD Director Says NTIA is Working on Changes to Letter of Credit
-
Community Broadband4 weeks ago
Rural Broadband Provider Touts Cooperative and Coalition-based Models
-
#broadbandlive3 weeks ago
Broadband Breakfast on Wednesday, November 1, 2023 – Broadband Deployment from India, Australia, South Africa
-
Funding4 weeks ago
A Deep Dive into the BEAD Program’s Matching Funds
-
Broadband Roundup4 weeks ago
NTIA Announces Middle Mile Funds, NDIA Director on Closing Digital Divide, More Tribal ACP Outreach Funds
-
Broadband Roundup4 weeks ago
FCC Waives Hurricane Idalia Rules, North Carolina Awards, Fiber Deployment in Kansas
-
Broadband Mapping & Data4 weeks ago
Broadband Breakfast Webinar on Broadband Geospatial Planning
-
Open Access3 weeks ago
Gigapower Exec Pitches Value of Open Access Networks to Maximize BEAD Money Efficiency