Researchers find that major foundation model providers can feasibly comply with the EU's draft AI Act, which would improve transparency in the entire ecosystem
Stanford researchers evaluate foundation model providers like OpenAI and Google for their compliance with proposed EU law on AI. Twitter: @dynamicwebpaige , @ampanmdagaba , @mario_gug , @alexcengler , @dwhitena , @zennerbxl , @rishibommasani , @heikirk , @responsableia , and @irenesolaiman Twitter: @dynamicwebpaige : “No foundation model provider achieves a perfect score, with ample room for improvement in most cases. Therefore, we consider whether it is currently feasible for organizations to fully comply with all requirements.” 👇Fascinating read, from @Stanford: https://crfm.stanford.edu/...? [image] Arseny Khakhalin / @ampanmdagaba : tldr: current LLMs don't comply with the new EU AI policy, but they are not far off; they can if they want to; and it will make AI landscape better. Super interesting! https://twitter.com/... Mario Guglielmetti / @mario_gug : “foundation model providers generally do not comply with draft requirements to describe the use of copyrighted training data, the hardware used and emissions produced in training, and how they evaluate and test models.” ⬇️ https://crfm.stanford.edu/... Alex Engler / @alexcengler : This is a good & valuable analysis of measures are being taken by large AI providers However, we should not confuse -what companies are doing now- with -what compliance with the AI Act means- The first may inform the second, but it is not the second https://crfm.stanford.edu/... [image] Daniel Whitenack / @dwhitena : 💡 Balancing innovation with compliance presents a unique challenge. Practitioners are trying to understand how our favorite models might fit into this new world of regulation. See this @StanfordHAI analysis: https://crfm.stanford.edu/... [image] Kai Zenner / @zennerbxl : Cheers for sharing @IreneSolaiman. I'm very surprised I need to say as I don't like all elements of Art 28b. HF did an excellent job it seems. Maybe your French colleagues can talk to @EmmanuelMacron & show him that ‘no-regulation’ is as flawed as the first French GPAI proposal? https://twitter.com/... Rishi / @rishibommasani : The blog post covers many more findings and recommendations, but what I want to highlight is everything is transparent: the requirements, grades, rubrics, and more. https://crfm.stanford.edu/... https://github.com/... Heiki Riesenkampf / @heikirk : I am very curious how many of the top LLM API providers will be bothered to comply with the current version of the AI Act vs just pulling out of the EU market. Current status here: https://crfm.stanford.edu/... @responsableia : Do Foundation Model Providers Comply with the EU AI Act? Stanford researchers evaluate foundation model providers like OpenAI and Google for their compliance with proposed EU law on AI. https://crfm.stanford.edu/... [image] @irenesolaiman : New work on foundation model provider compliance w the EU AI Act! A striking range: some score <25%(AI21, Aleph Alpha, Anthropic) one scores ≥75%(Hugging Face🤗) continually (fascinating+) stellar work @RishiBommasani @kevin_klyman @dzhang105 @percyliang https://crfm.stanford.edu/...