Research: AI's ability to complete lengthy software engineering tasks has doubled roughly every six months, but there is a “messiness tax” for real-world tasks
METR has had a very influential work by Kwa and West et al on measuring AI's ability to complete long tasks. X: @kirillzzy , @boazbaraktcs , @benshindel , @jasonfurman , @jasonfurman , and @sama X: Kirill Avery / @kirillzzy : if the bots are going to be doing human jobs, then the humans better be getting paid some other way 🤔 Boaz Barak / @boazbaraktcs : Wrote a blog post with my thoughts on how AI could impact economic growth. [image] Ben / @benshindel : I realize this view of history is extremely common these days, but I think ppl underestimate the technological and cultural progress between 100k ya and the rise of cities 5k ya, and again from there to the height of the Bronze Age, and then to the classical era, and so forth. [image] Jason Furman / @jasonfurman : I'm thinking of the engineering studies about ways that households could weatherize their homes, reduce electricity bills & help climate. Then economists study actual people and they did everything so imperfectly the actual results were much more disappointing. Is AI like that? Jason Furman / @jasonfurman : Fascinating on the economic implications of AI from @boazbaraktcs. Boaz is more optimistic than me. But also vastly more knowledgable about AI than me. One issue I wonder about is the engineer (studying how it works in a lab) vs. the economist (studying people in the wild). [image] Sam Altman / @sama : interesting post from @boazbaraktcs: https://windowsontheory.org/ ...