A look at the challenges of academic AI research, as costs to develop generative AI models grow rapidly and tech companies' salaries drain academia of talent
e.g., establishing shared computing resources. https://www.washingtonpost.com/ ... Glenn K. Lockwood / @glennklockwood : Academia is losing its ability to make progress in AI due to competition with industry investment. 1. Is this a problem any more than others areas of R&D with commercial relevance? 2. Are salary and GPUs really the only reason? https://www.washingtonpost.com/ ... Nathan Lambert / @natolambert : Not much new information in this piece, but I appreciate that the gap between industry and academia for AI is becoming such a public issue. Thanks @drfeifei for working on this! [image] Vijar Kohli / @vijarkohli : The GPU moat is real. Nvidia chips have so much demand that they cannot fulfill all customers orders in real-time. We've spent years reducing the cost of compute for software. But now costs are rising and becoming a competitive advantage for the application layer. $NVDA Meredith Whittaker / @mer__edith : Notable that this longstanding problem, which I and a few others have been naming for ~a decade, is now common sense. It's true. AI is fundamentally a technology controlled by Big Tech. But the current ‘solutions’ to this problem would extend, not dilute, Big Tech control. 1/ Meredith Whittaker / @mer__edith : The issue: Big Tech has the $$ infrastructure, data, ability to pay talent, and access to market which no one else does. So as academics, you either pay retail for access, or get it discounted/free by yoking yourself to Big Tech (via dual affiliation, or just being hired). 2/ Meredith Whittaker / @mer__edith : IRL, no academic can afford to pay retail ($100b training runs, y'all). So, academic labs vie for access/proximity to Big Tech infra in pursuit of doing ‘relevant’ research—something that should alarm fans of academic neutrality/those concerned w conflict of interest. 3/ Meredith Whittaker / @mer__edith : For more, a paper I wrote a few years ago: https://papers.ssrn.com/... 8/ [image] Dare Obasanjo / @carnage4life : “I tried to hire a very senior researcher from Meta. They said, come back to me when you have 10,000 H100 GPUs. That would cost billions and take 5-10 years to get from Nvidia” - @AravSrinivas, CEO of @perplexity_ai Outside of chips, AI is a rare tech trend favoring incumbents. LinkedIn: Prof. Dr. Patrick Glauner : Academics can do consulting on the side or found a startup. There're many opportunities for those academics who could earn that well if they worked in industry fulltime. … Catherine Breslin : Over the past decade, AI research has moved from academia to industry, as per Stanford's AI Index report of 2023. … Stuart Shulman : This is a fascinating take on an important set of questions. How do we keep public problems in focus and academic institutions staffed if the best talent is leaving academia for big tech firms? Russell Wald : The Stanford Institute for Human-Centered Artificial Intelligence (HAI) has been ringing the alarm bells since 2019 on the lack of access and opportunity for academic AI research. …
We'd do well to reduce the number of researchers across the board, tho. The extreme over-investment in ML research is real, it's wasting so much money and so much compute, and mostly for naught. We're seeing all these marginal improvements, 400+ activations, and the spill out e…
The GPU moat is real. Nvidia chips have so much demand that they cannot fulfill all customers orders in real-time. We've spent years reducing the cost of compute for software. But now costs are rising and becoming a competitive advantage for the application layer. $NVDA
Notable that this longstanding problem, which I and a few others have been naming for ~a decade, is now common sense. It's true. AI is fundamentally a technology controlled by Big Tech. But the current ‘solutions’ to this problem would extend, not dilute, Big Tech control. 1/
Despite having almost no money, academics still developed (just since 2020) diffusion models, FlashAttention, prefix tuning, DPO, essentially every neural IR model, many of the methods for long contexts, and the majority of the important benchmarks, among many other things.
Commercialization of research, and walled gardens is killing academic research and collaboration. All in name of revenue/profit without giving back to general education.
We have been sounding the alarm since 2019 @StanfordHAI and my colleague @drfeifei has been the primary champion to alert policymakers on the lack of access for academic AI research. This WaPo story summarizes the challenge well. https://www.washingtonpost.com/ ...
The issue: Big Tech has the $$ infrastructure, data, ability to pay talent, and access to market which no one else does. So as academics, you either pay retail for access, or get it discounted/free by yoking yourself to Big Tech (via dual affiliation, or just being hired). 2/
Interesting article on academia's current inability to compete with the private industry on AI. Also discuses proposals to increase competitiveness via large scale collaboration — e.g., establishing shared computing resources. https://www.washingtonpost.com/ ...
Academia is losing its ability to make progress in AI due to competition with industry investment. 1. Is this a problem any more than others areas of R&D with commercial relevance? 2. Are salary and GPUs really the only reason? https://www.washingtonpost.com/ ...
Not much new information in this piece, but I appreciate that the gap between industry and academia for AI is becoming such a public issue. Thanks @drfeifei for working on this! [image]
IRL, no academic can afford to pay retail ($100b training runs, y'all). So, academic labs vie for access/proximity to Big Tech infra in pursuit of doing ‘relevant’ research—something that should alarm fans of academic neutrality/those concerned w conflict of interest. 3/
“I tried to hire a very senior researcher from Meta. They said, come back to me when you have 10,000 H100 GPUs. That would cost billions and take 5-10 years to get from Nvidia” - @AravSrinivas, CEO of @perplexity_ai Outside of chips, AI is a rare tech trend favoring incumbents.