Six AI and ML experts detail what they learned from building real-world applications on top of LLMs over the past year, including common prompting pitfalls
‘What We Learned from a Year of Building with LLMs (Part I)’ — https://www.oreilly.com/... Hallucinations are still a big problem; RAGs are better than fine-tuning; long context models don't help much; overemphasising some benchmarks hurts your overall performance X: Dr. Donut / @bebischof : Proud to bring you: A Year Building With LLMs, a three part essay published with O'Reilly https://www.oreilly.com/... We get into the weeds on what it takes develop incredible LLM powered applications Advice from: @eugeneyan, @charles_irl, @HamelHusain, @jxnlco, @sh_reya and I. Andrew Reed / @andrewrreed : What We Learned from a Year of Building with LLMs (Part I) Great read with many practical tips for anyone applying LLMs to real-world use cases 👏Kudos @eugeneyan @BEBischof @HamelHusain @jxnlco @sh_reya @charles_irl 🔗 https://www.oreilly.com/... [image] @charles_irl : My personal favorite in this section: “the rumors of RAG's demise are greatly exaggerated.” [image] Dan Becker / @dan_s_becker : Writing about LLMs has so much hype and academic stuff with little actionable insight. This article is clear, real and practical. It's a great read https://www.oreilly.com/... Arjun Ram / @arjunram : Fantastic write up on building with LLMs https://www.oreilly.com/... - I found myself nodding with all the hard learned lessons. Gets as detailed for building. Bravo 👏🏼 👏🏼 @eugeneyan @BEBischof @sh_reya @charles_irl @jxnlco @HamelHusain Matt Popovich / @mpopv : Everyone is a manager now, so make sure you know how to manage [image] Hamel Husain / @hamelhusain : My colleagues and I distilled practical advice re: LLMs into this three-part series. Lot's of bangers. One of my favorite excerpts from this part in the screenshot Advice from: @eugeneyan, @BEBischof, @charles_irl, @sh_reya, @jxnlco and myself See: https://www.oreilly.com/... [image] @beyang : Very nice survey of LLM best practices from practitioners building for real-world use cases. They even have a nice quote from yours truly on retrieval methods: https://www.oreilly.com/... [image] @charles_irl : In this post, @eugeneyan, @BEBischof, @HamelHusain, @jxnlco, @sh_reya & I share our tactical tips for working with LLMs, from structured outputs to caching Stay tuned for two more posts covering the operational (hiring, product) & strategic (durability, competition) perspectives [image] @charles_irl : dear diary: today @simonw called me an “experienced LLM hacker”. having pinched myself, i am now en route to purchase a lottery ticket. https://simonwillison.net/... [image] Ethan Mollick / @emollick : This is an interesting overview of what it is like to actually build production software with today's LLMs. It also shows how weird LLMs are to work with from a software perspective & how much we have to learn. It is why I stress co-intelligence for now. https://www.oreilly.com/... Shreya Shankar / @sh_reya : i'm having a super fun time collaborating with @eugeneyan, @BEBischof, @charles_irl, @HamelHusain, & @jxnlco on a 3-part series on working with LLMs. i learned so much from them that i really think it's the best resource on applied LLMs. here's part 1: https://www.oreilly.com/... Dr. Donut / @bebischof : I'm partial to this section, h/t to @willkurt with whom I chatted about this framing a lot: [image] Forums: Hacker News : What We Learned from a Year of Building with LLMs r/LocalLLaMA : What We Learned from a Year of Building with LLMs