Source: OpenAI demoed a breakthrough called Strawberry to US natsec officials; one of its top uses is to make training data for a flagship LLM codenamed Orion
In case you were wondering why Sam Altman cryptically posted a picture of strawberries earlier this month, the answer almost certainly …
LLMs are so done. The “big breakthrough” on this model is generating training data to fill the infinitely widening gap between ALL THE INFORMATION EVER PRODUCED IN THE WORLD and what's required simply to make a model that can do basic math and solve the NYT Spelling Bee. [embed…
ngl it's weird that this is called this because if you search “openai strawberry” right now, you'll find a bunch of people arguing with chatgpt about how many “r"s are in the word strawberry [embedded post]
Putting this (pay-walled) story together with an earlier Reuters story, it's sounding a lot like the goal here is synthetic data for better fine-tuning of models on “long-horizon tasks.” — People are not factoring in the likelihood of a change in this direction. www.reuters.co…
You do get the feeling that implicitly or explicitly, the US nat sec blob has told OpenAI and others that they need the read in on everything well before its launched so that they can decide how to react well in advance There's a reason we don't have personal AI agents yet
Interesting to consider that despite OpenAI having raised $13B (most of it from Microsoft), it still needs to raise more money to keep operating, thanks to its large losses. Consider that Uber raised $13B before going public (then $8B during its IPO.)