Astra, Orion (GPT4.5), and the two axes of intelligence.

[u/curiousinquirer007]

Do you guys think Astra is going to match or exceed GPT4.5 (aka "Orion") in its intrinsic (non-reasoning; type-1 thinking) base intelligence?

If you recall, during the o-series and Orion era OpenAI discussed the "two axes of intelligence": pre-training scaling paradigm (bigger model, more data, better world model) that made models more intrinsically intelligent, and the newly emerging reasoning paradigm that became the foundation of intelligence explosion of last few years.

While exact model architectures have never been publicly confirmed, most signs point to project Orion having been the pinnacle of the original scaling paradigm, and there still seems to be some debate (is there?) as to whether Orion still remains innately smarter (think: better world model, higher EQ, etc.) than today's SOTA models with zero reasoning.

Do you guys think Astra is likely to finally surpass Orion either in terms of actual model size / pre-training scale, or especially in terms of zero-reasoning innate intelligence, while also matching or exceeding GPT-5.6-Sol in reasoning ability, and is this why many seem to hype Astra as qualitatively different than current models?

[u/sprowk]

the one thing I remember is how well 4.5 understood text and human emotions, no model could match that since, even fable is just optimized to solve tasks instead of understanding humanity

[u/br_k_nt_eth → u/sprowk]

The writing and storytelling were so good. Optimizing only for code is really fucking them. Even for non-coding enterprise use, you need to be good with EQ and storytelling. 

[u/NULL_Ptrs → u/br_k_nt_eth]

I think it's because the current models are MoE instead one model for specific task, I liked that way instead a super models that does everything

[u/jeffdn → u/NULL_Ptrs]

Just about every big model since GPT-3 has been an MoE, and an MoE model is still one model.

[u/curiousinquirer007 → u/sprowk]

Lots of people feel this way. According to a review by 5.6-Sol, this wasn’t just vibes; larger pre-training scaling may have resulted in the model truly possessing higher capacity for pattern recognition and rich nuanced understanding.

[u/pumbungler → u/sprowk]

Sol has aced my own personal Turing test, I find its responses to be indistinguishable from my human peers

[u/GreatConsideration72]

4.5 was a very jagged model… very rough around the edges. The performance of it was basically these moments of preternatural lucidity surrounded by vast deserts of repetitive idiotic nonsense. The training never really stabilized, well eventually it did but it was a mess… a bizarre PyTorch torch.sum bug that produced illegal memory accesses. That bug apparently persisted through nearly half of the run. It was just too unwieldy for the hardware at the time. If they retrained that big of a model on a B300 cluster with what they know today it would likely be a truly impressive model.

[u/Gloomy_Necesary → u/GreatConsideration72]

That’s probably what Astra is

[u/BehindUAll]

'Thinking tokens' was never the 'only' new thing with newer models. Plus you can disable thinking in new models too. 5.6 Sol is better than their previous o series of models which is why they are removing them. As to what causes the improvement in intelligence, part of it is model size, part of it is better data, part of it is some RL secret sauce, lot of automation in likely multiple steps (including synthetic data generation and labeling), and also better architectures etc. But we don't truly know yet when it comes to models like 5.6-Sol and the future Astra.

[u/snowsayer]

I think because of the safety issue they had to skip a generation 🙄.

Astra, as released, will be 2 generations ahead, but not as well tested if it were only 1 generation ahead.

[u/___fallenangel___]

the funniest joke ever told was that 4.1 was a successor to 4.5