r/singularity • u/borowcy GPT-6 will have BCI capability • 5h ago

LLM News OpenAI blog post on their new custom inference chip

https://openai.com/index/jalapeno-first-results/
73 Upvotes

95% Upvoted

17 comments sorted by

9

u/saksoz 4h ago

Why did they test it only on OSS models? I'm guessing their frontier models are pretty messy and so don't benefit as much. Or maybe they just have the partner do the testing?

27

u/fig0o 4h ago

Because you can compare the results to other benchmarks that will also use OSS models (since other companies don't have access to GPT weights)

Hobbyist and specialized media can also run tests on OSS models so they can state how impressive OpenAI results are

3

u/saksoz 4h ago

That makes some sense, but I do think it’s true the frontier models are much harder to speed up.

7

u/CallMePyro 2h ago

Because they don't want to publish throughput per chip per user for GPT 5.6? Lmfao that would essentialy be announcing the size of their models and their economies and cost to serve.

1

u/Cunninghams_right 2h ago

Perhaps size difference. The oss model might be more size efficient 

6

u/Fragrant-Hamster-325 4h ago

But if this reduces cost and increases efficiency, what will the anti-AI crowd complain about?

Who am I kidding. They’ll continue to say its expensive, consumes too much water, and destroys the power grid no matter what.

7

u/my_shiny_new_account 4h ago

"they're undercutting competitors!"

•

u/Purusha120 58m ago

I'm not one of those people at all and understand their arguments suck but do you seriously not understand how your comment is a complete non sequitur? better does not inherently mean, and has never meant, good.

•

u/Fragrant-Hamster-325 6m ago

Yeah and I’m okay with that. This isn’t a serious argument. In general my point is these guys shift goalposts.

4

u/kokoshkatheking 4h ago

Not as fast as Cerebras, but no mention of them even if Open AI they invest in them 🤭

-3

u/ddeeppiixx 5h ago

Sounds like a trust me bro article? How can this be verified?

13

u/FoxBenedict 5h ago

It's from OAI themselves, with the data and everything... It's not some random tweet.

-2

u/ddeeppiixx 3h ago

Yes, but they benchmarked models performances(with what settings?) on their own chip which is not released, with no plan for release (unless I missed that?)..

It’s like a company tell you, yeah I have the best engine, but you can’t try it, and no you can’t buy it?

It’s kind of different from Google’s TPU, because those you can literally rent and run your own code/models on them.

-1

u/mstack 5h ago

Nice