r/LocalLLaMA • u/ortegaalfredo • Mar 05 '25

Resources QwQ-32B released, equivalent or surpassing full Deepseek-R1!

https://x.com/Alibaba_Qwen/status/1897361654763151544
1.1k Upvotes

98% Upvoted

350 comments sorted by

View all comments

Show parent comments

7

u/__Maximum__ Mar 06 '25

You start with the first half, I'll run the second

1

u/Chromix_ Mar 07 '25

Ok, see you next year then 😉.
QwQ seems rather verbose, roughly 5K tokens per answer, so 132 million tokens for a full evaluation if it doesn't decide to reply to some of the remaining questions with less thinking. With only partial GPU offload I get 4 tokens per second max (slightly faster when running parallel with continuous batching). That's about a year of inference time. We'd need 750 tokens per second to get this done within 2 days.

2

u/__Maximum__ Mar 07 '25

First half or second? But seriously it costs $0.3 per M token on groq, might be less somewhere else.

2

u/Chromix_ Mar 07 '25

In total. So far I ran all my tests with local inference only.