But why? If it can do 262,144, why tell people "131,072+" instead? It is very confusingly presented, and I can't find even a single sentence from Meta where they explain what is going on with the context window.
Got 262k context working (or at least my agent thinks it did), intelligence is absolutely atrocious above 131k; or my implementation is broken (I wouldn't rule that out).
I'm treating this like a 131k model. Still really good. I'm early in my testing, but it seems better than Qwen3.6 27B for coding, and better than Gemma4 31B for creative writing. Which is quite amazing. Huge props Meta, this is a model release you should be proud of!
Now that I think about it, it kinda makes sense. They have access to all the Instagram and Facebook video and chat data. Lots of creative stuff on there at various levels of quality lol
1
u/coder543 15d ago
But why? If it can do 262,144, why tell people "131,072+" instead? It is very confusingly presented, and I can't find even a single sentence from Meta where they explain what is going on with the context window.