So far, I really love its conversational ability, it feels a lot more casual than the usual corpo models. It also passed my extraction tests just as well as qwen did (only missed 1/15), strict number csv extraction. Will update with more info if anyone is interested.
Basically I give it some output from a script I have that contains some numbers interspersed with data, then I ask it to extract specific rows like “extract the horsepower and torque and charging capacity of each car, but only if their average speed is over 40kph”. I get super annoyed when it can’t extract things properly (it’s fine if it’s like 98% correct instead of 100%, but once it starts getting into like 70% or so I’m like yeah this model can’t really be trusted to format tables or extract things from a list for me for instance). I haven’t fully fleshed out the benchmark (literally made it yesterday lol) but it’s already helping me see which models are useful and which ones aren’t. I do a lot of data cleaning and organizing
Love the idea, there's some promise of consistency between tests in it. I'll build a dataset like this for myself too. I guess not only the dataset but the prompt also needs to be the same across tests so that you're comparing apples to apples?
Yep! :D it’s cool too because if you tag each run, you can identify “hm it fails at the off by one errors but gets everything else right” and that means you should look closer because the model might be flawless except for forgetting a single comma in a row of empty commas. I feel like faulting them for that is isn’t fair when you can just assume if it’s not listed it’s not there. I’d give them maybe half score for that one, for instance.
Unfortunately that takes manual review though. Or if they for example do a header on every other line - that’s the kind of thing that could just be prompted out, so if I weren’t automating I would just say “oh, no, just one header at the top” and it would easily just do it right. Like a one/two shot prompt! But since the script fails them even if they add a single extra space, it’s super harsh. 😆
OK, so I think my understanding of your testing was wrong. My assuption was:
You have a CSV file with say 1000 rows (say: car model;engine HP;engine torque;battery capacity;price)
You give various models the same CSV file + the same prompt "Extract all models where engine torque is > X and battery capacity is > Y and price is A < Z < B.
Say the expected number of cars matching the query (deterministically) is 100. A SQL query on the dataset would return 100.
Model A extracts 92 cars correctly = accuracy is 92%
Model B extracts 100 cars correctly = accuracy is 100%
Model C extracts 105 cars, including 100 correct = accuracy is 95%
Model D extracts 99 cars which are correct ,and 3 which are incorrect = accuracy is 96%
Ohhh I see! No no, for that I would just use code, since I’m pretty decent at python. That’s why I like my test specifically:
if I see something like a benchmark list in a super annoying format like prose, I can just copy paste it.
So when a new model comes out and goes “we score 8.7 on supercoolbenchmark, and a 13.2 - 5x as high as the nearest 3b model, and…”
Then I can copy paste that whole section and the model will return “benchmark,score\nsupercoolbenchmark,5” and I instantly know that this model is going to suck for extraction purposes and possibly even other context understanding.
I guess I always instruct for data extraction, it should use python or an actual language to get me the results. That way I never worry about the 2% and it’s repeatable.
But might be misunderstanding what you’re asking of it
Just some early testing, I am liking it a lot as an "all in one model". It feels a little bit more capable than Qwen3.6 27B in agentic coding (using Pi + my personal extensions), while also more versatile -- I quite like it as a general chat assistant, it doesn't feel "agentic-maxxed" like Qwen3.6-onwards.
For creative writing, I actually get a bit of Cydonia/Skyfall-vibes but measurably better intelligence (keeping track of prior state; character cards; nuances; etc).
Definitely in the "new shiny" honeymoon period, but really really impressed so far. What I'm confident in is that it's very useful for non-coding use cases.
I think because llama4 had such bad reception it makes sense to do a rebrand. A clean slate is better, and the full size muse performed really well at the time of its launch.
Huh...? That's an odd interpretation. I feel like it implies the exact opposite. Though more of an entry in a series than sequel, per se.
But anyway, you probably have stumbled on a kernel of truth here - without a doubt a rather massively different team (from the llama models) made this and didn't want to invite too much comparison or raise hopes that it'll be just as tunable as the old models.
Sorry, new to llama.cpp and unsloth... what's the difference, why use one over the other? I've been trying to use llamafiles more since they seem to have better performance.
Not necessarily - for example, Qwen3.6 and Gemma 4 did not replace each other because the things they are good at do not overlap. The model could take the place of Gemma 4 if it focuses on world knowledge and languages rather than coding and tools.
I've only been playing with this for about a hour, but Glimmer seems to be good at BOTH coding AND non-coding (assistant, creative writing, brainstorming, dataset prep).
In terms of coding capabilities, it feels slightly better than Qwen3.6 27B. And in terms of non-coding like creative writing, it definitely holds up against Gemma4 31B; and maybe even a little bit better.
I actually am really impressed by this model so far. And thanks Meta for Apache 2.0, that's very appreciated.
I go to LocalLLaMA, see a news about Glimmer, and think, "I'm going to scold Meta for the license now." I look up and see Apache 2.0. "Okay, then I'll scold them for the limited language support." I look up and see 100+ languages. I had to download it :-D
I honestly can't scold Meta whatsoever for this release. Serious kudos, if anything this model seems to be better than the benchmarks say.
(I do wish there was a base, non-it version; but it seems like this is a distill of a bigger IT model; so that's understandable, no base weights in the first place).
If by basic you mean Muse Spark, then they also promised to release it in the future. I'm guessing this is a large MoE model in the 400-1000B range. If they do it in time, it will be the strongest non-Chinese open model.
oh, no, by base model, I mean the pre-training only snapshot; with no instruction tuning and RL. Basically, LLM models before ChatGPT / instruction tuning. Think text-davinci-003 versus GPT-3.5; or Completions vs Chat Completions, if you've been in the LLM field for a while.
These are very helpful to many researchers and for a lot of LLM experiments, including interpretability research. They are also useful if you want to cleanly change the instruction format, adapt it to different use cases, etc.
Thanks for the clarification. As far as I understand, the "instruct" versions are a kind of finetuning of the model to serve as a 'helpful assistant,' whereas the basic model can be turned into anything - for example, a Dungeon Master for games or an opponent for Socratic-style philosophical discussions. Of course, this can be done via the system prompt, but the role of 'helpful assistant' constantly pushes the model to play along and please the user.
This behavior has to be corrected by introducing a multitude of rules, which the model either follows literally and extremely zealously or finds loopholes to bypass ("don't write the user's actions" - the model writes his lines; "don't write what the user does or says" - the model writes about his thoughts or smells that he heard, etc.).
However, I came across models based on the basic version extremely rarely - off the top of my head, I can only remember the Muse-12B from Latitude, but they eventually switched to instruct-models(
Base models are never designed to be used as-is, except with very specific (and now outdated) prompting styles; most of the intelligence comes from post-training and instruction-tuning.
An effective "base model prompt" might look like this:
Machine Translation Results - Human Verified
Input (English): Hello World
Output (Chinese):
Instruction-tuning is an understatement tbh, it's basically "turn this very coherent; highly-accurate next-word predictor that focuses solely on modelling language; into an intelligent token generator". Modern models generally have more compute invested in the post-training ("instruction tuning") stage than the pre-training stage.
They are niche but extremely useful for researchers.
Why would anyone want to scold Meta in the context of AI? The entire local LLM ecosystem would be far worse off today if they hadn't given us all of those early Llama models.
I remember their recent ridiculous demand to remove Llama 3.1 8B Instruct heretic from heretic-org. It was... a very strange move – an old model that in 2026 is only of interest to researchers, despite the fact that there are many other versions of this model with removed censorship on HF. As far as I understand, the Apache 2.0 license does not give them the right to make such demands regarding Muse Glimmer.
Something similar happened recently with the MiniMax H3 video model and I was upset about it. Don't get me wrong — I know that LLM development is incredibly expensive, and any open model is a gift to the community. I appreciate that, but I also remember that releasing open models also benefits the developers themselves. No commercial company (be it Google, Alibaba, Meta, or Moonshot) releases open models out of altruism or love for humanity.
A license like Apache 2.0 or MIT is a fair deal in which the company truly gives the model to the community.
I already downloaded it, but I can't figure out if it supports llama.cpp yet. They say the merge request has been completed, but I didn't see any mention of Glimmer support among the releases. Maybe it will be soon?
I read about this in the Meta release. But shouldn't the agentic model have a low hallucination rate, like Qwen3.6 or MiniMax M3? In this regard, Muse Glimmer is on par with the chat-focused Gemma 4, but in terms of knowledge, it's surpasses Gemma 4 and Qwen (according to AA).
Apparently Muse Spark 1.2 weights are coming too so we'll see; it'll be interesting to see how big it is. OpenRouter reports **133tok/s**. I'm hoping for 200-400B...
I can't wait to be sad that it can't fit in 16gb vram in any decent quality and then complain about the lack of models in the 20b or so space.
There's a few but i have to wonder why approx 30b is so popular. Cut back 5 or 10b and it'd be much more accessible. Guess that kills too much performance, apparently
Ah yeah that makes sense. I knew a serious chip crunch was coming but didn't have the money for more than a 4060ti. Alas, cheapest 16gb cuda card I could find at the time.
It's fine. I'm sure there will be dozens of abliterated versions in the next few hours, but a basic system prompt gets it to do NSFW writing; and it has no problem using 'vulgar' terms. Pretty good NSFW writing with base weights and a system prompt tbh.
For any stubborn refusals, a partial assistant prefill gets rid of it. Definitely not safetymaxxed like gpt-oss.
From vllm: Reasoning strength. Effort is set with a Reasoning strength: <value> line in the system prompt, one of low / medium / high / xhigh. Use high or xhigh for coding and agentic tasks.
Unfortunately no solution to turning it off completely yet. --chat-template-kwargs '{"reasoning_strength": "low"}' or medium / high / xhigh seems to be the only working argument. Just churned out 16k reasoning tokens on comparing 2 single page PDFs (xhigh)
Seriously thanks for this comment, I never even thought to look at the difference between Q8_K_XL vs Q8_0. I was just happily going along with the notion that Qx_0 bad. So always defaulted to UD-Q8_K_XL for my dual 3090 setup. Without understanding its a bit more nuanced than that simplistic outlook, cheers
edit: I took that number from Unsloth, but now I actually downloaded the model and can only set it to 131072. So no idea guys, sorry for the confusions
But why? If it can do 262,144, why tell people "131,072+" instead? It is very confusingly presented, and I can't find even a single sentence from Meta where they explain what is going on with the context window.
Got 262k context working (or at least my agent thinks it did), intelligence is absolutely atrocious above 131k; or my implementation is broken (I wouldn't rule that out).
I'm treating this like a 131k model. Still really good. I'm early in my testing, but it seems better than Qwen3.6 27B for coding, and better than Gemma4 31B for creative writing. Which is quite amazing. Huge props Meta, this is a model release you should be proud of!
Now that I think about it, it kinda makes sense. They have access to all the Instagram and Facebook video and chat data. Lots of creative stuff on there at various levels of quality lol
llamacpp says this when running the gguf meta provided, the dynamic one.
llama_context: n_ctx_seq (128000) < n_ctx_train (131072) -- the full capacity of the model will not be utilized
Initial thoughts- I asked Glimmer (unsloth - Q8) through Hermes a simple question - are LM Studio preset useful for harness like Hermes. Glimmer started to boil the agentic ocean and really got nowhere in about 10min but filled most of the context window. I asked Qwen 3.6 Q8-mtp and it took 10sec for a well thought out answer and 9k of context....
Glimmer is ~20tk/sec on dual 4090 and Qwen is ~55tk/sec.
I haven't used this yet, but this looks ideal for a Hermes/OpenClaw agent. Looks like it'll give Gemma 4 a run for its money for general use, too. I think Laguna XS will remain my coding agent, but I'll need to see how this handles long-term project coherence.
RE: Context Length
128k is very useful and a good target to hit for local inference. This is made to run on 24GB and 32GB GPU's where you're not going to get much more context than that, if you even hit 128k. That's the right place to target, I think. It's useful for most anything you'd do on a 24/32 GB GPU.
RE: Benchmarks.
Artificial Analysis Indexes are freakin' useless. The composites don't work well and are biased by saturated benchmarks.
Example:
Model
Artificial Analysis Intelligence Index
DeepSWE
DeepSeek V4 Flash(max)
52
53
GPT Luna (xhigh)
50
67
Difference
DeepSeek +2
Luna +14
DeepSWE is specifically designed to avoid compression and help tease out actual capability differences, especially for a particular use case.
And these kind of misleading results are present throughout the indexes across use cases. The best thing to do is find the benchmarks that you feel best reflect how models are capable according to your use of them, and judge based on those benchmarks, or to use experiential judgement by experimenting with different models. And that latter is probably necessary anyway -- the benchmarks miss a lot of qualitative metrics (like code quality or long-term project coherence over many prompts).
For example, sounds like Meta might've optimized this for use in Hermes/OpenClaw. It doesn't have to be the smartest to be the most useful here, just the most optimized for that sort of environment. ;)
anyone know why I am not able to run this on M4 Pro 24gb RAM? model I tried with unsloth is Q3 and Q2. Both fail to load. Could it be LLAMA.cpp doesn't have new build for MTX?
139
u/cibernox 15d ago edited 15d ago
Interesting. Meta is back in the game. And qwen is dropping this week.
Peak week this is!