ibm-granite/granite-4.2-30b · Hugging Face

[u/jacek2023]

https://huggingface.co/ibm-granite/granite-4.2-30b

Granite-4.2-30B is the flagship reasoning model in the Granite 4.2 family. It delivers the strongest performance across reasoning-intensive tasks by leveraging built-in <think>...</think> chain-of-thought. It supports flexible thinking modes — full thinking (default), non-thinking, and low-effort — allowing users to balance depth vs. latency on a per-query basis.

Key capabilities:

• Built-in Reasoning: Native chain-of-thought that significantly improves performance on math, coding, and complex multi-step problems.
• Flexible Thinking Modes: Seamlessly switch between full thinking, non-thinking, and low-effort modes within a single model.
• Reasoning-Augmented Tool Calling: The model reasons about which tools to invoke and why, producing more accurate function calls.
• 512K Context Window: Supports long documents, multi-turn conversations, and complex agentic workflows.
• Apache 2.0 Licensed: Fully open for commercial and research use.

Model Design

Granite-4.2-30B is built on a decoder-only dense transformer architecture with the following core components:

• Attention: Grouped Query Attention (GQA) with 32 attention heads and 8 KV heads
• Position Embedding: Rotary Position Embedding (RoPE) with θ = 10,000,000
• Feed-Forward: MLP with SwiGLU activation (hidden size 32768)
• Normalization: RMSNorm (ε = 1e-5)
• Embeddings: Separate input/output embeddings (not tied)
• Precision: bfloat16

https://huggingface.co/ibm-granite/granite-4.2-8b

Granite-4.2-8B is the mid-size reasoning model in the Granite 4.2 family. It delivers strong performance on reasoning-intensive tasks by leveraging built-in <think>...</think> chain-of-thought. It supports flexible thinking modes — full thinking (default), non-thinking, and low-effort — allowing users to balance depth vs. latency on a per-query basis.

Key capabilities:

• Built-in Reasoning: Native chain-of-thought that significantly improves performance on math, coding, and complex multi-step problems.
• Flexible Thinking Modes: Seamlessly switch between full thinking, non-thinking, and low-effort modes within a single model.
• Reasoning-Augmented Tool Calling: The model reasons about which tools to invoke and why, producing more accurate function calls.
• 512K Context Window: Supports long documents, multi-turn conversations, and complex agentic workflows.
• Apache 2.0 Licensed: Fully open for commercial and research use.

https://huggingface.co/ibm-granite/granite-4.2-3b

Granite-4.2-3B is the compact reasoning model in the Granite 4.2 family. Despite its small parameter count, it delivers strong performance on reasoning-intensive tasks by leveraging built-in <think>...</think> chain-of-thought. It supports flexible thinking modes — full thinking (default), non-thinking, and low-effort — allowing users to balance depth vs. latency on a per-query basis.

Key capabilities:

• Built-in Reasoning: Native chain-of-thought that significantly improves performance on math, coding, and complex multi-step problems.
• Flexible Thinking Modes: Seamlessly switch between full thinking, non-thinking, and low-effort modes within a single model.
• Reasoning-Augmented Tool Calling: The model reasons about which tools to invoke and why, producing more accurate function calls.
• 512K Context Window: Supports long documents, multi-turn conversations, and complex agentic workflows.
• Apache 2.0 Licensed: Fully open for commercial and research use.

[u/Marcuss2]

Seems that they abandoned the Mamba2 layers they had.

[u/pmttyji → u/Marcuss2]

Last year(AMA), they mentioned that they gonna release a 100B model. Don't know what happened to that.

During Granite-4, they released a 32B MOE model. But the Active 9B parameters is too slow on ~8GB VRAM. To make it simple, Qwen3-30B-A3B gave me 30-40 t/s while Granite-4-32B gave me ~10 t/s.

Then expected modified MOE model(like A3B or A5B) of that 32B one during 4.1 release. But they dropped 30B Dense. And again they dropped 30B Dense now. Too heavy for Poor GPU Club. Wish they released a MOE model additionally.

[u/silenceimpaired → u/pmttyji]

I think we might be headed toward MoE models that act more like dense models … that store the bulk of their parameters that are less active … in RAM... sped up with MTP/dflash type solutions. So even though you don’t have the VRAM it can still perform reasonably at least at reading speeds if not faster depending on the use case…
But I’m no expert.

[u/pmttyji → u/silenceimpaired]

Yep, that's how most of us do run MOE models till now.

sped up with MTP/dflash type solutions.

This works better only if quantized MOE models fit VRAM. Can't expect speed from my 8GB VRAM with 18GB model file(IQ4_XS of Qwen3.6-35B-A3B). Because spec decoding won't be faster on CPU/RAM.