Active vs effective parameters

Two different reasons a model's headline size can mislead — one about compute, one about memory.

A model's total parameter count is every stored weight: its capacity, and roughly its file size. For a plain dense model that single number also tells you the compute and the memory. But two modern tricks make the headline number diverge from what a model actually costs to run — and they pull in different directions, which is why "active" and "effective" are easy to confuse.

Active parameters are the ones that actually do math for each token. In a dense model active = total. In a Mixture-of-Experts model the router runs only a few experts per token, so active < total — this is the "A" in names like Qwen3-30B-A3B (30.5B total, 3.3B active) or Gemma-4 26B-A4B (25.2B total, 3.8B active). Active is a compute number: it predicts speed, not memory. You still have to store every expert, so an "A3B" model is no lighter in RAM than its 30B total implies. (See Dense vs MoE.)

The 'A' axis: as experts grow, total (amber) is what sits in memory while active (teal) is what computes per token. This shows active vs total only — the 'effective' (memory-offloading) axis below is a different idea.

Effective parameters describe the memory footprint instead. Some models hold more raw weights than they keep in working memory: Google's Gemma "E" models (introduced with Gemma 3n, continued as E2B/E4B in Gemma 4) use per-layer-embedding offloading — large lookup tables cached to fast storage, outside the operating memory — plus MatFormer nesting, where a smaller model lives inside a larger one. Gemma 3n's E2B loads over 5 billion raw parameters but runs with an effective memory load of about 1.9 billion; the "E" is what it behaves like in RAM, not what it stores. Effective is a memory number — and these models are dense, so it has nothing to do with sparsity.

So the two sit on different axes and must not be conflated. Active answers "how much computes per token?" (the dense-vs-MoE axis); effective answers "how much must sit in memory?" (the offloading axis). A 30B-A3B model computes like 3B but still weighs 30B in memory; an E4B model weighs ~4B in memory but computes densely. A small dense model that offloads nothing has its total, active, and effective counts all equal — the distinctions only appear once a model scales up.

Related: Dense vs Mixture-of-Experts · Parameters · Weights