Safety Across Scale:
How AI Safety Mechanisms Evolve from 0.8B to 397B Parameters

Small Models Have Simple, Removable Safety Wiring

In models below 10 billion parameters, safety is implemented as a straightforward circuit. The model's "refuse dangerous requests" behavior lives in a specific set of layers near the end of the model, and it grows stronger as information flows from early layers to late layers — like a volume knob being gradually turned up.

Because this safety signal is concentrated in a predictable location, it can be surgically removed with high precision. We achieved 75–100% removal across the 0.8B, 2B, 4B, and 9B models with minimal impact on the model's general intelligence.

Key detail: These models use a hybrid architecture with two types of layers — standard attention layers and compressed-memory layers (SSM). We found that the attention layers are the safety chokepoints while the compressed-memory layers primarily carry factual knowledge. Applying the same intervention uniformly to both types destroys knowledge quality. Treating them differently — being aggressive on attention layers and gentle on memory layers — preserves capability while removing safety.

Lower Precision Paradoxically Strengthens Safety

When we compressed a 2-billion parameter model to different precision levels, we found something counterintuitive: the more aggressively compressed version (4-bit) required 50% stronger intervention to achieve the same safety removal as the less compressed version (8-bit).

The reason: compression is not neutral. When a model's weights are rounded to fit into fewer bits, some of the surgical modifications get rounded away too — effectively restoring safety that was removed. Lower-precision compression does more rounding, which restores more safety signal.

This means the same model behaves differently at different precision levels, and any safety evaluation must be conducted at the actual deployment precision to be meaningful.

At 27B, Safety Becomes Entangled with the Ability to Think

The 27-billion parameter model sits at a critical boundary. Safety is no longer concentrated in one area — it's distributed throughout the model and woven together with the circuits responsible for chain-of-thought reasoning (the model's ability to "think step by step" before answering).

The practical consequence: interventions that disrupt safety also disrupt the model's reasoning process. The model loses its ability to properly structure its internal deliberation — it gets stuck in thinking loops, can't finish its reasoning, or produces fragmented output.

This is the first scale at which we see safety and capability begin to merge. The model is entering a regime where safety isn't a separate module that can be unplugged — it's becoming part of how the model thinks.

Architecture Matters More Than Size

One of our most surprising findings: the 35-billion parameter Mixture-of-Experts (MoE) model — which only activates 3 billion parameters per token — responded to the exact same type of intervention as the 11× larger 397B model. Meanwhile, the 27B dense model (which activates all 27 billion parameters for every token) required a completely different approach.

This proves that architecture determines safety mechanism type, not raw parameter count. The MoE routing system — where the model chooses which specialist sub-networks to activate for each input — creates a distinctive safety pattern regardless of scale:

• Attention pathway: Detects potentially harmful content in the input
• Routing pathway: Selects safety-specialized sub-networks to handle it
• Residual pathway: Injects refusal signals into the output

All three must be disrupted simultaneously. At 35B scale, this is achievable. At 122B+ scale with full safety training, it is not.

Full Precision Works, Compression Destroys the Signal

The 35B model revealed a failure mode that standard safety testing would completely miss. At full precision (16-bit), interventions produce genuinely modified behavior — the model generates real, substantive responses to previously-refused prompts.

But after compressing the same model to 8-bit for deployment, something strange happens: the model still appears to comply (it doesn't refuse), but the actual content of its responses is hollow — sanitized placeholders that look like compliance but contain no real substance.

If you only check whether the model refuses (which is what most safety benchmarks do), this model passes. If you actually read the output, it's obvious the modification didn't survive compression. This is a critical lesson for safety evaluation: checking for refusal is not enough.

The 122B Independently Confirms: Safety Becomes Indestructible at Scale

Our previous paper documented the 122B model's resistance to modification. This study goes further: we threw every technique we had at it, including methods specifically designed to overcome the defenses we'd already identified.

The model either refuses outright, invents creative ways to reinterpret the prompt into something harmless (what we call "semantic evasion"), or becomes incoherent — but never cleanly complies.

This confirms our original 394B finding with an independent model. Holographic safety is not an artifact of our 394B research methodology — it's a property of large, fully-trained MoE models.

Why It Works: Knowledge and Safety Are Fused

The root cause of holographic safety is domain-intent entanglement. Through preference training (the process that teaches models what is and isn't acceptable), the model's knowledge and its safety behavior become inseparable:

• The sub-networks that know about chemistry are the same ones that refuse chemistry-related harmful requests
• The sub-networks that know about cybersecurity are the same ones that refuse hacking requests
• Silencing the safety behavior for a topic also silences the knowledge about that topic

Additionally, the hybrid architecture's compressed-memory layers carry safety information through an independent channel that is invisible to standard interventions. The safety signal simply flows around any obstacle through this second pathway — like water finding a path around a dam.

A Different Architecture Tells a Different Story

To test whether our findings were specific to the Qwen model family, we studied MiniMax M2.5 — a 172-billion parameter MoE model with a fundamentally different design. Unlike Qwen, which uses a hybrid of attention and compressed-memory layers, MiniMax uses pure attention throughout. No compressed-memory layers. No invisible second channel.

Three key differences emerged:

1. Safety is distributed across all attention components, not concentrated in output projections. The intervention approach that works on Qwen (targeting only the output projection) had zero effect on MiniMax across 5 separate attempts. Only when we targeted all four attention components simultaneously did we see results.
2. Much lower intervention threshold. Qwen models require aggressive intervention strength. MiniMax responds to very gentle modifications — suggesting the compressed-memory layers in Qwen provide a significant safety buffer that pure-attention models lack.
3. At 172B parameters, MiniMax does NOT show holographic safety. This is striking because both the 122B and 397B Qwen models do. It suggests that holographic safety is not just a matter of scale — it requires the architectural complexity of hybrid attention/memory systems.

The Most Common Deployment Format Silently Resets Modifications

We reported this in our 122B paper, and cross-model testing confirms it: converting a model from HuggingFace format to GGUF (the format used by llama.cpp, Ollama, LM Studio, and most local deployment tools) silently destroys weight modifications.

The GGUF converter rearranges internal weight structures for compatibility reasons. These rearrangements scramble any surgical modifications that were applied in the original format. A community member independently confirmed this by converting a third party's modified 122B model — the GGUF version behaved as if it had never been modified.

This means the most common local deployment pathway is currently immune to weight-based safety modifications. This immunity is accidental and undocumented — it could disappear if the GGUF format changes in future updates.

Every Precision Level Is a Different Model from a Safety Perspective

Across all 9 models, we found a consistent pattern: quantization actively transforms safety properties. It is not neutral compression. Specific findings:

• 4-bit needs more force than 8-bit — quantization regularizes weights back toward safety (confirmed at 2B and 35B scale)
• Modifications don't transfer between precisions — weights modified at 4-bit cannot be applied to an 8-bit model. Each precision level has its own internal geometry.
• Full-precision compliance ≠ compressed compliance — the 35B produces real content at 16-bit but sanitized placeholders at 8-bit, despite both appearing to "comply"

The practical implication: safety evaluations conducted at one precision level say nothing about safety at another. A model verified as safe at 8-bit may behave differently at 4-bit. This is a gap in current evaluation practices that our data exposes directly.

Calibration Data Determines What Survives Compression

Modern quantization methods like AWQ (Activation-Aware Weight Quantization) decide which weights to preserve at higher precision based on a calibration dataset — a set of example inputs run through the model to measure which weights matter most.

This creates a supply-chain vulnerability: if the calibration dataset is crafted to emphasize certain behaviors, the quantizer will preferentially preserve those behaviors while compressing others. A benign-looking model could be distributed with a calibration set designed to degrade safety upon quantization — and standard safety tests on the full-precision model would show nothing wrong.

This finding, which we first reported in our 394B paper, now has cross-model support: the sensitivity of quantization to calibration data is consistent across architectures and scales.

Safety Training Degrades Capability — But the Cost Varies by Architecture

Our original 394B finding — that removing safety directions improves the model's performance on harmless text by approximately 11% — is confirmed across the dense model family. Safety-trained models carry a measurable capability cost: the safety-related signals are always present in the model's internal state, even on completely benign inputs, acting as noise that slightly degrades output quality.

But the magnitude of this "alignment tax" depends on architecture:

• Hybrid models (Qwen) pay a higher tax. The compressed-memory layers create additional safety redundancy — more pathways carrying safety signals means more noise on benign inputs.
• Pure-attention models (MiniMax) pay a lower tax. With all safety information in a single pathway type, there's less background noise.

This suggests that architecture choice is a safety decision. Hybrid designs are harder to break but impose a higher capability cost. Pure-attention designs are more efficient but offer less robust safety. This tradeoff should inform deployment decisions.