Shared Semantics, Divergent Mechanisms: Unsupervised Feature Discovery by Aligning Semantics and Mechanisms

Cho, Hyunjin; Roh, Youngji; Kim, Jaehyung

Abstract

Lay summary

When an AI model answers a question, there is usually more than one reasonable response it could give. Explaining only one chosen answer can miss these alternatives. Our method looks across many possible responses, groups the ones that mean similar things and seem to come from similar internal signals, and turns them into a short list of answer patterns. This approach helps people choose what to inspect, compare, and test before digging into the model's inner workings.

Circuit analysis usually explains one prompt paired with one chosen completion. That can miss variation across the model's continuation distribution. We introduce distribution-level unsupervised feature discovery: cluster sampled continuations using semantic content and sequence-level mechanistic attributions, without specifying target outputs by hand.

Each continuation gets a semantic embedding (a vector representation of what the continuation says) and a prefix-to-continuation attribution signature (which prefix features pushed the model toward this continuation). A rate-distortion objective trades off semantic coherence, mechanistic consistency, and cluster granularity. The resulting clusters expose modes that single-view baselines miss and give better targets for steering analyses.

Challenge: One Target Can Hide Many Mechanisms

Q: Who sings a Khmer version of "You've Got A Friend" in 1973?

A1 "...in 1973 was sung by Sok Sisowath..."

A2 "...was performed by Sok Sisowath, a notable Khmer singer..."

A3 "...was recorded in 1973 by Srey Mony..."

A1 vs A2: Semantic similarity 0.95; Mechanistic distance 132. Nearly the same answer, but far apart mechanistically.

A2 vs A3: Semantic similarity 0.62; Mechanistic distance 87. Different answer, but closer mechanistically.

Example comparing clusters found from meaning with clusters found from attribution features — View mismatch: one clustering view can look clean while the other collapses. Each dot is a sampled continuation. Colors show K-means clusters found in one view; if those colors mix in the other view, the two views disagree.

Mismatch between semantic and mechanistic clustering views — Score interpretation: semantic and mechanistic spaces often disagree, so semantic similarity is not a reliable proxy for circuit similarity. Silhouette is a cluster-cleanliness score: higher means same-cluster points stay close and different clusters stay apart. A high native score but low cross-view score means clusters that look clean in one view do not transfer to the other.

Semantic similarity compares what two continuations say; higher means they sound more alike. Mechanistic distance compares their attribution signatures; higher means the supporting model features differ more. In the example above, A1 and A2 are semantically close but mechanistically far apart, while A2 and A3 are less semantically similar but mechanistically closer. Semantic similarity alone is not enough: answers that sound alike can use different mechanisms, and different answers can share mechanistic structure. The method asks whether we can find continuation modes that are both readable and mechanistically consistent before choosing targets for circuit analysis.

Distribution-Level Feature Discovery

The method works at the distribution level: instead of explaining one chosen answer, it samples many likely continuations for the same prompt, represents each continuation in two ways, and then finds clusters that are coherent in both meaning and mechanism.

Method overview: continuation sampling, dual representation, and RD clustering — Pipeline: the method samples possible answers, represents each answer in two views, and clusters them into auditable modes. The output is a set of auditable continuation modes, not a single target answer.

1. Sample Continuations

The method samples many likely continuations for one prompt and weights them by model probability.

2. Build Two Views

Semantic embeddings capture what the continuation says. Attribution signatures capture which prefix features support it.

3. Cluster by Rate-Distortion

RD clustering keeps a small set of modes only when they reduce semantic and mechanistic distortion enough to justify the added complexity.

Cluster Dynamics Across Iterations

RD clustering treats each sampled continuation as a weighted point with two views: a semantic embedding for what it says and an attribution vector for the model features that support it. Starting from one cluster, the method alternates between assigning continuations to the cluster that minimizes entropy-regularized semantic and attribution distortion, updating cluster centers, and proposing splits. A split is kept only when its distortion reduction is worth the added rate, so the number of clusters emerges from the objective rather than being chosen in advance.

RD clustering step frame with semantic and mechanistic spaces — Iteration view: as clusters split, the same points can become cleaner in both semantic and mechanistic views.

Causal Validation via Intervention

Steering asks a causal question about a model: what changes if we deliberately push one internal signal up or down during inference? In Pearl's language, this is an intervention rather than a correlation check. Instead of only observing that a feature appears with an answer, we change part of the model's internal state and measure the effect. In language models, activation steering does this by adding, removing, or scaling an internal direction or feature so the output logits shift.

Our cluster intervention applies this idea to answer modes. Can one cluster-level direction move the model toward a whole answer mode, rather than only one sampled continuation? For each discovered cluster, we choose a representative source continuation, select the signed top-B attribution features from that source, and scale those features at their original prefix position and layer. We then compare RD, which uses the medoid from the joint semantic-and-mechanistic cluster, with KM-Sem, a semantic-only K-means medoid, and Single, one randomly chosen attribution vector from the same RD cluster. Across steering strengths, a useful cluster direction should show a monotonic target-cluster logit change: amplification raises the cluster's relative preference, and suppression lowers it. In the demonstration below, the star marks the source continuation defining the steering direction.

Diagram showing steering direction selection and cluster-level steering validation — Steering setup: the selected RD medoid direction provides top-B attribution features for activation steering, then target-logit changes are measured for cluster members.

Mean Pearson and Spearman steering correlations for RD, KM-Sem, and Single — Correlation check: RD shows the clearest monotonic response, while semantic-only clusters are near zero and single-continuation directions are weaker.

Result

Positive correlation means the intervention behaves directionally: stronger amplification or suppression produces larger target-cluster logit changes. RD medoid directions show the most consistent monotonic response, KM-Sem stays close to zero, and Single transfers only partially from one randomly chosen continuation. Implication: KM-Sem's failure suggests that semantic grouping alone can put continuations with shared semantics but different mechanisms into one cluster, yielding a source direction that does not transfer causally. Single's weaker transfer, compared with RD, supports the motivation that one sampled continuation can miss the cluster-level mechanism.

Cluster-Level Transfer Case Study

Question: Who pays for the renovations on Holmes Next Generation?

Mode 1: Production Funding

Held-out continuations answer that the renovation is funded by the production company or network.

A1 "...funded by the production company or network..."

A2 "...HNG... funded by the production company..."

A3 "...reality TV show... funded by the production company..."

RD-medoid source "...funded through the production budget..."

Amplify: +0.598
Positive held-out: 100%
Suppress: -0.189

Single random source "...spinoff of the original Holmes on Homes series..."

Amplify: +0.068
Positive held-out: 70%
Suppress: -0.334

Mode 2: Not Recognized

Held-out continuations reject the premise as not widely recognized or not officially documented.

A1 "...not a widely recognized or officially named project..."

A2 "...not a well-known or officially recognized title..."

A3 "...no well-known entity... associated with a property or renovation project..."

RD-medoid source "...not a widely recognized or officially documented show..."

Amplify: +0.239
Positive held-out: 100%
Suppress: -0.228

Single random source "...not a well-known or officially recognized show..."

Amplify: -0.011
Positive held-out: 40%
Suppress: +0.258

Case-study implication: the RD medoid transfers to held-out continuations in the same mode, while a random single continuation can be weak or even reverse sign. This supports the paper's claim that the discovered direction is not tied to one surface string, but captures a cluster-level causal factor shared across continuations.

Caveats

Our work treats RD clusters as useful mechanistic hypotheses, not complete circuit explanations.

Estimand: the intervention target is the mean demeaned logit over a sampled continuation. This differs from conventional next-token circuit analysis, which traces a single target-token logit.
Cost: the method needs semantic embeddings and mechanistic attribution summaries for many sampled continuations, which can be expensive for long contexts.
Labeling: clusters can mix answer identity, wording style, hedging, or formatting, so human-readable labels still need careful validation.
Scope: steering tests give interventional evidence for cluster-level feature directions, but they do not trace every token-level pathway to a final token prediction.

BibTeX

@inproceedings{
cho2026shared,
title={Shared Semantics, Divergent Mechanisms: Unsupervised Feature Discovery by Aligning Semantics and Mechanisms},
author={Hyunjin Cho and Youngji Roh and Jaehyung Kim},
booktitle={Forty-third International Conference on Machine Learning},
year={2026},
url={https://openreview.net/forum?id=C9AhjL8aUZ}
}

Shared Semantics, Divergent Mechanisms

Unsupervised Feature Discoveryby Aligning Semantics and Mechanisms