Publications & Writing

Manuscripts

Demand-Driven Modularity in Fine-Tuned Transformers

Yiran Xu, Robert P. Dick
Manuscript in preparation, targeting ICML 2026.

A study of when and why modular structure emerges in Transformers.
Through multi-task fine-tuning experiments (e.g., Math vs. Sentiment), we find that
modularity does not arise from input distribution shifts but emerges only under
strong functional conflict. We also show that optimization favors shared low-rank
subspaces, and modularity is an efficiency–accuracy tradeoff rather than
a natural property of the architecture.

• PDF: coming soon
• Abstract: coming soon
• Slides: coming soon

Technical Reports & Writing Samples

Activation Subspace Rubrics for Evaluating Long-Form Reasoning

Technical report draft, 2025.

An exploration of how activation geometry in LLMs can be mapped to
interpretable rubrics for reasoning quality, process correctness, and creativity.
We identify latent subspaces consistently correlated with “good reasoning” signals
and validate them via causal interventions.

• PDF: coming soon
• Code (private): available upon request

Notes

This page no longer uses Jekyll Scholar.
All bibliography and .bib features have been fully removed to ensure GitHub Pages builds correctly.