About

Research Interests

My research interests center on understanding and building systems that exhibit general intelligence. Specifically, I work on mechanistic interpretability -- reverse-engineering what neural networks actually compute, understanding the algorithms learned during training rather than just input-output behavior. I am drawn to scaling laws and their implications for architecture design: how compute, data, and parameters trade off, and what this means for choosing between transformer, state-space, and recurrent architectures. I study transformer and diffusion model architectures in depth -- attention mechanisms, state-space models like Mamba, and the architectural inductive biases that shape what models can learn. I work with fine-tuning techniques including LoRA, QLoRA, and quantization methods (MXFP4, GPTQ, AWQ) to understand their effects on model behavior beyond just parameter efficiency. A central thread in my work is mesa-optimization -- investigating models that learn to optimize internally during forward passes, creating emergent optimization loops that are not explicitly trained for. Finally, I focus on mathematical reasoning in AI, pushing LLMs to solve competition-level mathematics through tool integration and structured reasoning.