Work

Research Experience

Undergraduate Researcher

P-Square Lab, IIT Roorkee
Present

Working with Prof. Parikshit Pareek on Deep Generative Models.

  • Investigating Bilinear MLPs to design more interpretable neural architectures.
  • Finalized a Sparse Diffusion framework for high-energy physics applications (CERN). [Read Paper]
  • Designed a sparsity-aware Variational Autoencoder (VAE) for modeling data with only 0.2% active pixels, introducing a weighted loss to stabilize training.
  • Building a foundation model for amortized kernel hyperparameter discovery.

AI Engineer Intern

Elimentary
Past

Gained production-grade AI experience working on Local LLMs.

  • Engineered local LLM APIs to enable secure and efficient model deployment.
  • Developed comprehensive evaluation pipelines to benchmark and validate model performance.
  • Optimized workflows for production-ready AI systems.

Publications

  1. bilinear_mlps.png
    Structural Disentanglement in Bilinear MLPs via Architectural Inductive Bias
    Ojasva Nema, Kaustubh Sharma, Aditya Chauhan, and Parikshit Pareek
    Preprint Preprint 2026
    Architectural inductive bias supports structural disentanglement which, in turn, helps unlearning and generalization.
    PDF
  2. pfns_spectral.png
    Amortized Spectral Kernel Discovery via Prior-Data Fitted Network
    Ojasva Nema, Kaustubh Sharma, Srijan Tiwari, and Parikshit Pareek
    Preprint Preprint 2026
    An interpretability-driven framework for amortized spectral discovery from pre-trained PFNs with decoupled attention.
    PDF
  3. mlp_vs_attn.png
    Dissecting Attention and MLP Roles: A Study of Domain Specialization in LLMs
    Ojasva Nema, Kaustubh Sharma, Abhiraj Bharangar, Manjot Singh, and Srijan Tiwari
    Preprint Preprint 2025
    Investigating how LLMs internally represent and process diverse domain knowledge. Found that attention routes domain identity while MLPs store domain-specific logic.
    PDF Website
  4. Image-alchemy.png
    Image-Alchemy: Advancing Subject Fidelity in Personalized Text-to-Image Generation
    Ojasva Nema, Kaustubh Sharma, Cherish Puniani, and Amritanshu Tiwari
    ICLR Workshop ICLR DeLTa Workshop 2025
    A personalization method for diffusion models advancing subject fidelity in text-to-image generation.
    PDF