CARD: Cross-component Audio Representation Distillation for Encoder-Free Audio Captioning
Removes the audio encoder at inference through cross-component knowledge distillation. A 13.2M-parameter projector supports encoder-free audio captioning.
PhD Researcher, University of Essex · AI Engineer (NLP), Hivedome Ltd.
Ganesh Pavan Kartikeya Bharadwaj Kolluri
(Karthik Kolluri)
I study what speech and audio language models lose when we make them more efficient: how much accuracy can be recovered, and whether every speaker is affected equally. Alongside my PhD research, I build LLM applications for real industry workflows.
About
I'm a PhD researcher in the School of Computer Science and Electronic Engineering at the University of Essex and an AI Engineer (NLP) at Hivedome Ltd. As the KTP Associate, I work on a Knowledge Transfer Partnership between the University and Hivedome, part-funded by UKRI through Innovate UK.
My research examines efficiency and fairness in speech and audio language models. I investigate encoder pruning, how LoRA fine-tuning can recover recognition accuracy, and how changes in model performance differ across speaker groups. I also explore knowledge distillation for audio captioning without an audio encoder at inference.
At Hivedome, I'm developing ASK ITAS, an AI assistant for commodity trading and risk management. Previously, I worked as a Machine Learning Engineer at Eidosmedia in Milan, building language-model and retrieval systems for news publishers. I also developed my MSc research on automatic newspaper pagination into a production system.
News
- CARD: Cross-component Audio Representation Distillation for Encoder-Free Audio Captioning accepted at IEEE SLT 2026. Read preprint.
- Pruning for Efficiency, Paying in Fairness accepted for an oral presentation at IMPACT-SPEECH (EMNLP 2026). Read preprint.
- On the Role of Encoder Depth accepted at SPEAKABLE (LREC 2026). Read paper.
Earlier news
- Preprint of CARD released on arXiv.
- Started as the Associate on a Knowledge Transfer Partnership between the University of Essex and Hivedome Ltd.
Research themes
01
Compression
How much of a speech model can be removed before recognition accuracy deteriorates? I study encoder-layer pruning in speech–language systems and how much performance can be recovered through parameter-efficient adaptation.
02
Distillation
Can an audio captioning model operate without an audio encoder at inference? My CARD research transfers knowledge from a pretrained audio teacher into a compact projector and language model, allowing the teacher to be removed after training.
03
Fairness
Who bears the cost of a smaller model? I investigate how pruning affects recognition accuracy across speaker groups, because overall error rates can conceal meaningful differences.
Selected publications
Selected work on efficient speech and audio AI, from reducing model size to examining who is affected by compression. Full abstracts and citations are available on the Publications page.
Pruning for Efficiency, Paying in Fairness: Demographic Disparities in Pruned Speech-LLMs
Shows that pruning can widen recognition disparities between speaker groups even when aggregate accuracy appears stable or improves in a reported experimental setting.
On the Role of Encoder Depth: Pruning Whisper and LoRA Fine-Tuning in SLAM-ASR
Demonstrates that selective Whisper encoder pruning plus LoRA can outperform the study's unpruned baseline while reducing total parameters by 7–14%.
Education and industry experience
-
2025 – present
AI Engineer (NLP) / KTP Associate, Hivedome Ltd.
Developing ASK ITAS, a knowledge-base question-answering assistant for commodity trading and risk management, with a tool-using agent architecture in development.
-
2025 – present
PhD Researcher, University of Essex
Researching model compression, knowledge distillation and fairness in speech and audio language models.
-
2022 – 2025
Machine Learning Engineer, Eidosmedia, Milan
Built retrieval-based question-answering systems and domain-adapted language models for publishers; developed an MSc reinforcement-learning pagination project into a production system.
-
2020 – 2023
MSc in Artificial Intelligence, University of Bologna
Graduated with distinction. Thesis: Autonomous Pagination for Newspapers from a Predefined Plan.
-
2020
Data Science Intern, ZeoMinds IT Solutions, Hyderabad
Worked on data preparation, exploratory analysis and predictive-maintenance models.
-
2015 – 2019
BTech in Computer Science and Engineering, Sasi Institute of Technology (JNTU)
Graduated with distinction.