Tiny-Align: Bridging Automatic Speech Recognition and Large Language Model on the Edge

Tiny-Align introduces a resource-efficient framework for cross-modal alignment between ASR and LLM on edge devices, enabling personalized audio-based interactions. It achieves 50x training time speedup with improved alignment quality, making it the first work to study efficient ASR-LLM alignment on such devices. The framework uses a novel projector, BridgeFormer, based on a transformer encoder architecture without positional encoding, which provides a larger embedding space than existing designs. An instruction injection mechanism further improves result quality by about 50%. Tiny-Align supports high-quality interactions for individuals with disabilities through an ASR interface. The text also discusses various research papers and technical reports on LLMs and their applications in speech recognition, generation, and processing, including personalization, fingerprinting, online depression detection, adaptive and incremental ASR, automatic speech recognition, multimedia-assisted LLMs, cross-modal alignment, prompt tuning, retrieval-augmented generation, and cross-modal training for activity recognition.