LAFMA: A Latent Flow Matching Model for Text-to-Audio Generation

Abstract

Recently, the application of diffusion models has facilitated the significant development of speech and audio generation. Nevertheless, the effectiveness of the method is accompanied by the extensive number of sampling steps, leading to an extended synthesis time necessary for generating high-quality audio. Previous Text-to-Audio (TTA) methods mostly used diffusion models in the latent space for audio generation. In this paper, we explore the integration of the Flow Matching (FM) method into the audio latent space for audio generation. The FM model is an alternative non-autoregressive method that trains continuous normalization flows (CNF) based on regression vector fields. We demonstrate that in text-guided audio generation, Latent Flow Matching (LFM) significantly enhances the quality of generated samples, achieving better performance than prior models. Moreover, it reduces the number of inference steps to ten steps almost without sacrificing performance.