I’m evaluating a few neural vocoders (HiFi-GAN, Vocos, etc.) and I’m seeing the same type of artifact across all of them. I’m not sure exactly what it is, and I’m looking for help identifying it and figuring out how to reduce or remove it.
Overall, the models can generate very good audio. In some cases the output is almost indistinguishable from the ground truth. But in other cases, there’s a consistent artifact that’s clearly audible.
I’m not an audio expert, so I tried to inspect the signals visually. I plotted both the ground truth (GT) and the predicted audio (PRED) as:
- waveform (time domain)
- spectrogram (frequency domain)
For the GT audio https://drive.google.com/file/d/1u_p9j29khFrnuqVoHxt8dQ9RgTcjyhpc/view?usp=sharing:
- The waveform looks “healthy” or “full”.
- The spectrogram also looks clean and well-structured, and the audio sounds perfect.
For the PRED audio https://drive.google.com/file/d/14dsXOZR7JmOwnmNG3kFh6p5Cr7nagrBP/view?usp=sharing:
- The waveform looks much “thinner” than the GT, like it’s missing energy or detail.
- In the spectrogram, there are visible “smudges” or “stains” that line up with the artifacts I’m hearing.
My intuition is that I might need a better loss function that explicitly penalizes these types of artifacts so the generator learns to avoid them.
In the model used to generate the PRED audio, I already implemented a Multi-Resolution STFT loss, which helped to some extent, but the artifacts are still present. This seems to happen with multiple vocoders, some more than others, but the artifact is clearly audible in all of them.
I’d really appreciate if someone with more audio/ML experience could:
- Help identify what kind of artifact this is (e.g., noise, aliasing, phase-related, etc.), and
- Suggest what I should look into to fix or at least minimize it (e.g., different loss terms, regularization, changes in the discriminator, training setup, etc.).
Thanks in advance for any insights.