Sound pops with Creative X-Fi USB

[Kaetemi]

I'm using a Creative X-Fi USB with a 5.1 speaker set attached to it.
When the sound is configured to "Analog Surround 5.1 Output", I regularly hear very random pops and cracking sound, like an old LP.
It's mostly hearable when a loud sound is fading out, and when manually changing volume.
Sometimes it glitches into a continuous popping. Killing pulseaudio fixes it for a while, but that gets annoying after twenty times per day.
When in Stereo Output mode the problem does not seem to happen, but that's not very useful either.

[Kaetemi]

Issue remains when removing PulseAudio and using ALSA.