It seems the issue is power saving mode on linux, at least for ubuntu, turn off sound card power saving. There should be similar settings for other distributions.
For Ubuntu, edit the file:
sudo nano /etc/modprobe.d/snd-usb-audio.conf
Add the following content:
options snd_usb_audio power_save=0
Save and reboot.
No noise on battery less than 15%, no noise on less than 10%.