before made your dump try fill all unused space with zero
http://manpages.ubuntu.com/manpages/xenial/man8/zerofree.8.html
( other possibility > https://unix.stackexchange.com/questions/44725/clear-unused-space-with-zeros-btrfs
on a second time you could use conv=sparse option when use dd or just directly compress on the fly