Setting the latency to 128 fixed that issue for me, also it didnt work very well with the retroarch the shell shipped with I recommend following this guide RetroArch Megathread to get an updated version, there are also some better cores that are more optimized for the gameshell