I think there’s a slight tolerance issue with the slot and the cards are getting stuck on ejection. I have to use tweezers to get it out and it makes me nervous if the click hasn’t actually disengaged that I might eventually break the slot 
Is this just a me thing or more universal?
You’re not alone, happens to me too.
Good to know! The plan is to not swap much if I can avoid it.
Me too :-/ Have to use tweezers
I press it in with my fingernail and let it go as quickly as possible and it will basically pop out of the port and fly across the room if I don’t aim it somewhere safe. This has been the easiest way for me to get it out so far.
If your fingernails aren’t suited, the included allen wrench will do the job as well.