Nah, you're fine. As long as you don't spend a ton of time upside down (say hours), it's harmless

Uuh most exercises in general aren't "safe", if your asking that literally, you are always risking something for many of them, particularly those with weights involved. But since when do humans do only whats safe for them? =) I do handstand push ups too, though I use a wall since I lack the balance for now, but I'm working on it so I can do handstands for when I start learning breakdancing. You must, absolutely must breathe regularly for any exercise though, particularly those that will put pressure on your head such as vertical push ups. Not doing so could pop a blood vessel. So it isn't safe if you hold your breathe with all that blood rushing to your head.