This is going to sound really ridiculous but I can't help thinking the worse every time I think about this issue. Here goes...
When I finish masturbating, I clean my hands with water and soap, yet I always get paranoid there may be sperm lingering around on towels and other household objects and that someone could get pregnant if they come into contact with it.
Stupid I know, right? Reassurance would be very nice though
Hey, I don't think you are silly at all. I have a similar but different situation with my partner. We have an incredible and very active sex life, which if course means lots of semen. As soon as ejaculation occurs, regardless of where it occurs, my partner leaps into action to clean all traces away. At first I thought was so gentlemanly but now I really want him to relax, enjoy his orgasm for a few minutes at least and cuddle with me. He understands this when we talk it through but it's almost like he is possessed until it's all cleaned up.
Now here is the thing ... There is zero chance of pregnancy ever occurring for us. It's purely about an aversion to his own semen. Funny man!
Sperm needs a very specific environment to survive. The temperature of the average room is enough to kill it.
As to someone inadvertently getting pregnant, I wouldn't worry. the likelihood is sooo low as to make it a non issue for most situations.
Perhaps if you wiped up sperm with a towel and within about 10 minutes a woman used that spot of that same towel to aggressively dry her vulva... Even then she would have to be at the right point in her cycle.
Basically you are stressing over nothing. Is it possible-Sure it is. Is it likely-HELL NO!
Either way, it's good to be on the safe side, just don't need to go to extremes about it.