It seems likely that if you wiped your fingers thoroughly then all the semen was removed and the chances for pregnancy are minimized.
Also, the semen does not survive very well when exposed to air, and the spermatozoids lose their viability and motility thus further decreasing the chances for fertilization.