You cannot yet be pregnant, but you can be on your way to getting pregnant. It takes a few days for a fertilized egg to attach tithe uterus wall and show up as pregnant.
Semen cannot go anywhere, your vagina is closed at the top. Only his sperm can swim up your cervix with the rest of his semen coming back out.
It probably means that he just ejaculated very little. It sometimes happens. Give him a day break to allow his body to replenish his semen and sperm stores.
If you had your orgasm shortly after his, your uterus can dip into the semen pool at the top of your vagina and suck some up. You will know if your orgasm and the timing was different than other times. It should however not make such a drastic change.
Best of luck TTC!