Hi, and thanks for your medical question on eHealth forum!
It could be that the previous guy was infertile. It could also be that antisperm antibody generated against his sperms in your vagina were strong enough to prevent fertilization. It can depend on a number of other factors too. However, yes, chances are there that there was some problem with the previous guy. Hope this helps. Take care!