You cannot assume that you are not fertile just because you failed to get pregnant when you were on birth control, even if you were somewhat careless with it.
Women are only at peak fertility a couple of days a month. There are signs of ovulation, but birth control prevents ovulation when taken as directed. Even what it is not taken exactly as directed (say a late or missed pill here and there), ovulation can, but often does not, occur right away.
It takes some couples a year or more of actively trying before they get pregnant. A fertility specialist will not even agree to see a couple until they have been actively trying to conceive for one year or more.