Irregular cycles and infertility ?

I have no idea how this works with fertility, but I have a question about the longterm effects of birthcontrol and fertility. I was prescribed birthcontrol since I was 18 to control the extreme heavy bleeding I would have, and so I stayed on it (I believe it was Seasonale... the kind that only allowed you to have four periods a year) for just about three years. I have since then been off of birth control for almost a year now. I have noticed that my cycle is TERRIBLY irregular, sometimes almost a MONTH late. I dont understand this... it seems to have gotten a bit better now, but still sometimes a week or so late. Im wondering if this might have any longterm effect on my fertility as well??? Im kind of concerned...help!