Remember when you told people you were pregnant and they all said, "Oh...!" Well, this his part of that "Oh...!" Every couple - EVERY SINGLE COUPLE WHO HAS HAD CHILDREN! (if you say, "Not me & my husband!" YOU ARE LYING!) has had to make huge adjustments in their relationship when they add an entirely new human being! This is a fact of life. You cannot have your heart learn unconditional love and not have some issues with the love of your life.
Not only are you busier than you ever thought you'd be - everything in your life has changed, right down to what you think of when you go to sleep at night. Most new moms want to hold and cuddle their new babies all day! Why do you think nothing else ever gets done? Because babies sleep 90% of the time at first and still those new moms (me included!) never get anything done. We can't put those babies down.
Some men feel jealous of this new creature taking all their attention away. Some don't. But it is an adjustment for EVERYONE. You aren't any worse than anyone else.
If it bothers you, you could set aside some 'couple' time every few days. Where you put the baby to bed and hang out together - just the two of you.