imho, yes you can feel bothered. But you don't have to. Why does it bother you?
Personally, I went through this and it did bother me, but it was more because I was feeling inadequete and having self-confidence issues. I felt that he was looking at these other girls (who I would never measure up to) because he didn't enjoy looking at me or something.
I started to feel totally unconfident and never wanted him to see me naked, which was a huge problem (I had also gained weight which didn't help). Eventually we finally talked about it and I feel much better. I explained how I felt and it was important to me that he understood and tried to make me understand that it wasn't about me.
I got more open to the idea and now it's something that we do together. There are going to be times when one of you is in the mood and the other isn't, and you'll have to learn other ways that you as a couple can deal with it. And you'll definately have to talk about it in the meantime. People don't talk about it, but negotiating your sex life is a hugely important part of marriage.