Fantasies in itself isn’t a bad thing, but sometimes guys can get carried away by them, and when the opportunity presents itself, they will take advantage of the situation or even create one.
He already tested the water by saying that he would like to have one, so I would strongly suggest that people in situations like this one should discuss what is acceptable and what isn’t in their relationship. If both partners don’t agree, then it is time to move on.
This post is almost 2 months old but I hope the original poster made the right decision. It’s very traumatic to find out your partner has been cheating.