Hello and welcome to e health forum.
It is a very difficult situation in your case.
Since you have mentioned that you are already married and your new life was good, It is unclear why you still have any doubts in the current relationship.
It would have been understandable, in case you were unhappy in your relation with your husband, and were looking a way out. But since everything is going good, it would be in your best interest to not rock the boat you are currently in, but to make sure that it continues to sail in smooth waters.
You might inform your ex-boyfriend in plain words about your decision.
In case of any doubts, you might consult a psychiatrist for counselling and proper guidance.