Set your own ground rules and stick to them.
If you say it's dinner only - mean it - it'll do your own self esteem good as well.
Dinner does not mean sex or anything else.
Friends can go out to dinner together.
Just know what oyu intend and stick to it, make it clear you have no intentions of taking it further (esp not on this occassion)
then go out let mom look after the kids and just enjoy a night out.