I haven't had a baby but I have been a shoulder to cry on for friends who have had the same problem - relatives said 'don't be stupid, you can't call your baby that!
What they found is that once the baby was born and they stuck by their guns and named the baby, people soon kept their mouth shut and said no more - probably because they saw the baby as an individual rather than an unborn child so I think you will find that once your baby is born, you will find the same.
Personally, I think it's a very pretty name - not only pretty but unusual too.