Jake,
1. I would take a look at your parents and heridity. If you are a late bloomer it could be from you father's or mother's genes. You believe your body still can turn itself around by 24 if you are a late bloomer. It may take a few year more for it to bring your homones into equalibrium.
2. May be your body is not producing enough testoterone. There are cases of this where doctors will prescribe steriods to males to bring their levels up to normal. You may want to have a physical and have some test performed to check your testoterone levels.
3. Your genetics are the way that you are. If you find the right girl who loves you and you and return, they will love you for who you are, period.
4. Some of your concerns:
a. Facial hair - if it doesn't grow in fully, you will not have to shave as thoroughly. If you can grow a full beard or side burns, wear a gotee.
B. Pubic hair - many couples have sex in the dark. For some girls this could be a turn-on? Men's health say less hair at the base make your penis look bigger.
C. I don't think my un-erect penis is 3" unless relaxed. Men's health magazine states the average erect penis is 5"-6". Your height has nothing to do with your penis size. Bigger is not always better for satisfing women when having sex. Learning how to satisfy her will pay greater dividends in a relationship/marriage.
D. Many people are insecure about their body, but if a girl really loves you, she will except you for who and what you are. Be who you are, don't be afraid to take risks if you find a girl your attracted. Girls like to be persued. It took my wife 2 years of me pursuing her, before she realized that she loved me. As long as she doesn't tell you to get lost, wear her down! Be confident on who you are. You have a purpose and destiny. The bible states we are all fearfully and wonderfully made.
My recommendation is see a doctor and take a physical. Ask your questions. They can run test to see if you body is producing enough testoterone. If it isn't, they can regulate it for you through steriod creams, etc (doctor prescribed). Men's health magazine has put together eating plans to help the body's production of testoterone.
To me, it seem you'll be ok. Just seem to be a little insecure of your self.
I would highly recommend subscribing to men's health magazine to learn more about your body and what is normal and common to our bodies.
I hope this helps you,
tomz