I stopped growing at 14, I was 5'10 though, but still shows how it can occur. Much of it probably contributed to my semi-vegetarian diet, poor sleep, poor eating habits, yadda yadda. If you aren't going to drink milk, find other means of obtaining calcium. Some orange juice, etc are calcium fortified and whatnot. Don't do too much heavy lifting though, you aren't at the appropriate age for that. Calistenics, machines, light dumb bells are fine, but I wouldn't suggest doing power cleans, deadlifts, etc.
So you think milk is nasty, no pain, no gain =) Suck it up lil bro, and chug.