spanish_sarah,
If you had broken the femur, you would have known it, as you would not have been able to stand on it. I guess, it would have been possible to have a very tiny crack, which did not disrupt the periosteum, but that would have healed a long time ago.
However, it is possible to damage the large muscles that do around the femur in the thigh. It is not uncommon to develop myositis ossificans within the thigh muscles, especially within the quads.
In this, a large hematoma, or bruise, develops in the muscle. Then as it heals, scar tissue starts to form. In some cases, bone can actually form within this area also, this is the myositis ossificans.
This is seen in football players, jockeys, calvary officers (where their swords would hit on the outside of the thigh), bull riders, just about anyone who can get deep muscle bruises.
But, it is very common to have scar tissue form in the muscle after a significant trauma, even if the bone does not form. Scar tissues does not stretch as well as normal tissue.
If it is really bothering you, you might want to see an orthopedic surgeon. An x-ray of the area will tell if you have this problem. Good luck.