Hello,
Welcome to ehealthforum
Small painless bumps near the vagina can be sebaceous cysts,due to shaving,trauma,allergy etc.Since it is not associated with any pain,itching and is not even increasing in size it does not seem to be serious.
Sebaceous cysts usually disappear on their own after some time but if they are infected can develop into abscess.
Wash the area with Tea Tree oil mixed in water.
Wear loose cotton undies and avoid tight fitting clothes.
Keep observing them and if there is development of any symptom as discharge or pain then consider visiting a Gynaecologiost for evaluation.
Take Care!