It depends.
If you grew up with a lot of fat reserves chances are you had high amounts of estrogen in your body during puberty. If you find your chest is a bit "pointy" or "saggy" then this is not just fat, it's actual breast tissue. Unfortunately there is no perminant solution for removing this other than surgery. This condition is known as gynecomastia. Many men suffer from this, it's usually very light, although some men will develop full breast tissue and be required to had it surgerically cut out. There are things you can do to temporarily tighten this area, in the case of gyncomastia you will need heavy weight, low reps. (low weight / high reps is for reducing fat.)
however, if this is something that has recently popped up, perhaps you gained quite a bit of weight, and now you're trying to lose it, then it's likely fat, and you'll need to the low weight / high reps, and some cardio to help target your chest and reduce fat.
Justin