Do I need to see an endocrinologist?

I've always struggled with my weight, but the last year has been extremely difficult. I eat fairly healthy, exercise 3 days a week, and yet I still seem to be gaining weight. I've been growing more and more tired and lazy, not motivated to do much of anything. Nothing drastic has really changed with my diet or every day routine that I feel would cause this.

Nearly everyone in my family has hypothyroidism. I have my primary care doctor check during my physical each year. The blood test always comes back on the higher side, but still in the accepted rage (the latest was 3.24). She feels around and finds nothing abnormal. However, all the signs seem to still point towards a thyroid issue. I've been debating on whether I should try seeing an endocrinologist for a second opinion. My aunt is in the "normal" range, but the endocrinologist did further testing and found that her thyroid was actually affecting her. She's been on medication and has since felt a lot better. I'm wondering if I could be experiencing the same thing.

Has anyone experienced a similar situation with their thyroid? Would it be worth it to see someone or just a waste of time if the blood tests are normal?