Slightly Elevated Wbc?

Last week I had a CBC with differential for the first time in years, and everything seems to be great, except I was told my WBC count is slightly elevated at 11.0. I went back to be retested two days ago, and my WBC count decreased to 10.5 (I don't have a copy of the results, but the nurse informed me that the other areas were once again fine), but I'm to be retested in a few weeks to see if it goes any further due to my worries about this.

I'm terrified this might be leukemia, but my doctor says he is not concerned about that at all and that leukemia isn't even on his mind, that it's most likely an infection or stress that's causing these numbers. I don't have any symptoms. I have General Anxiety Disorder and Panic Disorder and have been suffering from extreme anxiety lately, and I've had a post nasal drip with a sore throat for a little over a week now, but I'm not sure if this is causing these numbers. I am a hypochondriac, so this has been very distressing for me.

Should I be concerned about these numbers, and could this be an indicator of leukemia?