what does he want?

i have a friend that i have known since high school. i dated him about 3 years ago but it wasnt really dating it was more like friends because he never showed affection other than hugging me rarely. i am dating him again and its the same thing. right now hes in the mood where everything i say annoys him. i dont get it, he says we are dating and taking it slow but he never puts his arm around me or anything. is this a common thing with bipolar they rarely show affection? i am a little confused. are we just friend are are we really dating in his mind? i say are we just friends or what and he says thats a stupid question. i dont think its very stupid since he shows no more affection than when we were friends