I often have the same problem---i dont know what it is but you arent alone

I do this all the time. I have adjusted to this by pointing and saying "Right?" they will then say right or no the other right. I also tell them to point in my line of view. This decreases the confusion. I haven't figured out a way to combat this driving alone besides shaking my right hand when deciding left and right. You shake hands with your right hand.