It's not too late for you to get plan B (the morning-after pill), if you really want to put your mind at ease.
If your doctor said that the medication can alter the effectiveness of the pill, then you are at risk to get pregnant. To what degree, we don't know. The pill you took monday morning would have theoretically protected you for the day on monday until you took the medication, but since sperm can live inside you for up to 5 days, you are at an increased risk for pregnancy due to the medication.
If you do get plan b, make sure to speak with a pharmacist or your doctor and find out if the medication will affect plan b as well.