Your scrotum is supposed to feel colder than the rest of your body. It acts as a temperature regulator because your sperm need to stay at a certain temperature to be healthy. You'll notice that when you go swimming or are really cold, your scrotum tightens... this is so that your testicles are closer to your body and can be warmer. The opposite happens when you're too hot.

Normally, to be at their healthiest, your sperm have to stay at a temperature lower than your core body temperature. This is why you'll notice that your scrotum/testicles are cooler than the rest of your body. It's absolutely nothing to worry about!