Here's the easy way to think about it- take every pill, in order, until they are all gone, and then start at the beginning of a new pack.
Of course, you're right, you don't have to take the inactive pills. But, if you don't take them (or at least wait until that much time passes) then you might confuse your cycle the next time around, or have some spotting between periods. Usually people only skip the inactive pills if they are trying to alter their period, like skipping it altogether.
So if you're interesting in keeping your period on the schedule of the pills, take the inactive pills and take them all.
If you'd rather make it so your period doesn't come, don't take them, just start the next pack. But if you do this, know that you might have some spotting and irregular bleeding.
Of course, (and it sounds like you know this) ALWAYS take all of the active pills, no matter what!! And never have more than a 7 day break for your period.