The brown ones are iron supplements that are just to make sure that you stay in the routine of taking a pill the same time every day, even when on your period. Why don't you want to take them? They're just iron supplements.
If you want to skip your period, yes you would start a new pack instead of taking the brown ones. If you aren't going to skip your period, then you either take the brown ones or not, but don't start a new pack for however many days the brown ones would last. If it's four days, then don't take any pills for those four days, then start a new pack.
In the future, if you have questions about your pills, you should talk to your doctor or pharmacist. Your doctor may have prescribed these specific pills because you need the extra iron. In that case, you need to ask your doctor if it's okay to skip those. Most "reminder" pills in other birth control packs are just sugar pills. There may be a reason your doctor wants you on these.