We had a post on this topic a while back and there were a lot of facts in it.
Like the above post said, if you have had cats before and have changed their litter box at all then you already have that toxi.......Thing....It is only dangerous if you get subjected to it for the first time while pregnant.
Chances are that is not going to happen.
The article said that this toxi.....Thing doesn't even develop until the cat's poo sits for two to three days. So, if you change the litter daily, no worries. But, still use gloves and wash your hands directly after changing. But if at all possible, just have someone else change the litter.
Cats do step into their litter box, but with as much as they groom, I seriously doubt that you will get it from your cat. But, better safe than sorry.