Gynaecology is for girls and their bits.
Reading your post, I thought you were inserting things into your penis and was going to say maybe you have irritated the lining of it.
But yes I would get your prostate checked out and remember to sterilise anything you are going to insert first, especially if it has been in your bum as this can be a major casue of re-infection.