Never keep quiet when people speak against you. Spiritual warfare is a war of words. It says "No weapon formed against you shall prosper and any tongue that rises up against you in judgement YOU SHALL CONDEMN.."
Being nice doesn't work in spiritual warfare. Announce your victory