When you see ,the numbers of your enemies increased at the place ,were you are residing,it is always better to shift your residence to the place where environment/people are congenial to you.Being the God ,the lord Krishna had shifted to Dwarika,in the same way the great saint Kabir also shifted from Varanasi to Maghar.This you may not do for yourself ,but you must do for the happiness of your family.