Parent problems?

Me n my ex got into a huge fight on the dec 4th i took our son n left thinking itll all blow over in a few days then i could go bac n everything would b ok WELL. My family wants me to take him to court n rip his rights away n i dont wanna do that to him hes my sons father n i still love him my heart blongs to him n my son but he called c&y askin for help on vists etc i told him point blank hes not goin to a house i cant go to bcuz i wont get him bac n im not letin it happen i wanted to make it work out for us for bcuz of our son i need help idk wha to do or where to turn its feels like my life is getin ripped away im tryin to come to an agreement wit him on when he can have our son over night and all that til he has his own place along wit me but when hes bein an *** half the time n i dont even do anythig to piss him off he just changes