Can custody be jeopardized from someone sleeping over?

My STBX has threatened and harrassed me since I moved out. I stupidly started a relationship three weeks before we signed our seperation agreement. The agreement states that we are free and clear to live our lives free from interference from the other and as we deem morally responsible. We have joint custody of our children. Although I am the dependant spouse and make $20k less per year, I did not.ask for alimony or I need a roomate to help me with my expenses. I would like to have my boyfriend move in with me, but stay in his own room when I have my kids. They love having him around but to their knowledge, he is just my friend. They are only 4 and 6. My STBX.says that if he is here overnight that he will take my kids from me. Is this possible ?

If the agreement does not prevent it you are in the clear, of course he can sue for custody but he will have to prove that having this person around is detrimental to the kids. Judge’s attitudes are, for the most part, becoming more modern and they do not issue rulings preventing parents from living with whom they please.