Custody

If I received a divorce in NC and we had a seperation agreement that was notorized and in the seperation agreement there was a paragraph that stated custody arrangments are those arrangements legal and binding. In the papers it states we have joint custody with him having primary physical custody and me secondary.We would like to switch it to me having physical custody so hw can live with me.I live in SC , my ex in VA.Do we have to go thru court to change the arrangements or can he just come live with me.I guess what I need to know is if the papers are a legal custody agreement or just an agreement between the two of us that were signed in lawyers office…This would be an amlicable agreement between the two of us for he doesnt want him to live with him.My son has been living with my mom in NH since the school year began because my ex didnt want him with him anymore.Just need to know where to begin as my ex is totally dropping the ball and wanting me to handle it on my end and I have no idea where to begin.His tells me his lawyer is saying we don’t have to go thru the state of NC where we where divorced and that the seperation papers arent a legal contract for custody just a agrent between the two of us but I thought the seperation agreement was a legal agreement as to who could have custody.Want to make sure the transition of my son here to SC is handled leaglly and properlly so I can enroll him in school.