It sounds as though your ex has the right motivation. Changing custody situation is not a bad thing, and any parent who wants to spend more time with their child should be encouraged, as long as it’s not specifically out of spite to hurt the other parent. I understand that it may be difficult to accept after this long, but it sounds to me as though your ex may be serious about this woman. If she has her son primarily with her the majority of the time then that is probably why he wants more time. Without knowing specifics, it may also be that after 4 years there have been changes that would make it easier for him to spend more time with your son. Have you talked it over with your son? (if he’s old enough)
Your ex has the right and ability to go for custody at any time as you do. I suggest that it may be easier on both of you and your son to compromise. Try to work out a joint custody arrangement with equal time. The arrangement you have now does not offer a lot of time even with you being flexible. That’s essentially only 6 overnights per month and that’s with him spending Friday & Sunday nights on his weekend. Try it for a few months and then if it works for your son, have the child support changed. You do not necessarily have to go to court over this and it would be cheaper to work it out between yourselves. The best scenario for a child is to have both parents as actively involved in their lives as possible and that can only happen if the parents are willing to work together.