There are other options:
- Collaborative Law
- Schools for kids with social needs, like the Milton Hershey School.
Is there any relationship with the mother at all? In order for mediation to work, you would have to be able to offer something that she needs in exchange for what is best for your child. Some mediators work on a sliding scale so you might be able to do this for less than $300/session and in as little as one or two sessions, depending on whether your mediator assigns homework.
Collaborative law involves an attorney but not litigation. Again, do you have anything that you can offer as a carrot to get her to participate?
The social service people might be able to help you suggest the special school option to your spouse. This is a residential school where the children live in a same sex home with a houseparent couple. Boys & Girls Town (in the Midwest) is similar. Milton Hershey School has children from all over the US and Puerto Rico. Children must be three or older to attend. You and your spouse can have visitation and vacations. The carrot is:
They provide food, clothing, medical, etc. for your child until graduation. They provide a full college scholarship and a laptop commuter plus a Senior suitcase (what you need for college dorm living). Your children would live with other children who deal with issues of addiction in their biological family.
I too hoped to decide child custody with my STBX without litigation. We were not successful because he has money and a hungry attorney. I really admire those in the legal profession who are pursuing Collaborative Law. But we live in a broken world and there is still a need for litigation.
May you and your children find peace.