Can I do anything about this?

Hi, I have full physical custody of my child (7 yrs old) and my ex has joint legal custody. This custody arrangment has been in place for 2 years now. At the time of the arrangment I tried to arrange for supervised visits but the judge did not grant that based on the fact that my ex lives with his parents. He was diagnosed as bipolar and is on medication currently. My issue is that when my child goes to his house for his visitation, she ends up spending approximately 90% of her time with his mother while he works his part time job. We have had several issues arise such as the grandmother sleeping in her bed with her on a routine basis, bathing her and my child coming home with scratches in inappropriate places, and making decisions regarding my daughter without speaking to me first, aslo they refuse to take her to a doctor for medical attention when she is sick. On several instances my daughter has run a fever over 100 degrees and they did not take her to the doctor; I have had to take a day off when she comes home on Mondays in order to take her to the doctor and get her the necessary medical treatment. I have taken my daughter to her primary care physician and raised these concerns to them and they agree that there are red-flags but no hard evidence of any wrong-doings. Just as a little back-ground, I work a full-time job and live in my own apartment. He lives with his parents still 3 years after our separation and is still only working part-time. Is there any legal action I can take either to decrease her time over there since he’s not spending that time with her to begin with? Also, I’m trying to move about 1 1/2 hrs away (still in the same state) which will affect the current custody arrangement; can he prevent me from doing so? My motivation for moving is to go back to school for my second degree and also to be closer to my support system? Thank you for any help and advice you can offer!

You could file a motion to modify custody.