Will what i signed in mediation come back and hurt me?!

my ex-husband and i have been separated for 10 months, for the first 4 1/2 months i had our 3 girls, and we had a verbal agreement they would stay with me primarily and he’d get them every other weekend and majority of the school breaks. since there were no court papers, when he had them for a visit he filed papers saying i had abandoned them, giving him temporary custody of our girls. we’ve gone through mediation and at 1st he only agreed to let me see them 4 hours a week, then he agreed to 6 hours a week, now it’s i get them friday from school and he picks them up on sunday at 8pm. that agreement was signed just a few weeks ago and is valid for 3 months. i’m wondering if me signing the agreement will hurt me in getting my girls in the long run, because i’m scared it will look like i’m not fighting for them. i am fighting for them, but i also don’t want to yank them around while they’re in school and put even more stress on them which is why i agreed to the mediation. 2 of my girls have iep’s and my other one has a pep and i know that going through a big battle right now would only hurt them. i love my girls more than anything and would give the world for them to be back with me, i know they’d be better with me, especially when it comes to schooling and their dr. appts. even tho they live with their dad right now, i’m still the one making all their appts. and keeping track of what goes on at school and attending their conferences. my middle girl had given me the impression she was doing fine in school and their dad said it was going great, but when i went to the conference she had 3 F’s and 1 D, my youngest was failing 2 classes and my oldest was at risk of being held back for her math and lack of effort in class. since then i’ve made it a point to work with them every weekend to improve their grades and get my 2 oldest ready for the eog. my middle girl now has 2 B’s and 2 C’s and keeps improving, my youngest is at grade level in all but her writing, and my oldest is close to being on average with her class in her math now, and i am very proud of all my girls for working so hard on the weekends. i’ve asked them what does their daddy do to help them in school, and they tell me nothing, then i ask if they asked daddy for help, because he can’t know unless they ask. they tell me yes, they ask, but daddy either says he’ll help later but never does, or he tells them he doesn’t have time to help them. most weekends, i spend time with my girls getting the work done that they didn’t complete in class and doing the homework they couldn’t figure out how to do without an adults help. there’s also the bath issue, they’re not taking baths or showers regularly and several times i’ve asked them when was the last time they had a bath or shower when i picked them up from school on friday or when i’d see them for a little while on saturday and they’d tell me the last time they had one was when i gave them one the week before, and that most times when they as their daddy he tells them, no, they don’t need a bath and to just go to bed or go to their room. i’d give anything to get my girls back, and i hated signing the agreement but i also couldn’t stand the idea of yanking them back and forth in a big battle while they need to be focused on their schooling and staying positive for that. is there any chance that signing the mediation agreement will come back to hurt me in the long run, or is there still a chance i can fight for my girls and win?! Thanks!

Regardless of what you agreed to in mediation you can go to court to seek a modification.