(See my previous post on your first post about initiation and admissibility.)
Yes, phone records can be used and subpoenaed to show that unusual contact occurred between the parties in question. (Thank God, because it’s one of the ways I’m able to defend myself against a CC/AA accusation. There was virtually no phone contact between myself and the person I’m accused of having an affair with prior to their separation.)
Probably what would happen in your instance is that he would have to pay an attorney a sizeable retainer. That attorney would first send a threatening letter to try and force you into a settlement. Then there would be the request for your financials and other such information. Chances are that once he saw that you had no money to get, he would encourage his client to drop the case. However, if out of vindictiveness the client chose to go through and bear the full cost up front for getting a judgement, it could still go to court.
And, yes, as much as I don’t condone having an affair, it is pretty much a form of legalized slavery via extortion at this point.