Call him out. There is obviously a problem that has to be dealt with. If he refuses to talk, you obviously have no hope of communication; and relationships can't exist without communication.
If he is willing to talk but unwilling to fix things, you still have no other recourse but to end the relationship.
If he's willing to talk and fix things, then you should be able to figure out what to do.