Never, it is never justified. To do so you will be committing adultery and it doesn't matter what he has done to you. That is on him. You just need to worry about your soul...