Yes! People need to understand that a bad character having an uncomfortable story isn't to magically get him forgiveness. Something HAPPENED to that character to make them the way they are, and it matters just as much as what put the female or male lead in their situations. He isn't the abusive bastard ex husband for no reason, unlike what we see so often in fiction.
Instead of being used as a quick, cheap tool to make the male lead look better than he is, he plays a unique and important role in the story. It's uncomfortable, but it's important. This didn't happen between him and Na Haesu is 5 years, or 10. This was nearly 20 years of suffering through this. That is a fourth of most peoples lives.
Honestly people who say that are very shallow, in my opinion. Because even if he had gone through that it doesn't excuse him being an asshole now. He was naive and innocent back then. Used and taken advantage off. I could excuse that but now it was a series of continues decision that he turned out an asshole, and that's entirely on him.
A person can have a terrible upbringing and still be a good person. At the same time they can still be a terrible person too.

I feel like a lot of people keep talking about how they shouldn't show this and they're just trying to get us to forgive his wrongdoings.
But I feel like you can sympathize for something people have experienced and see why and how it has come to this without forgiving him.
It's just part of building his character and characters like this, that are actually planned out and make you feel mixed emotions, are the best.
Do I feel bad for him and sympathize with everything he's gone through? YES
Does it validate and excuse his actions? No, no it doesn't.
Does it make me feel bad and want to forgive his current actions? No, everything that's happening now is a result and consequence of the actions he choose to do and continues to do.