I think you really misunderstood the ML and maybe missed a few dialogues that explained the why of his actions. The ML is actually not only super understanding but also very smart and not driven by feelings but by wits. He’s obviously a good emperor for a good reason.
One thing that we can say is driven by selfishness is him asking MC to come with him. But in no way did he “not care” for how he felt. He knew that MC didn’t especially care about going or not so there was no feeling to ignore. The only thing MC cared about was for his foster dad to be safe and to be disowned and ML knew about it even before he told him, having already ordered the foster dad to disown MC. After all, ML also cares the most about MC’s foster dad.
The ML didn’t “watch it” happen when MC was slapped. He waited. Why? Because he cannot, as an emperor, just go burst into there. Why? Because as an emperor he represents the whole country and has a duty. That’s why when the eunuch asked him if he was going personally to see them he asked “which side should I chose?”
To protect my wife, an imperial concubine, the first pregnant one?
Or a lowly artist with no background?
And that’s why he followed up and said “if she is smart, she will punish her servant”. Why? Because even though he cannot ax publicly, she knows she would lose his favour if she acts against his people in such a shameless way.
You need to pick up the clues in the dialogues because there are plenty and they explained extremely well his way of thinking which is absolutely just. If he had darted there mindlessly, it would have started a rumour or have done even worse and as an emperor, he has to hold not only an image for the country but also protect the dignity of the mother of the country’s future prince/princess.
Also the thing about “begging” is that it was a sudden thought he had and it made him laugh. And then he thought about it again later when he realized it would indeed never happen and then laughed again. It’s just to show that while he is smart, he is also just silly af and looking for fun
I love the ML of this story because of how intelligent and fun he is. He is also very romantic and silly but an extremely smart ruler. I personally prefer an emperor who knows what his job entails than just the basic fake emperor who in romance stories just has the title but not the duties.

I think the ML is kind of selfish. He could clearly tell that the Mc agreed to go to the palace, not because he wanted to but because he couldn't go against the emperors wishes. And the ML just didn't care how he felt. Also it annoyed me so much the the ML didn't want to help the Mc deal with his concubine at first because he wanted to the Mc to beg him for help. And yes I know he helped him like the very next chapter but it's still messed up that he waited. I don't really hate the ML but he's can be really unlikable at times. Hopefully he will change with time, because I'm going to be so mad if the other concubines start messing with the Mc (which will happen eventually) and the ML just watches it happen again.