"born from an affair" was random lowk

I think it was mentioned in the earlier that he's son of the mistress

unless you mean the phrase itself?

I mean thats probably what he hears all the time since he was taken in so that's probably a one of the roots of his low self esteem

orrr I'm sure he also think joohwan likes him being chaebol heir and whatnot