Responses
Just because he doesn't fall in love with one or more girl, it doesn't mean this is not harem. The three adopted daughters, two noble ladies, two girls from magic tower. If every important characters is girl, everyone blushing when they see him, wearing unnecessary sexy clothes and do unnecessary sexy poses, you know it's harem story.

Ukh... I always hate this kind of harem. Some korean author really like trope "bunch of young barely legal girls obsessed with much much much older man who think she's his daughter or student".
I bet all his "daughter" also will in love, obsessed and will do anything for him.