Spoiler...
That's actually right. He forgot how his manager looked like. He just remembered the silhouette and personality which is similar(not same) to garam. He tried to find pictures about his manager, but couldn't as his manager always liked taking photos instead being on them. He cared for his manager but dw, he didnt love him romantically. Though in the current raws(till where I read) there fight scene hasn't been shown but the fight was indeed massive and right after his manager left, he did in a fire accident due to a short circuit. So he feels guilty and lonely about it that the only one who cares about him left and wouldn't come back.
Though he is technically right at the end, everyone does leave him. Even garam, keke. That's enough spoiler!

So I think the manager doesn’t actually look THAT much like Garam. I think the professor is seeing the managers as a blurry sort of Garam because he cared so much for the two. He hasn’t seen the manager for at least 10 years and given how things left off and the grief of his death, I doubt the professor looked at very many pictures of him either.
So when he finally meets someone who cares for him like the manager did, he keeps having memories of the past, but can’t quite remember what he looked like, so the memory of the manager ends up taking the form of Garam. The person the professor is currently starting to care for/ fall in love with
Just my interpretation. If we ever get an actual picture of the manager and he looks exactly like Garam I’ll eat my words I guess