Okay. Allow me to clarify. For 4 years, MC slept with ML during his rut. But ML doesn't remember anything because his rut used to be so bad that he lost all consciousness/memories of what actually used to happen during rut. He doesn't even know that he spent his rut with a person for 4 years. All he remember was a sense of loss. As if something precious to him has been taken away (that precious thing is mc, his rut partner)
And as for mc. He never saw ml's face as the room used to he pitch black. His eyes were blind folded, & he wore gloves. He never had any other info about the person(ML) he's sleeping with during his rut. All this precaution/barrier so that they never form any kind of emotional connection towards each other.
So, at present time when they met, as real person, as Ideun & Jian, both of them doesn't know about that fact that they used to sleep together for 4 yrs.
Spoiler - but that connection already formed. That's why Ideun (ml) is so down bad for mc.

Agree with so many comments here. The novel is quite engaging slow burn (from Jian's side) love story. Manhwa doesn't even showing the half of the character building, incidents, or dialogues. So much is being cut off that it's genuinely getting on my nerves. In that phone call, Ideun insists Jian to take the job in his company without telling it's his company. But Jian didn't budge. So, Ideun asked him out for dinner as a way of introducing to that "friend" of Ideun whom Jian slept with. Manhwa doesn't showing how much down bad Ideun is for Jian.
If this goes on, I'm sure people who only read manhwa gonna start hating on the character without understanding them properly.