It’s hard to say if he ever regretted his past actions (at least his past actions from this series) because it’s never explicitly stated, but then there are still 2 whole sequels after this that show the evolution of their relationship.
So, no, I think the only time he apologized was in Seringe, then in a side story (I don’t want to spoil what he did to make him apologize, but it had nothing to do with sex, though).
Also, at the end of the sequel series, he confuses Tae with Jae and his actions there do weight heavily on his mind.
Overall, Ilay is still Ilay, but he does truly care for Tae (in his own way).

Novel readers! Did ilay at some point apologize for this arc?