I already left a pretty lengthy comment about this, but Yayima is doing the vast majority of the work in this relationship.
Sure, Torikai confessed first, but after that? Yayima is the one who asked him out on an actual date, planned more dates after that, gave him a house key, took him to his favorite restaurant, celebrated his accomplishments, and even suggested they move in together. At nearly every major step, it's been Yayima taking the initiative.
To the point that even Yayima points out that the only thing Torikai seems comfortable being vocal about is wanting sex.
https://www.mangago.me/read-manga/contradict/uu/br_chapter-354985/pg-4/
Take in mind, at the time they had a fight over the promotion it was also torikai who didn't communicate.
And honestly, the whole "he hasn't said 'I love you'" argument doesn't matter much anymore in my opinion. It's pretty clear, thanks to his awful ex, that even if he had said it, Torikai would still be treating the relationship like it's destined to end. Not if it ends, but when.

People who are picking sides on who's winning the "worst at communication" award between these two are missing the point that it's a joint reward, tbh. Torikai a fuckin' wall who struggles to open up cuz of past trauma, and has to be pried open. But like...for all he's doing, Yajima hasn't even said a direct "I love you" yet. Of course Torikai's gonna struggle to accept it. Anyways, I'm locking them in a room till they talk about their feelings. Communication is key.