It's just as simple as the rest of the story. He thinks about that in a moment of extreme exhaustion and sadness and loneliness (when we all say stupid stuff, even worse than this) and he deflects it immediately because he's not in fact alone anymore. The rest of the episodes are great and they made a lot of progress in their relationship so it bothers me that people are complaining (not saying bitching so nobody is offended) over a single panel that is only an intrusive thought he declines and is part of his development as a character. His mind is all over the place because the mother that abandoned him is dying and the father who doesn't call him at all calls him to be in charge of her soon-to-happen funeral.... Sorry if his mind plays him dirty for over a second.
To be honest if I knew the receiption I wouldn't have upload the chapters at all. They were released like two months ago and the fandom received them with happiness and cheers. There were no toxicity nor complains over such a "small potato" (yelena belova dixit). No war, which is so refreshing sometimes. There are two more episodes (six in this format) released in japanese and there is still more development in their relationship in there.
The fact that his first impulse was to consider being unfaithful—as a result of mental exhaustion—says a lot about him. That doesn't make it "more understandable," because it simply demonstrates that he doesn't trust his current partner enough to call him in the first place; if he feels so lonely and abandoned, why not call his partner?

I'm really pissed off because I uploaded chapter 81 like 2 hours ago and nobody approved it. The last dialogue is part of a longer scene that was mid-cut and now everybody is complaining over a piece of monologue that is not even finished.