It's a mantra. Yui knows that the society thinks they are wrong but does not want to be pulled by it because he wants to take responsibility of Kei. Pills help calm turbulent mind. He ought to go crazy if he doesn't. I mean who wouldn't. Their current situation is sick AF.
This is just my inner question : the last page yui mumbled that tey're not wrong. If they're not, then why must he drank those pills..?