In short, he was afraid of being torn apart in the school jungle, of being beaten by other ppl just like Taesan. And since he DID go through all that sht cuz of him being an idiot, he wanted to keep distance, then realised Siwon loved him just like Junwoo liked Taesan,
Im so confused, someone PLEASE explain to me in more details #-.-)