Responses
He obviously cares about Ilwoo, but it's a multitude of things. The most prominent thing is that needs to get out of that town so be can escape his father. The other bastard can do that for him, but Ilwoo can't. And he's terrified. It's also that the bastard had been with him the longest and through everything, so there's also a sense of security and safety, even though he's abused by the bastard.
I don't know what exactly baekhee was thinking, whether he wanted to protect ilwoo by keeping him away from gogyeom, or he just really doesn't care about him? At least ilwoo can fight gogyeom to the death even though he doesn't have money