Why didn't he just say he was seo-woo himself?!

Probably because jaewoo spoke up first he probably was going to but missed the chance. Plus he can't really correct himself then and there for obvious reasons