Yeah, I think my theory seems to be more and more likely. In the past, something happened where either he had to hide or was trapped, and gave his uniform as a way to comfort him, but for one reason or another (I'm guessing an accident based on how he seems to not remember anything about this today), he ended up not coming back, and this current scenario is a parallel to that. Hope it won't be an exact parallel and will end up with him being able to rescue him, like rectifying what he couldn't do when they were kids.
Calling it right now, I bet what happened was someone else locked him up as a kid, and the other guy noticed, and handed him his uniform to act as a blanket while he tried to save him ir find help. So he ended up finding comfort from the scent of his uniform, but had trouble when for whatever reason they couldn't meet again. It is all a huge misunderstanding.