He has to fully commune with the memories to unlock their full power, there might be a penalty if he just half-asses it and have too many unfiltered memories inside him.
I think his story balances out because he has to fight those above him every time and also the people that want him executed are also cheats within themselves
My point is that he is fighting with only one style, as the girl the assassin or as the knight, he doesn't make "his own fighting style" it's like copying others...not being himself. And now he has to find a new sword to spread his wings because him training with the knight's memories who was at maximum level of a walker can't accomplish the wings?

I feel like MC growth can only progress with others memories and can't help but think he is cheating