This couple has a serious communication issue...

The reason behind his rejection to spar with her is probably: he doesn't want her to evolve into his rival. That'd be repeating the same mistake as his previous life.
The man is so pitiful, life after life his goal has always been only to have her. He obviously did something in his previous life so that they're restarting it now (in an alternative universe?) since he said he gave up everything for her last words--though there is a miscommunication somewhere there too, his idea is to possess her but her idea is to become his knight.