I know its a little too late to ask..

The prince was originally a regular guy who transmigrated into this ancient-era style world. The black-haired person seen in earlier chapters (now possessing the body of the orange-haired woman) supported him from the shadows with his system, serving as his gold finger and occasionally taking over his body to clean up after his mistakes.

Being as the prince's power and achievements came from that silent support in the shadows, he called him "Grandfather" to show respect, and the black-haired host calls him "Grandson" because he's much older than him, having gone through different worlds but keeping the same young appearance.