The queen poisoned him. Or someone close to the queen

Spoiler
.
.
.
.
.
.
.
It was his maid. Minor character maid Otua, if I remember correctly. It was a maid from the Empress's side that poisoned Perez's food when he was little. She changed ships after realising that Perez is safer way for her to survive and practically became a double agent. She will be the core witness that the Empress ordered the poisoning.