You know I keep thinking about this and I believe its a typo. He does not look 50 but more like his early 30s. Still a gap but not as bad. He married his older brother's fiancé, so the Duke could have been 18 when they wed. Even looking at other characters you can tell they are middle age but he is definitely not one of them.
No one has to agree but I also like to think FL is 19 or 20 to stretch time between after graduating from the academy, finding out she is infertile, going through the stages of grief for a year, then corresponding with the prince before his passing.
Especially since, back then, people died between the ages of 50 and 60
She mentioned when she saw him at the beginning that his age was around her father.
I am aware but the parents look much older that the Duke to be "around their age." Again I am going with a typo. In my mind he is in his early 30s not his 40s or 50s otherwise he would have lines around his cheeks to indicate aging. See the other elder characters as a reference.
Although the age gap is a bit concerning, given the fact that, when this story is plotted in the timeline, in the medieval era, this was a usual occurrence, this wasn't morally wrong or ostracized. We cannot put a story set in the medieval era to be judged based on modern moral ethics and our understanding. I think if we look into that with this type of perspective, then it seems understandable.