Its not actually alternative timeline where Hardy becomes noble. Hardy IS a noble. His actual name is Lionel Parco, second son of duke of parco. Now the current duke of parco is imperial administartor
The story actually reimagines as Abel and Hardy met as nobles rather than Abel meeting Hardy as stagecoach driver
Nope, Hardy's family did not "fall", he got kicked out from his family. Or more accurately, his brother arranged for him to become a commoner as the rumors and talks around his failed marriages and his father's punishments is taking a toll on Hardy
And as for "Alternative Universe" thing, many people here dont really dont whats going on (except for people who read through the raws with translation) and some assumed that Hardy is cheating in current timeline after all the things he did to conquer Hardy and so the context is not really clear
If I read your comment without me going through the raws, with the story currently available telling me that Hardy is a commoner who worked as stagecoach driver, I would have assumed that its a different timeline where Hardy was a noble
However Hardy IS of noble origin and him at gambling parlor drinking and gambling fits in the timeline as he indulged in these to ease the pain of his multiple marriage failures. So for me its more likely appear as alternative timeline rather than alternative universe
I think the Hardy's family "fell" thing is most likely told either by Hardy himself or assumed by Abel as the Parco dukedom is far away and Abel dont really know the true background of Hardy. Just that he came from fallen noble family and is working as stagecoach driver as a commoner.
What's weird is that Oscar knows the full background of Hardy and he could have told Abel about it, but he chose to hide it (initially) as securing Abel's position as duke is more important for him than to tell Abel about Hardy.
Oscar is the villian of this story, he knew about Hardy but chose to ignore it and used him as a chess piece to secure Abel's footing while Hardy blindly trusted Oscar as his ally and friend (along with another scum, Gray)
*Abel cheating in current timeline after doing all sorts of things to conquer Hardy (my bad, made a mistake)
With all that said, three people are primely responsible for Hardy's current scenario
Gray for manipulating others into avoiding Hardy and breaking off all of his engagements
Oscar for failing to protect him from his Master's cruel whims when he shamelessly styled himself as Hardy's brother
Abel for forcing himself on and into relationship and transforming his body without his consent (his love is truly cruel - Hardy says that in ch 28 which is even more ironic since at that point he didnt even know about the transgressions done by Oscar and Abel)
Now Hardy trusts no one and is now relying on Gray which is exactly what Gray wanted. If he learns of Gray's transgressions too, it'll be too much for him to bear

I checked the raws to see what all this fuss in the comment section here is about and... HOW COULD ANYONE MISTAKE AN VERY OBVIOUS AU BONUS EXTRA ABOUT "what if hardy was still a noble when they first met" AS ABEL CHEATING??? IT'S VERY OBVIOUSLY HARDY AS A NOBEL, IT'S VERY OBVIOUSLY ALTERNATIVE UNIVERSE BONUS CONTENT AFTER THE SEASON FINALE, SO CALM YOUR TITS EVERYONE AND DON'T GET ALL OFFENDED OVER SOMETHING THAT ONLY HAPPENED IN YOUR OWN FANTASY
....like y'all act all so offended but it's so obviously just AU bonus material that it makes me feel you want him to cheat bc I literally can't understand how anyone could look at that and just NOT instantly realise it's an AU bonus content... Like WTF???