Fuyuu's Questions (1)

Guys I need help finding a book, it's male mc and I don't remember the title of it.
The beginning is like an apocalypse where it is being lead by a superior hunter who's a girl and her like second hand is a guy who's like in the lowest rank but just as powerful. Basically they were defeated and the guy found the girl dead and she is holding a crystal that allows to travel back into the past and the guy used it.
He travelled back in the past where they have to choose the difficulty of their tower and instead of choosing the easiest one like back then, he chose the hardest and decided to guide these two/three other people inside the hardest tower. And like they are also transported back to Earth from time-to-time in order to stop an upcoming apocalypse. The last thing that I remember is there's a ghost inside his tower and has a red cape and basically guides him for a while (the ghost used to be a player but got ambushed while resting and got killed.

Please help me find it I want to read it again