I'm so happy for you Hyunho, you're growing some balls.
In my perspective Hijikata met future-Gin and future-Gin made the effort to comeback after 5 years to that specific time so that Hijikata won't be alone on his birthday. That's why Gin knew when it was gonna rain even though there were no news that there will be rainfall, and why Gin suddenly talked about the reason why he said that 'mean' thing when Hijikata came to visit Gin.
I don't think Hijikata was alone nor it was a figment of his imagination because the umbrella Gin gave Hijikata was still being held by him.
It's funny how we all agree that Zombie Man is a hot thot ლ(´ڡ`ლ)