Out of curiosity, where did Musashi's mom go?

I asked myself the same damn question and also about sensei's family.

I'm going to guess either she died or left him since he was a gangster