Why did the old man say; What about the saying that men are afraid of midnight and women are afraid of noon?

In the past, because of feudal ideas, many people believed in fortune telling, and it was generally believed that a person's fate could be seen from the date of birth. For example, this proverb called "Men are afraid of midnight and women are afraid of noon" is believed by the older generation, so what does this proverb mean?