Since Buddhism says there is no self, why does a new life have memories of past lives? Think you own it?

For example, the god of his previous life stayed in a woman's body, and he thought it was me; I stayed in a man's body all my life, and then I thought this man's body was my own. In fact, there is no real me. He thinks I am a wrong understanding.

In this life, because I am a child now, some memories left in my last life will be clearer, so I will say that I was XXX in my last life and have been somewhere. When you grow up, you will gradually forget the past life because of the influence of the environment.