What if a soul did have a supernatural backup of my memories? It seems silly to think that a non-physical "magical" entity should keep an exact duplication of an extremely complex and somewhat error-prone system. The human brain is robust but far from error-free. If my soul corrects my memories as it stores duplicates of them, or doesn't mimic the way my memories fade and change slightly with time, then it cannot be me. Any life that uses those pristine memories would fail to understand most of the things that its previous host had ever done. It wouldn't understand the person it had become, so most of the memories would be useless - events without a personal context in which to frame them. Any soul that outlives my physical body and keeps my memories has to take with it a copy of my flawed memories and blindspots.
Maybe my soul will continue only as a stream of consciousness with no memories at all or with jumbles of incomprehensible memories. This soul would have to start again on another plane of existence or with a new earthly incarnation. Without the benefit of preserved learning and understanding from its previous life it has no concept of self, no communication, no frame of reference. It can't think in terms of "Wonderful! A whole new life to enjoy!" if the being it has become can't remember its previous incarnation. The only solace provided by this idea of soul is that it stops you having to think about what it must be like to not exist at all. But it really is just as hard to think of what it would be like to know nothing and understand nothing.
Life beyond death with only jumbled incomprehensible memories has at least some correlation with reincarnationist ideas. (It's when you hear about a person having memories of a past life as a mollusc that you have to question the source of the jumbled memories.) Religions that promise punishment for the souls of those who have disobeyed really require a soul to have a fair memory of a past life. Imagine enduring punishment as a soul with no clue of what you'd done to deserve it. Once a being has no knowledge or understanding of past sins what is the point of punishment? It can't be a corrective tool. How can a being change its ways when it can't remember them? Such punishment can only be for the vindictive pleasure of the punisher.
I lost my soul when I realised it didn't make sense.