This is so not the case.
If you think about your own life and how it's shaped who your are, you know that it's all a culmination of what your life was like before. Yet society expects children to forget about their lives before their adoption. "But he's so much better off" or "How can he possibly love those people" or "Why would he want to remember that" and so on.
I can't tell you how many people ask me if Jason ever talks about his life before us. If he ever mentions his brother, foster family or birth-mom. And when I say yes, we talk about them a lot actually, I get some very shocked looks. Yes, he was only 4 at the time, but those 4 years are pretty powerful. More happened in his first 4 years than happens in most adults first 20. He can't forget them or bury them and to do so would be dishonoring Jason and his past. And expecting him to would be telling him that we only love part of him, not all of him.
All of the experiences a person goes through in their life, whether young or old, shapes how we see the world, interact with it and cope with what life hands us. The same is true with children who are adopted. And to expect different if to say your life didn't matter before your life here started.