Countries like Sweden, Lithuania or Greece shouldn't have immigrants. However countries like England and France who forced their language, culture and religion onto many of their colonies while also looting and destroying many of them should have immigrants. Not a hard concept.
"If you don't want immigrants you shouldn't have colonized, this is your country's punishment" is a radioactively self-defeating argument if you think about it for more than 10 seconds