It was actually the multiple Nazi salutes, constant use of Nazi rhetoric, and vocal support for Germany’s far-Right political party which includes Nazis.
Fun fact. For the 1966 World Cup, England debated denying visas for North Korea. FIFA responded that if any players from any teams were denied visas,they would relocate the World Cup, even at the last minute.