Existing studies emphasize that the neoliberal ideology creates anti-immigrant attitudes. They attribute this to the influence of radical right-wing parties, which combine a pro-market ideology with authoritarian social conservatism. However, this claim has not been fully tested. To understand the mechanisms behind this association, our study analyzes data drawn from a representative survey and an online survey experiment conducted in Japan. Our results demonstrate that an association between neoliberalism and anti-immigrant attitudes exists even where radical right-wing parties are absent. Furthermore, the results of our experiment, where immigrants' skill levels and country of origin are varied in the vignette, show that respondents espousing neoliberal ideology are sensitive to the skill level of immigrants in that they strongly oppose low-skilled immigrants, while welcoming high-skilled immigrants. These results suggest that the association between neoliberalism and anti-immigrant attitudes is not simply a result of the influence of radical right-wing parties but rather stems from concerns over their future welfare burden.