While tourist locations across Asia, Europe, and North America have been going all out to attract Chinese travelers for the 2015 Lunar New Year holiday, mainland shoppers have received a much chillier message in Hong Kong—namely, angry locals literally telling them to go home.

Over the past month in the lead-up to Chinese New Year, the popular cross-border shopping destination has seen several major flare-ups of anti-mainland sentiment at some of its major shopping centers. Protests ostensibly aimed at curbing “parallel trading” (cross-border smuggling to avoid tariffs) have upset and intimated mainland shoppers, with demonstrators chanting “mainlanders go back to the mainland,” using the derogatory term “locusts,” and even getting involved in violent altercations.

While it’s not clear whether this small yet enraged group of demonstrators prevented any smuggling, they did deter mainland Chinese visitors for the holiday period. In the first five days of Chinese New Year, the number of visitors from the mainland has decreased for the first time in 20 years. According to Hong Kong’s Immigration Department, Hong Kong saw 675,155 arrivals from the mainland between Wednesday and Sunday, marking a 0.16 percent drop from the same period last year. The change from last year is dramatic, when numbers rose by 13.7 percent over the same time period.

Cable TV footage on anti-mainland parallel good traders march in a shopping mall in #HongKong. #9wu #umbrellamovement pic.twitter.com/h5uBqgN0is — Fion Li (@fion_li) February 15, 2015

These disappointing numbers come at an especially bad time for Hong Kong’s high-end retailers after a turbulent 2014, and have many wondering if they should “give up” on the region as a major source of growth. Many luxury brands had previously depended on Hong Kong for significant sales as mainland shoppers flocked across the border to avoid high tariffs at home—Hong Kong is estimated to take up to 10 percent of some luxury brands’ total global sales, including 8 percent of all sales for LVMH’s brands.

This trend has also caused the flood of the “parallel traders” that have prompted much of Hong Kongers’ anti-mainland sentiment. While this attitude is widespread in Hong Kong, the demonstrations themselves have been limited to small groups. Many locals express frustration with sold-out baby formula and crowded transportation, hospitals, and stores, but the protests themselves haven’t reached more than 200 people.

While minuscule in comparison to Hong Kong’s massive student democracy demonstrations in the fall of 2014, these anti-mainland demonstrations are likely to be much more damaging to Hong Kong retail sales in the long run. Many major luxury companies posting lackluster Asia-Pacific sales for 2014 have been laying the blame on the Occupy Central protests: Richemont, Prada, Gucci owner Kering, and Hong Kong luxury watch retailer Emperor Watch & Jewellery have all recently cited the student protests as reasons for disappointing sales numbers. While there’s no doubt that sales saw a drop for many brands during the student demonstrations, Hong Kong mainland visitor growth and retail sales had already been declining for months before the fall protests due to ongoing anti-mainland sentiment. In February 2014, around 100 protesters stormed the city’s Tsim Sha Tsui luxury shopping district to harass mainland shoppers long before the Umbrella Movement was underway.

While the students had occupied spots in main Hong Kong shopping districts and have now begun staging shopping mall walkthroughs carrying symbolic yellow umbrellas, they have distanced themselves from the anti-mainland groups which set out for malls to harass mainland shoppers. On February 8, one of the anti-mainland demonstrations turned chaotic, resulting in violence, police use of pepper spray, arrests, and store closings. While still small, the protests appear to be gaining steam: a recent South China Morning Post report found that 274 people were signed up for a scheduled Sunday protest calling for the cancellation of a multi-visit tourist visa to Hong Kong for mainlanders.

It’s not just protest chaos that has put a dent in retail sales over the past year, however. Mainland China’s anti-corruption campaign is likely having an effect, since sales of “gifting” items that have been hurt on the mainland such as watches have suffered in Hong Kong while some sectors have remained relatively healthy. In addition, a stronger Hong Kong dollar against the yuan means that mainlanders have less price-related incentive to shop there as opposed to Europe, where the euro is relatively weak.

Retailers eyeing the situation should expect a continued diversification in shopping locations as Chinese tourists head abroad to Japan, South Korea, Taiwan, or Europe. South Korea and Japan have been doing especially well over Chinese New Year: Seoul department store Lotte saw 74.9 percent year-on-year Chinese spending growth for Chinese New Year, while Seoul’s Hyundai luxury fashion sector saw 72.4 percent growth. Those worrying about whether they should “give up” on Hong Kong shouldn’t write it off altogether: Hong Kong still saw 16 percent growth in the number of mainland tourist arrivals for 2014 remains mainlanders’ easiest place to access for tariff-free shopping. That being said, it can no longer guarantee an easy path to sales growth for brands, which must stay up-to-date on Chinese tourists’ newest travel and shopping hotspots.