One of the men swung a sword and shouted Japanese curses at passers-by as his companion filmed him