There are two different kinds of things called a "router", with two different pronunciations, originating from two different verbs "route" and "rout". The confusing part of this is that the two pronunciations overlap the two different things.

A router as above (computer thing, from verb "route"), or anything else which routes something, is "rooter" or "rowter" depending on how you pronounce "route" (US English has both "root" and "rowt", British English has only "root")

A router (woodworking tool, from verb "rout", an electrical one is shown above) is "rowter", never "rooter", in both the US and the UK and other countries. The dictionary entry may be referring to the wood tool only.

Source: Oxford Online Dictionary & Cambridge Dictionary