I just had to run this in Windows as an administrator:

bcdedit /set {bootmgr} path \EFI\ubuntu\grubx64.efi

Or, if you are using PowerShell:

bcdedit /set "{bootmgr}" path \EFI\ubuntu\grubx64.efi

GRUB appears fine on boot and I can boot into Ubuntu and Windows 10 fine.

As pointed out in the comments (by David Faure), the final argument to this command depends on your particular system configuration. You can use this command to determine the correct path for your system:

bcdedit /enum firmware

This will list all the installed boot managers and associated EFI paths, and grub should be somewhere in the list.