While imperialism may result in some benefits for those under rule, weighing up individual cases of good and bad misses a crucial point about the structures of empire