Vivek Pathak says automation, artificial intelligence and robotics will have a transformative – but unpredictable – effect on jobs and therefore we must ensure the next generation is flexible