Usually Ansible lives in a Git repository. Development teams have access and raise change requests directly to devops team (!). Usually most git changes will be a few new lines. Ansible will ensure the state is consistent, and then check and apply the new state as described in the commit message. Small changes can be tested using a Continuous Integration server onto the Hadoop DEV cluster, before propagating to other environments.

Changes to Cloudera Manager or Ambari can also be driven through API requests, so that no manual task will need to be implemented. As far as security is concerned sensitive information such as passwords are encrypted and shadowed in Ansible through the vault.

This is the level at which Hadoop infrastructure should be managed, and this is how at '51zero' we are currently building and managing our Hadoop infrastructure on Landoop