1. ReLU

A great default choice for hidden layers. It is frequently used in industry and is almost always adequete to solve a problem.

Although this graph is not differentiable at z=0, it is not usually a problem in practice since an exact value of 0 is rare. The derivative at z=0 can usually be set to 0 or 1 without a problem.