Squinting in the midday sun at Union Square Park, Jon Batiste picked up his melodica, the mouth-blown reed instrument and keyboard that he calls a “harmonaboard.” As a small but enthusiastic crowd formed, he began a loosely swinging version of “My Favorite Things” with members of his Stay Human band playing instruments including tuba and tambourine.

Little about that scene earlier this month suggested Stephen Colbert’s announcement five days earlier that, when he takes over as host of CBS’s “The Late Show with Stephen Colbert”...