The Justice Department is investigating whether a group representing some top technology firms is unfairly trying to smother a free rival technology for delivering online video that is backed by Google Inc., according to people familiar with the matter.

Much the way firms battled in the 1980s over VHS and Betamax video formats, tech rivals...