It seems to me like the concern about the negative impact of knuckle cracking brought up in the literature is if it leads to arthritis. Which of course isn't the only harmful thing it could cause.

The Ig Nobel in medicine was in 2009 awarded to Donald L. Unger for research around this very thing. Unger cracked his left hand knuckle systematically for 60 years while leaving his right hand uncracked. Showing that in at least his case it didn't cause any harm. I believe he published his result here.

Beyond that there seem to be two larger studies. The first one published in the British Medical Journal titled Effect of habitual knuckle cracking on hand function shown that there are indeed some problems caused by cracking knuckles:

To investigate the relation of habitual knuckle cracking to hand function 300 consecutive patients aged 45 years or above and without evidence of neuromuscular, inflammatory, or malignant disease were evaluated for the presence of habitual knuckle cracking and hand arthritis/dysfunction. The age and sex distribution of the patients (74 habitual knuckle crackers, 226 non-knuckle crackers) was similar. There was no increased preponderance of arthritis of the hand in either group; however, habitual knuckle crackers were more likely to have hand swelling and lower grip strength. Habitual knuckle cracking was associated with manual labour, biting of the nails, smoking, and drinking alcohol. It is concluded that habitual knuckle cracking results in functional hand impairment.

The second one, published in The Western Journal of Medicine, titled The Consequences of Habitual Knuckle Cracking examined 28 people in a Jewish home for the aged, with an x-ray. It finds no harm:

The data fail to support evidence that knuckle cracking leads to degenerative changes in the metacarpal phalangeal joints in old age. The chief morbid consequence of knuckle cracking would appear to be its annoying effect on the observer.

So it seems like the the answer is that there's some harm, but it's probably so negligible it's not worth worrying about.