Following up on the earlier answers by Ilythya and john crow, I did some research into the theories that ham (in the actorly sense) derives from ham fat or from hamfatter (or from both).

Ham fat or ‘The Ham-fat Man’?

John Ayto, Word Origins, second edition (2005) includes this comment in its entry for ham:

Ham in the sense 'performer who overacts', first recorded in the late 19th century, apparently comes from an earlier hamfatter 'bad actor', which may have been inspired by the Negro minstrel song 'The Ham-fat Man'.

This bolsters Ilythya’s answer; but john crow’s answer finds support, too, in Martin Harrison, The Language of Theatre (1993):

The most plausible derivation, with a more detailed and credible etymological story, is that in C19, before the invention of Leichner make-up, powder make-ups were combined with some form of grease or oil before application. Amateurs, or actors on a low income—that is, those who tended to be inferior—were forced to employ cheaper substances (rather than the professionals’ sophisticated oils) to apply their make-up, hence the nicknames ham-bone and ham-fat(ter) from the fact that they used ham-rind and other unpleasant greases as a medium. Ham-fatter may come directly from a negro minstrel song, ‘The Hamfat Man’.

Michael Macrone, Animalogies (1995) brings the “ham grease” and “Hamfat Man” theories together in a somewhat different way from Harrison [combined snippets]:

The title [“The Hamfat Man”] refers to the minstrels' use of hog grease to clean the black off their faces. It's very likely that the tune inspired the phrase ham actor or ham for short, although such epithets as hamfat man or hamfat actor may predate it. In any case "ham" appeared on its own by 1882, when someone referred to himself, in a letter to in a letter to Illustrated Sports and Drama News, as "no ham, but a classical banjo player."

So one source says that ham fat was used to blend with powder make-up before application, and another that ham fat was used to remove blackface makeup (presumably burnt cork) after application.

Earliest occurrence of ‘Hamfatter’

According to Iris Blake, “Burlesque: Music, Minstrelsy, and Mimetic Resistance” (2013), “The Ham Fat Man” was an 1863 minstrel song show, which also (she says) is the first song to include the words “hootchy cootchy,” albeit in a primitive phonetic spelling. The chorus of the song:

Ham fat, ham fat, zigga zolla zan, Ham fat, ham fat, Tickle olla tan; oh! Walk into de kitchen, as fast as you can, Hoochee Koochee Koochee, says the Hamfat Man

Early notice of the term hamfatter appears in John Farmer, Americanisms Old and New (1889), where the term doesn’t refer to performers of any kind:

HAMFATTER. —A recent name in some quarters of New York, for a second-rate dude or masher, and more especially applied to the habitués of the Rialto in that city. [Example:] I’ll warrant that these ladies who complain have, if the truth were known, strolled up and down Broadway by the Fifth Avenue Hotel and the Hoffman, and were they so fortunate as to receive an admiring glance from the well-dressed and more prosperous professional brother of the HAMFATTER, they were not offended, forsooth. —New York Herald, July 29, 1888.

Another source indicates that in New York City the term hamfatter was applied colloquially to non-actors by 1879. From George Sala, “All the Fun of the Fair,” (December 3, 1879), in America Revisited, sixth edition (1886):

Every American who does not wish to be thought “small potatoes” or a “ham-fatter” or a “corner loafer,” is carefully “barbed” and fixed up in a hair-dressing saloon every day.

However, in “Spangles and Sawdust,” in the Nashville [Tennessee] Union and American (November 6, 1879), a “flying trapeze” artist tells the reporter that the term refers to second-rate performers in his profession:

”This is the first [circus] show I ever left in this way. I traveled with Forepaugh’s establishment four seasons, and never had any trouble. I’ve been with this show since the 12th of June last, having joined it at Clinton, Iowa. When DeHaven proposed this concert business, I told him I was no ham-fatter, and—“ “Ham-fatter?” “Yes, ham-fatter. That’s the name we give a man in our profession who is a poor performer. I’ve been in the business since I was ten years old, and I’m a little over twenty-five now.”

It’s difficult to think of a form of entertainment where the idea of applying ham fat before a show seems less suitable than in a trapeze act—but that consideration evidently didn’t prevent the minstrelsy term hamfatter from becoming circus slang, too.

The first instance of a performer specifically promoted as a “ham-fatter” appears in an ad for the Vaudeville Theatre in the San Antonio [Texas] Light (April 8, 1884):

Then comes the noted ham-fatter, CHARLIE FRYE. “If you can guess what he is doing you can have it.”

The same ad mentions that Frye will participate later in the show as a character in “The exceedingly laughable Ethiopian interlude, entitled, Nitro-Glycerine!” which is evidently one of the high points of the evening.

Other uses of ham fat in popular entertainment

From Rosemarie Ostler, Let’s Talk Turkey: The Stories Behind America’ Favorite Expressions (2008):

According to tradition, ham fat was part of the equipment of old jazz musicians. A 1966 New Yorker piece on Louisiana music alludes to this claim: “Most of the musicians playing in these clubs are old men…. They’re hamfat musicians. In the old days, the rough musicians kept pieces of ham fat in their pockets to grease the slides of their trombones.” By the time [George] Sala used the word in his book [in 1879], it had become synonymous with a third-rate performer or other low-class person.

Conclusions

I was surprised that multiple sources endorsed the theory that ham fat as preparation for makeup (or as a post-performance cleanser of makeup) inspired the minstrelsy song “the Ham Fat Man.” An alternative theory might be that “Ham Fat Man” was simply a stock minstrelsy name—like Tambo, Jim Crow, and Mr. Bones—that had no reference to practical uses of ham fat on the stage.

In any event, the argument that “ham actor” is derived from hamfatter seems quite strong, even though the term hamfatter was also used in connection with forms of entertainment besides stage acting, and indeed was sometimes applied to people who were not entertainers at all.

UPDATE (3/24/2016): I consulted a new (to me) glossary of theater terms to see what it might add to the discussion. Here is the entry for ham in Don Wilmeth, The Language of American Popular Entertainment: A Glossary of Argot, Slang, and Terminology (1981) [combined snippets]: