Show your villain personally doing something despicable.

A villain that does nothing but sit on a throne (or peer down from the top of a giant tower) is hardly a villain at all. It's hard to feel even remotely intimidated by a guy or gal who just sits around giving scary orders all day. A perfect example of this is Galbatorix in the first book of The Inheritance Cycle, whose most villainous acts are ordering the destruction and torture of some terrorists (because yes, that's what the Varden are), taxing the people, and destroying some villages we didn't even care about.

Don't make the same mistake. Put some grime on your villains' hands. If they don't personally torture someone, at least have them watching in approval - or better yet, interrogate the protagonist.

Have the villain do something awful to something or someone we care about.

If we aren't emotionally invested in something or someone the villain harms, we aren't really going to be affected when something bad happens to it. We might know on a rational level that what the villain did was wrong and awful, but that's all we'll feel - a sort of "oh, that's so awful!" response in the detached sense you'd feel about a car wreck killing a family of strangers or hearing about some random kid being bullied.

Rationally, we know that Grand Moff Tarkin destroying Alderaan to show off the power of the Empire and spite Princess Leia was utterly despicable. But there's little to affect us emotionally. Because we never saw Alderaan, let alone got to see and become attached to anyone who lived there, we don't feel any sense of loss. We also can't properly empathize with Princess Leia's loss, either, because we have no idea what she lost.

Voldemort may have been Harry Potter's big bad (and there's no denying that he was about as bad as they come) but Voldemort's evil acts often happened off-page or to people we didn't have any emotional investment in. Yes, it's sad and tragic that Harry lost his parents, but we never knew them and so we don't feel any sense of loss. Compare with Dolores Umbridge - she tried to expel Harry from Hogwarts, banned him from the Quidditch team, and tortured him in detention. By then, we cared about Harry as a person, and we knew what Hogwarts and Quidditch meant to him. Dolores Umbridge, for the record, is hated and despised more than Voldemort.

Sauron in The Lord of the Rings fails to scare anyone because he's either a vague force of menace (in the books) or a floating eyeball atop a tower (in the movies). It's the ring you come to loathe because it constantly puts Frodo into life-threatening danger and threatens to corrupt him and those around him, coupled with the agony and torture Frodo has to endure just to destroy it.

It's very easy for audiences to brush off random people being hurt or wronged. Kill off thousands, even millions of faceless people, and most still won't be affected too much. But hurt or kill one person they've come to care about? Then they'll get hurt and angry.

Make sure your villain has a believable motive or agenda.

First of all, unless you don't intend your work to be taken seriously, you should never write a villain who does anything Just Because It's Evil. Nobody actually works that way.

Check your villain's priorities. While there's nothing unrealistic about someone whose priorities would seem really messed up to other people (such as putting higher priority on settling an old grudge than saving the life of a family member), you should generally always ask yourself if the villain could realistically be making better use of xir time/money/effort to reach xir goals, and if so, then why isn't xe?

Upon occasion, I see villains who do things with pretty much no motive at all. I don't mean characters like the Joker in The Dark Knight, who are clearly in it for the trolololz. I mean villains like Jerry Jenkins and Tim LaHaye's shadowy Luciferians in Left Behind. For whatever reason, the evil shadowy Luciferian group spends billions funding the construction of a new Babylon in Iraq - and they actually call it New Babylon. Realistically, the whole project would be a colossal waste of time and money. Aside from there being no real need to build a new city, New Babylon's desert location is a completely impractical place to put a city. Plus, the city of Babylon has pretty much no cultural or religious significance to the vast majority of people alive today (including Luciferians), so why try to revive it at all? Long story short, the only reason this is done is because LaHaye and Jenkins believe the Tribulation needs a literal Babylon. Don't be these guys. Don't have your villains do something for no other reason than it just "has" to happen.

Also, mental illness should never be substituted for a real motive. Aside from the fact that it can be very offensive, it's just lazy. Even people who do villainous things under the influence of a mental illness have a motive - for example, Jared Lee Loughner, who was declared mentally unfit to stand trial after shooting Rep. Gabrielle Giffords, had a motive - he hated the government, and he especially hated Giffords after he felt she had evaded a question he once asked her.

Avoid bargain bin Nazis.

First, there's nothing wrong with taking inspiration from the Nazis. They have a lot of horrendous stuff under their belt that makes for absolutely atrocious villains: brutally killing people based on race, sexual orientation, ability (getting gassed to death was the nice option - many victims weren't so lucky), working people to death in in concentration camps, performing horrible experiments on people, and using mangled religious beliefs, pseudoscience, and outright lies to try to get people to go along with it all.

And then there are the bargain bin Nazis. You're supposed to hate them based on their apparent similarities to Nazis (and sometimes they're actually supposed to be Nazis)... except the similarities are primarily superficial - eg, similar uniforms, marching, flags, similar methods of saluting, etc. The evil plans of the bargain bin Nazi have little to do with actual Nazi belief and practice, and instead mainly amount to a generic "kill/exploit everyone I don't like and RULE THE WORLD!" plot. The core problem here is that the writer is using fairly superficial things to mark the villains as bad instead of really properly showing the villains doing something bad... or bad beyond a generic evil plot.

If you're going to use Nazis as villains, use Nazis. If not, don't use superficial Nazi trappings as a shortcut to make what essentially amounts to a generic world-dominating villain look extra-evil. Your villain's evilness should be able to stand on its own merit, and watering the horror of what the Nazis did down to "kill/exploit people and take over the world" is just tacky and distasteful.

Don't have your villain talk like a cheeseball.

Awhile back, I noticed that when some people tried to write villains, they would lapse into a certain speaking style - a sort of effete formal speech that just sounded cheesy as anything. In general, villains who speak like this have vocabularies liberally sprinkled with the following words:

According

Amuse/amused/amusing

Blast/blasted

Bothersome

Doom/doomed

Droll

Inferior

Interesting

Imbecile

Meddling/meddlesome

Most (adverb, eg, "that is most interesting...")

May

My dear (when addressing a female adversary)

My friend (when addressing a male adversary)

Quite

So (interjection, eg, "so, we meet again...")

Troublesome

Well (interjection, eg, "Well, well, well...")

Worthwhile

Wretched

Note that I'm not saying that your villain can't ever use these words, but they shouldn't be part of xir regular vocabulary. If your villain ever says anything along the lines of "so, this is a most interesting development, my meddlesome friend," or "your meddlesome antics are so amusing, my dear, but you are still doomed," you should probably consider getting a new villain, because at this point it sounds incredibly cheesy. (On the other hand, if you want to write cheese, then stuff your villain's vocabulary to the gills with this stuff.)

Be careful with those smirks.

There's technically nothing wrong with a smug smirk now and then, but novice writers and roleplayers abuse it relentlessly. What's wrong with a "devilish grin" or a "wicked smile?" Or how about any number of other facial expressions? Most villains are people, and real people typically have a variety of expressions.

In some cases, (particularly with villain RP characters) people use smirks to try to show that their characters are cool, confident, and have everything under control - and should therefore be taken seriously and respected as an opponent. But very often the characters actually have very little villain cred to them - their actions actually show them to be at most a blustering bully who might be dangerous to a few people in the short-term, but in the long-term would end up getting very little accomplished before getting caught. Now if your villain is supposed to be a blustering bully, that's fine. But remember: if you want someone who comes off as genuinely intimidating and dangerous, it's their actions that are going to have to carry it, not their facial expressions.

Either have your villains learn from their mistakes or the mistakes of their peers, or dispose of them quickly.

The Sailor Moon anime was a pretty bad offender here. All of the villains used pretty much the same strategy throughout the show's run: send out exactly one person and one monster to try to collect the MacGuffin of the season. Despite the fact that many of these villains were in bitter competition with each other, none of them ever really tried to do anything to improve their odds against the senshi aside from trying a different monster. Anyone with half a brain would have noticed that one monster, no matter what it was, was no match against the sailor senshi.

Plus, the villains using the same tricks over and over (even with different gimmicks) gets rather old.

Character development and progression is for villains, too.

Most villains are people too, and people develop and change over time. Unfortunately, many writers get stuck on making sure their villain stays a villain or at least stays the same kind of villain, and as a result logical character progression is ignored - or worse, previous characterization gets completely contradicted.

For example, let's say that supervillain Greenwing was initially motivated by a desire to see the CEO of SyntheStar Biomechanics crash and burn, and all of Greenwing's villainous actions thus far have been directed toward attacking SyntheStar's Biomechanics' CEO. Then one day the CEO does go down, and Greenwing's longtime motive is fulfilled.

...Then a few episodes/issues later, Greenwing is willingly working with an international smuggling ring despite never having been shown as the kind of person who would get into illegal affairs for the sake of money. In fact, it completely contradicts Greenwing's previous characterization, as part of his beef with the CEO was that she'd apparently do anything for money. The only reason Greenwing was brought into this arc was because the author really wanted him in it, or just couldn't bear the thought of Greenwing becoming "less" of a villain.

Contrary to what some people say, a villain eventually turning "good" (or at the very least, less overtly evil and malicious) is not the worst thing to happen to a villain. The worst thing to happen to a villain is to remain or act evil for no other reason than because the writer or writers want to keep that character a villain.

If you want to keep your villain villainous, fine - but let it make sense for the character. Maybe Greenwing got pulled into the smuggling deal by someone he owed a debt to, or maybe it's SyntheStar products they're smuggling and he's doing it just to rub salt into the CEO's wounds. Either way, don't make a villain go OOC in the name of staying a villain.

Beware "2Evul4U!!" Syndrome

This is a term I use to describe cases where the activities and actions of the villains are intended to make them look SUPER HARDCORE EVIL AND SCARY, but in reality they’re so overboard and over the top that there’s no reasonable way that they or the setting could keep functioning but for the fact that the Author Says So.

Let’s say that Jennifer creates a villainous organization called Cerberus, and ballparks its membership at 50,000 members worldwide. To show off just how evil Cerberus is, Jennifer writes that when Cerberus hires new members, only perhaps one in ten ultimately survives the arduous screening process, which involves lots of death traps to weed out the weak and unworthy. But that would mean that for the 50,000 members Cerberus has now, somewhere around 490,000 people must have died trying to get in - which for comparison, is more people than live in Sacramento, California (475,516 as of the 2012 census). How Cerberus has time to get anything else done when it has 490,000+ bodies to dispose of and almost that many missing persons investigations to contend with is anyone’s guess.

Same goes for villains or villainous organizations that kill mooks and underlings for failure at the drop of a hat - if it takes so little to earn the death sentence, how do they manage to keep enough people employed to get anything done? Why don’t the underlings revolt and desert if they’re so likely to end up dead? Why would someone work for such an organization in the first place? What is the organization doing to take care of all the bodies? What are they telling the families and friends of the people they’re offing? (If you are trying to write a villainous organization that engages in any kinds of cover ups, see also Tips To Write Better & More Believable Cover Ups.)

Or, Kevin writes about a small town that is continually plagued by supernatural activity that frequently kills off its residents - a possession results in the death of an entire family one week, the next week a vampire attack takes out a dozen students, and a short while after that a curse transforms six people into monstrous beasts that have to be killed. The trend goes on for so long that there shouldn’t even be a town to speak of anymore. At this point the “scary” events stop being scary because audiences can no longer suspend their disbelief during the story. Remember - sometimes, less is more. It can be much more effective to focus closely on a small wrong or horror rather than try to shock people with sheer volume. Quality over quantity.