I was having a discussion with a friend about this question, and he said something along the lines of this:

I too assumed that Hermione was white because of my prejudices.

I disagree: it's not prejudice that makes my friend think Hermione is white. Instead, I think my friend picked up on a number of subtle clues embedded into the text of Harry Potter. Rightly or wrongly, Western culture and the English language use a number of vague and unusual techniques to convey information about race. This is why, while JKR never explicitly states the race of many of her characters, we can infer the race of many of these characters by taking a close look at the language, or rather the lack of language, that she uses to describe these characters.

I previously claimed that JKR only specifies the race of non-white characters. I've done further research, and I know that is not true. I've found a number of examples where JKR does discuss the skin color of white characters. (I'll get to those examples in a second). However, I still stand by the following idea, which is going to inform the rest of my answer:

I think it is possible that some readers never even notice what color the people in the story are. Don't notice, don't care. Whites of course have the privilege of not caring, of being "colorblind." Nobody else does. (Slate, A Whitewashed Earthsea)

Here's the claim that I'm going to spend the rest of my answer proving. JKR will specify the skin color of her characters only in a few specific circumstances:

If a character is a minority. This is because for most (almost all) minorities, skin color is an important part of their identity because it's the feature that the rest of society judges them by. If a character is evil. One of the defining features of Voldemort is that he is deathly (almost unnaturally) pale. Many other characters who are (initially) associated with evil are also described in similar ways. If there is a "biological reason" for describing a characters skin color, e.g. if a character is sick, scared, depressed, or (as quoted in the question) tanned from the sun. For example, when Tonks is depressed in The Half-Blood Prince, she is described as "pale". I would suspect that this is because being pale is a symptom of being sick.

In all other cases, when JKR does not describe the skin color of a character, it is because that character is white. Minorities in Western cultures do not have the privilege of ignoring their skin color; whites can go through their entire lives without thinking about the fact that they are white. This is because if you are white, you won't be systematically discriminated against because of the color of your skin; everyone else will be.

I can prove this by the fact that I only found one instance where a character was described as white, and none of the above reasons applied (see the bottom answer for a discussion about that). But I think it should also make sense that white characters don't think about or notice the fact that they are white. In fact, it's even awkward for people to say things like "my white friend" or "I'm white", even if those sentences are true.

In Learning To Be White: Money, Race and God in America, the theologian and college professor Thandeka describes an little social experiment she devised called The Race Game. The origins of this game were this: At a lunch together, a white member of the staff at Smith College asked Thandeka what it felt like to be black. Thandeka said that if her luncheon partner played The Race Game for one week, she would then answer the colleague’s question. The game consisted of one rule. For one week the woman was to “use the ascriptive term white” whenever she mentioned the names of “Euro-American cohorts. She must say, for instance, ‘my white husband, Phil,’ or ‘my white friend Julie,’ or ‘my lovely white child Jackie.’” The women never had lunch with Thandeka again. Over and over, when Thandeka presented the idea of The Race Game, white colleagues refused to do it. Thandeka remarks: “…in their [my white colleagues’] racial lexicon, their own racial group becomes the great unsaid.” (Notes on Questioning Whiteness as a Literary Practice)

So for a number of reasons, both the fact that white characters don't need to think about their race and the fact that it's uncomfortable for white characters to think about their race, it makes sense that Hermione wouldn't be described as white, at least in the same way that other minority characters are described as black.

In researching this answer, I literally searched through every book looking for references to characters skin color. I'm not going to include every quote I find because this answer would be too long. I also found one exception, which I've included at the bottom of the answer. Here are the results:

Minority Characters

"Thomas, Dean," a Black boy even taller than Ron, joined Harry at the Gryffindor table. (Sorcerer's Stone)

'Yeah, I see what you mean, Remus,' said a bald black wizard standing furthest back - he had a deep, slow voice and wore a single gold hoop in his ear - 'he looks exactly like James.' (Order of the Phoenix)

"Evil" Characters

[Voldemort] was standing in a dark, curtained room lit by a single branch of candles. His hands were clenched on the back of a chair in front of him. They were long-fingered and white as though they had not seen sunlight for years and looked like large, pale spiders against the dark velvet of the chair. (Order of the Phoenix)

Harry twisted around, his own wand held high, and saw that the Death Eater had ripped off his mask and was pointing his wand directly at Harry, who recognised the long, pale, twisted face from the Daily Prophet: Antonin Dolohov, the wizard who had murdered the Prewetts. (Order of the Phoenix)

And he threw the copy of The Quibbler to her. She caught it and stared down at the cover. Her pale, doughy face turned an ugly, patchy violet. 'When did you do this?' she asked, her voice trembling slightly. 'Last Hogsmeade weekend,' said Harry. She looked up at him, incandescent with rage, the magazine shaking in her stubby fingers. (Order of the Phoenix)

And from behind the six large figures before them came a seventh, smaller boy, smirking all over his pale, pointed face. It was Draco Malfoy. (Chamber of Secrets)

Sick/Depressed/Scared Characters

A young witch with a pale, heart-shaped face and mousy brown hair was sitting at the table clutching a large mug between her hands. "Hello, Professor," she said. " Wotcher, Harry." "Hi, Tonks." (This is in The Half-Blood Prince, when Tonks is feeling depressed because of Lupin. When they get engaged, Tonks is no longer described as Pale.)

'Harry, dear, are you sure you're all right?' said Mrs Weasley in a worried voice, as they walked around the unkempt patch of grass in the middle of Grimmauld Place. 'You look ever so pale . . . are you sure you slept this morning? You go upstairs to bed right now and you can have a couple of hours of sleep before dinner, all right?' (Order of the Phoenix)

Exceptions to my theory

"Well, well, well," said the Fat Lady, "Violet's just told me everything. Who's just been chosen as school champion, then?" "Balderdash," said Harry dully. "It most certainly isn't!" said the pale witch indignantly. "No, no, Vi, it's the password," said the Fat Lady soothingly, and she swung forward on her hinges to let Harry into the common room. (Goblet of Fire)

I'm not quite sure about this quote: if anyone has any ideas, let me know in the comments. I suspect this is because Violet is associated with alchohol (she drinks a lot with the Fat Lady), and her being described as pale might be because of that. However, I'm not 100% sure about that argument.