It really depends on who you ask. Someone coming from a mathematical background will often prefer code a seasoned programmer may find problematic. This makes it seem very subjective.

Additionally, you'll find that different languages have different ways to solve the same problem. In Python, for example, swapping two variables is trivial: x, y = y, x

In practice, most programmers would agree that there are some common criteria:

elegant code must be as short as possible

but as verbose as necessary

it should be easy to follow

each statement should be easy to understand

no statement should seem redundant or superfluous

the code should follow the idioms of the language

it mustn't make use of leaky abstractions

This is a very pragmatic view of elegance. Code mustn't be "clever" or it fails these criteria. But it will be maintainable and easier to adjust and extend.

That said, there is code that can be a work of art despite absolutely failing by most of the rules in this list. "Clever" code often falls in this category. It is very dense, often relies on a thorough understanding of the underlying implementations (say, memory management, or the exact implementation of the compiler) but nevertheless evokes a sublime fascination.

There is a difference between the elegance you would find in Yet Another Perl Hacker scripts versus code you would actually want to use in production. This is the reason why you'd never want Mel to be in your team even though you could call his major opus elegant.