Around my office, such a problem is called a Heisenbug, a pun on the name of the great physicist Werner Heisenberg, who first described the observer effect (the rule that observing any phenomenon will change it) and the uncertainty principle (the rule that you can know either where something is or how fast it is going, but not both).

The frustrating thing about a Heisenbug is that there is no way to know for certain you have fixed it. You may believe you understand the cause of a particular Heisenbug, you may address that cause, and the Heisenbug may appear to go away, but you never know..