Pentium FDIV bug

With a goal to boost the execution of floating-point scalar code by 3 times and vector code by 5 times, compared to the 486DX chip, Intel decided to use the SRT algorithm that can generate two quotient bits per clock cycle, while the traditional 486 shift-and-subtract algorithm was generating only one quotient bit per cycle. This SRT algorithm uses a lookup table to calculate the intermidiate quotients necessary for floating-point division. Intel's lookup table consists of 1066 table entries, of which, due to a programming error, five were not downloaded into the programmable logic array (PLA). When any of these five cells is accessed by the floating point unit (FPU), it (the FPU) fetches zero instead of +2, which was supposed to be contained in the "missing" cells. This throws off the calculation and results in a less precise number than the correct answer(Byte Magazine, March 1995).

At its worst, this error can occur as high as the fourth significant digit of a decimal number, but the possibilities of this happening are 1 in 360 billion. It is most common that the error appears in the 9th or 10th decimal digit, which yields a chance of this happening of 1 in 9 billion.

Intel has clasified the bug (or the flaw, as they refer to it) with the following characteristics (Intel):

On certain input data, the FPDI (Floating Point Divide Instructions) on the Pentium processor produce inaccurate results.

The error can occur in any of the three operating precisions, namely single, double, or extended, for the divide instruction. However, it has been noted that far fewer failures are found in single precision than in double or extended precisions.

The incidence of the problem is independent of the processor rounding modes.

The occurrence of the problem is highly dependent on the input data. Only certain data will trigger the problem. There is a probability that 1 in 9 billion randomly fed divide or remainder instructions will produce inaccurate results.

The degree of inaccuracy depends on the input data and upon the instruction involved.

The problem does not occur on the specific use of the divide instruction to compute the reciprocal of the input operand in single precision.

A 3-D plot of the ratio 4195835/3145727 calculated on a Pentium with FDIV bug. The depressed triangular areas indicate where incorrect values have been computed. The correct values all would round to 1.3338, but the returned values are 1.3337, an error in the fifth significant digit. Byte Magazine, March 1995.