The V8 PRNG

“Iwould have gone with mersenne twister since it is what everyone else uses (python, ruby, etc).” This short critique, left by Dean McNamee, is the only substantive feedback on the code review of V8’s PRNG when it was first committed on June 15, 2009. Dean’s recommendation is the same one I’ll eventually get around to making in this post.

V8’s PRNG code has been tweaked and moved around over the past six years. It used to be native code, now it’s in user-space, but the algorithm has remained essentially the same. The actual implementation uses internal API and is a bit obfuscated, so let’s look at a more readable implementation of the same algorithm —

Well, that’s still pretty opaque, but let’s slog through.

There is one more clue. A comment in older versions of the V8 source stated simply: “random number generator using George Marsaglia’s MWC algorithm.” A few minutes with Google teaches the following —

George Marsaglia was a mathematician who spent much of his career studying PRNGs. He also created the original Diehard battery of statistical tests for measuring the quality of a random number generator.

MWC stands for multiply-with-carry, a class of PRNGs that Marsaglia invented. MWC generators are very similar to classic linear congruential generators (LCGs) like our simple example from earlier (in fact, there is a one-to-one correspondence between MWCs and LCGs, see section 3.6 of this paper for details). Their advantage over standard LCGs is that they can produce sequences with longer cycle lengths with about the same number of CPU cycles.

So if you’re going to crib a PRNG off of someone, Marsaglia seems like a good choice, and MWC seems like a reasonable algorithm.

The V8 algorithm doesn’t look like a typical MWC generator though. It turns out that’s because the V8 algorithm is not an MWC generator. It’s two MWC sub-generators — one on line 5, the other on line 6 — combined to produce one random number on line 9. I’ll spare you the math, but each of the sub-generators have prime cycle lengths of about 2³⁰, making the combined cycle length of the generated sequence about 2⁶⁰.

If you’ll recall, we have 2¹³² possible identifiers, but now we know that V8’s Math.random() can only produce 2⁶⁰ of them. Still, assuming a uniform distribution, the probability of collision after randomly generating 100,000,000 identifiers should be less than 0.4%. We started seeing collisions after generating far fewer identifiers than that. Something must be wrong with our analysis. The cycle length estimate is provably correct, so we must not have a uniform distribution — there must be some additional structure to the sequence being generated.

A Tale of Two Generators

Before returning to the V8 PRNG, let’s look one more time at our random identifier generation code —

The scaling method on line 6 is important. This is the method that MDN recommends for scaling a random number, and it’s in widespread use in the wild. It’s called the multiply-and-floor method, because that’s what it does. It’s also called the take-from-top method, because the lower bits of the random number are truncated, leaving the left-most or top bits as our scaled integer result. (Quick note: it’s subtle, but in general this method is slightly biased if your scaled range doesn’t evenly divide your PRNG’s output range. A general solution should use rejection sampling like this, which is part of the standard library in other languages.)

Do you see the problem yet? What’s weird about the V8 algorithm is how the two generators are mixed. It doesn’t xor the numbers from the two streams together. Instead, it simply concatenates the lower 16 bits of output from the two sub-generators. This turns out to be a critical flaw. When we multiply Math.random() by 64 and floor it we end up with the left-most, or top 6 bits of the number. These top 6 bits come exclusively from the first of the two MWC sub-generators.

Bits from PRNG #1 are red, from PRNG #2 are blue.

If we analyze the first sub-generator independently we see that it has 32 bits of internal state. It’s not a full-cycle generator — its actual cycle length is about 590 million (18,030*2¹⁵-1, the math is tricky but it’s explained here and here, or you can just trust me). So we can only produce a maximum of 590 million distinct request identifiers with this generator. If they were randomly selected there would be a 50% chance of collision after generating just 30,000 identifiers.

If that were true, we should have started seeing collisions almost immediately. To understand why we didn’t, recall our simple example where we pulled triples from a 4 bit LCG. Birthday paradox math doesn’t apply — for this application the sequence is nowhere near random, so we can’t pretend it is. It’s clear that we won’t produce a duplicate until the 17th triple. The same thing is happening with the V8 PRNG and our random identifiers — under certain conditions, the PRNG’s lack of randomness is making it less likely that we’ll see a collision.

In this case the generator’s determinism worked in our favor, but that’s not always true. The general lesson here is that, even for a high quality PRNG, you can’t assume a random distribution unless the generator’s cycle length is much larger than the number of random values you’re generating. A good general heuristic is —

If you need to use n random values you need a PRNG with a cycle length of at least n².

The reason is that, within a PRNGs period, excessive regularity can cause poor performance on some important statistical tests (in particular, collision tests). To perform well, the sample size n must be proportional to the square root of the period length. Page 22 of Pierre L’Ecuyer’s excellent chapter on random number generation has more detail.

For a use case like ours, where we’re trying to generate unique values using multiple independent sequences from the same generator, we’re less concerned about statistical randomness and more concerned that the sequences not overlap. If we have n sequences of length l from a generator with period p, the probability of an overlap is [1-(nl)/(p-1)]ⁿ⁻ ¹, or approximately ln²/p for a big enough p (see here and here for details). The point is we need a long cycle length. Otherwise we’re making a mistake pretending our sequence is random.

Long story short, if you’re using Math.random() in V8 and you need a sequence of random numbers that’s reasonably high quality, you shouldn’t use more than about 24,000 numbers. If you’re generating multiple streams of any substantial size and don’t want any overlap, you shouldn’t use Math.random() at all.

If the algorithm that V8’s Math.random() uses is poor quality, you might be wondering how it was chosen at all. Let’s see if we can find out.

A Brief History of MWC1616

“The MWC generator concatenates two 16-bit multiply-with-carry generators […] has period about 2⁶⁰ and seems to pass all tests of randomness. A favorite stand-alone generator — faster than KISS, which contains it.” That’s the extent of Marsaglia’s analysis of MWC1616, which is the name of the algorithm that powers V8’s Math.random(). If you take him at his word, the algorithm ticks the box for most of the important criteria you’d consider in choosing a PRNG.

MWC1616 was first introduced by Marsaglia in 1997 as a simple general purpose generator that, in his words, “seems to pass all tests of randomness put to it,” a comment that betrays Marsaglia’s largely empirical methodology. He seems to have trusted an algorithm if it passed his Diehard tests. Unfortunately, the Diehard tests he was using in the late 1990s weren’t that good, at least by today’s standards. If you run MWC1616 through a more modern empirical testing framework like TestU01's SmallCrush it fails catastrophically (it does even worse than the MINSTD generator, which was outdated even in the 1990s, but Marsaglia’s Diehard tests probably didn’t have the granularity to tell him that).

As far as I know, there’s no mathematical basis for combining sub-generators the way MWC1616 does — concatenating subsets of the generated bits. It’s more typical to see bits from sub-generators mixed using some form of modulo arithmetic (e.g., addition modulo 2³², or xor). It appears that Marsaglia, himself, became aware of this deficiency shortly after posting MWC1616 on Usenet as a component of one version of his KISS generator. On January 12, 1999, Marsaglia posted the version of MWC1616 used in V8. Eight days later, on January 20, he posted a different version of the algorithm. It’s subtle, but in the updated version, the upper bits of the second generator are not masked away, mixing bits from the two sequences more thoroughly.

Both versions of the algorithm appear in other places, adding to the confusion. The January 20 version of MWC1616 (i.e., the better version) is in Numerical Recipes, labeled MWC with Base b = 2¹⁶, under the heading When You Have Only 32-Bit Arithmetic, and only after first advising that, rather than implementing one of the algorithms listed, you should “get a better compiler!” Pretty discouraging words for an algorithm that’s better than what V8 has powering Math.random(). Rather inexplicably (because it’s so obscure) the January 20 version of MWC1616 is also given as an example computational method in Wikipedia’s article on random number generation. Implementations of the older January 12 version are included in TestU01 twice, once labeled MWC1616 and a second time labeled MWC97R. It’s also one of the generators available in R (apparently it used to be the default).

So there are lots of places the algorithm can be found. It was obscure to me, but given the bona fides listed above I guess it’s not surprising it was chosen. Hopefully this article will serve as a warning, strengthening and confirming Knuth’s observation that kicked off this post —

In general, PRNGs are subtle and you should do your own analysis and understand the limitations of any algorithm you’re implementing or using

Specifically, don’t use MWC1616, it’s not very good

There are lots of better options. Let’s look at a couple of them.

The CSPRNG Workaround

To fix our identifier code we needed a replacement for Math.random() and we needed it fast. Lots of alternative PRNG implementations exist for Javascript, but we were looking for something that —

Has a long enough period to generate all of our 2¹³² identifiers

Is well supported and battle tested

Luckily, the Node.js standard library has another PRNG that meets both requirements: crypto.randomBytes(), a cryptographically secure PRNG (CSPRNG) that calls OpenSSL’s RAND_bytes (which, according to the docs, produces a random number by generating the SHA-1 hash of 8184 bits of internal state, which it regularly reseeds from various entropy sources). If you’re in a web browser crypto.getRandomValues() should do the same job.

This isn’t a perfect general solution for three reasons —

CSPRNGs almost always use non-linear transformations and are generally slower than non-cryptographic alternatives

Many CSPRNG systems are not seed-able, which makes it impossible to create a reproducible sequence of values (e.g., for testing)

CSPRNGs emphasize unpredictability over all other measures of quality, some of which might be more important for your use case

However —

Speed is relative, and CSPRNGs are fast enough for most use cases (I can get about 100MB/s of random data from crypto.getRandomValues() in Chrome on my machine)

In the limit, unpredictability implies an inability to distinguish the generator’s output from true randomness, which implies everything else we want out of a pseudo-random sequence

It’s likely that a generator advertised as “cryptographically secure” has been carefully code reviewed and subjected to many empirical tests of randomness

We’re still making some assumptions, but they are evidence-based and pragmatic. If you’re unsure about the quality of your non-cryptographic alternatives, and unless you need deterministic seeding or require rigorous proofs of quality measures, using a CSPRNG is your best option. If you don’t trust your standard library’s CSPRNG (and you shouldn’t for cryptographic purposes) the right solution is to use urandom, which is managed by the kernel (Linux uses a scheme similar to OpenSSL’s, OS X uses Bruce Schneier’s Yarrow generator).

I can’t tell you the exact cycle length of crypto.randomBytes() because as far as I know there’s no closed form solution to that problem (i.e., no one knows). All I can say is that with a large state space and a continuous stream of new entropy coming in, it should be safe. If you trust OpenSSL to generate your public/private key pairs then it doesn’t make much sense not to trust it here. Empirically, once we swapped our call to Math.random() with a call to crypto.randomBytes() our collision problem went away.

In fact, Chrome could just have Math.random() call the same CSPRNG they’re using for crypto.randomBytes(), which appears to be what Webkit is doing. That said, there are lots of fast, high quality non-cryptographic alternatives, too. Let’s put a final nail in the MWC1616 coffin and take a look at some other options.

V8’s PRNG is Comparatively Unsatisfactory

My goal was to convince you that V8’s Math.random() is broken, and should be replaced. So far we’ve found obvious structural patterns in its output bits, catastrophic failure on empirical tests, and poor performance in the real world. If you still want more evidence, here are some pretty pictures that might sway you —