RSA and DSA Public Key Signatures and RSA Encryption.

Previously

Public key encryption

Public key encryption, also known as asymmetric encryption, is encryption where instead of using one key which both encrypts and decrypts a message, there are two keys: one which encrypts and the other which decrypts, which are powerful tools that allow you to do a couple things.

Encryption

If you want to send somebody a message, you encrypt it with a standard symmetric key algorithm like AES or ChaCha20 using a randomly-generated key. Then they use the recipients public key to encrypt the symmetric key and send it. They can decrypt that key with their private key and use it to decrypt the message.

Sign messages

There are two main scheme for signing messages, which work slightly differently.

RSA is the simpler of the two and works like this: take a message, get its hash, then use your private key to decrypt said hash. Then you send the message and the signature, so the recipient can hash the message and then encrypt the signature with your public key. At this point, the decrypted signature should match the hash.

The key to this is that it should be infeasible to create a message that decrypts to a specific message without access to the private key. (Yes, this sounds backwards, but it’s just terminology; the private key always decrypts and the public key always encrypts. But with all good ciphers, choosing which direction is decryption and which is encryption is somewhat arbitrary).

DSA is slightly harder to describe, as it doesn’t do something easy to explain like encrypt and decrypt a message, but can be thought of as generating a message authentication code that can be checked with the public key.

RSA

First you need to collocate p , q , d (which is just p*q ), n , and e . n and e are the public key and p , q , and d are the private key. I didn’t have to implement this, but if you are curious as to how to create them then check out Wikipedia.

To encrypt a message, you raise it to the power of e mod n . To decrypt it, you raise it to the power of d mod n . n is referred to as the modulus, and it is important as it governs the size the message you encrypt has to be. The length is called k .

There is a shortcut for decrypting the message called the Chinese remainder algorithm. The private key will in addition to d also have p , q , dP (which is d mod (p - 1) ), dQ ( d mod (q - 1)) and qinv ( q ^ -1 mod p ). This allows you to instead of calculating c ^ mod n (aka c ^ mod (p*q) ), you can calculate m1 ( c^dP mod p ) m2 ( c^dQ mod q ) and h ( qinv * (m1 - m2) mod p ), and then m2 + h * q is the decrypted message. While this sounds more complex, all the operations are on smaller numbers making, it much faster. (I confirmed this when implementing; it was more than twice as fast).

One last thing is that the amount of time it takes to calculate the above is very closely related to the values of the message and the key, meaning if an attacker can measure how long it takes to decrypt something several times, they could figure out what your private key is. The solution is called “blinding,” and it involves adding another layer of randomness to the process. First you need to pick a random number r which is less than the modulus ( n ). Also r needs to be coprime with n , meaning that the greatest common divisor (largest number that divides both equally) is 1. Luckily we know both devisors of n ( p and q ), so all we have to do is generate a random number with a byte length the same as n and check that r is less then n , that r mod p !== 0 , and that r mod q !== 0 . To calculate the blind value with r pow e mod n (with e being the public exponent), then you multiply the message by that (mod n ) and calculate the encrypted value. Once you are done, you need the value to unblind it, which is r ^ -1 mod n , then multiply the decrypted value by that (mod n ) to get the unblinded value.

Originally I calculated the binding with code based on end-to-end, but the way they implemented modular inverse was very slow. When I replaced that code with one built into bn.js, my tests started completing in 45 seconds instead of 2 minutes 45 seconds. I’m not saying this to pick on end-to-end but to underscore how important for performance a good big number library is. That single unoptimized inverse took more then twice as long as all other functions involved in RSA decryption combined.

DSA

DSA has five parameters: p , q , g , x , and y . p , q , and g are public and shareable because they are both difficult to compute and don’t represent a security risk if multiple parties use them. To compute them, you first pick two lengths L and N ; these are your key length, with N being the length of the hash function (e.g. sha1 would be 160, and sha2 be the same as their name, e.g. sha256) with L being the size of the secret key. A common choice is: pick a prime q of bit length N . Then pick a prime p such that p-1 is a multiple of q (or put another way p-1 mod q = 0 ). G is a number where g pow q mod p = 1 ; this can be found by picking a number h and calculating h pow (p - 1)/q mod p . If the result isn’t 1, then that’s your g . This doesn’t need to be random, so you can start with h being 2 and work your way up. As you might recall from my Diffie-Hellman post, finding pairs of prime is not easy, so reusing them makes sense especially if you need to generate a bunch of keys.

The private key ( x ) is a random number less then q . The private key ( y ) is g pow x mod p .

A DSA signature consists of two output values: r and s . First, generate a random number k less than q . Now calculate r as ( g pow k) mod q . In theory, r could be zero and you’d have to start over, but to quote RFC 6979, “this is an utterly improbable occurrence”. You may notice that r requires neither the message nor the private key, thus this can be pre-calculated (which is helpful, as it is the most computationally expensive part). You calculate s by taking the hash of the message h , adding the private key multiplied by r to it, and multiplying the result by k pow -1 ( kinv ), all of this mod q , or in other words (h+x*r) * kinv mod q . If s is 0, pick a new k and start over.

It is vital that k be random because an attacker knows r , s , h , and q , so in the equation s = (h+x*r) * kinv mod q only k and x are unknown, so if the same k was used twice then you know the signer’s public key.

This is not theoretical – see this article for how a bug in the Debian random number generator meant that there were only 32,767 possible random numbers ( k values). Testing all of them for a given signature is not hard.

The solution to this is to use the technique outlined in RFC 6979 to use an HMAC of the private key and message to calculate k . This deterministically calculates k in an unpredictable way that is guaranteed (for a secure hash algorithm) to be unpredictable for a given private key and message.

To verify the signature, you calculate h = the hash of the message, then

w = s pow -1 mod q u1 = h * w mod q u2 = r * w mod q v = (g pow u1) * (y pow u2) mod p mod q

v should equal r .

Actually implementing it

If there has been a theme to these posts, then it is implementing the actual crypto algorithms are the easy part. The minutia you need to actually get it working is the hard part and public key signing/encryption is no different as it requires messages to be padded and for you to be able to read key files.

Padding

For RSA to work, the message has to be just the right size. It needs to be the similar in byte length to the modulus, but it can’t be bigger then the modulus (when treated as an integer). Additionally, for signatures you need to be explicit about what hash algorithm you used, else somebody could theoretically do something sneaky by substituting a weaker hash value. For encryption, you need to inject an element of randomness into the encryption process to prevent the same message from encrypting into the same thing (or two very similar messages, letting you figure out the key which encrypted it).

Signature padding

For signatures to pad the message up to the byte length of the modulus the method is

0x0, 0x1, 0xff ... 0xff, 0x0, digest identifier, hash

or in English, two bytes with the value zero and 1, modulus - (hash length + digest identifier + 3) bytes of value 255, a byte with value 0, the digest identifier and then the hash. The digest identifiers are given in section 9.2 of this paper but not for every digest Node supports, so for ripemd160 I had to just encrypt the signature that node produced to figure out the value that OpenSSL uses. I believe they are binary representations of the asn1 OID values of the hash algorithms, but it was far easier to reverse engineer them out of OpenSSL than figure out which where the ripemd160 one is defined.

Encryption padding

Padding for encryption has two different schemes: PKCS1 v1.5 and OAEP. PKCS1 is simpler, but

It is possible to generate valid RSAES-PKCS1-v1_5 ciphertexts without knowing the corresponding plaintexts, with a reasonable probability of success.

which can lead to attacks. So TL;DR: use OAEP.

PKCS1 v1.5

Take you your message M which you need to get up to the length of the modulus. You first generate message length - (modules length + 3) bytes of random numbers, none of the bytes being zero. The way I did this was by generating random numbers, discarding any bytes that were zero until I had enough. There are plenty of other ways to do this, but most of them are likely going to create non-uniform distributions of random numbers. The message is [0, 2] + non zero random bytes + [0] + message.

To remove the padding when decrypting, you just look for the second zero byte and the message starts in the next position.

OAEP

OAEP, which stands for Optical Assyrian Enlisted Pandas (or something equally irrelevant to how it works), is a padding scheme which is non-deterministic over its entire length (as opposed to PKCS1, where only the leftmost portion is random, and the right part is the message).

OAEP has a function it calls MGF (mask generation function). MGF can be defined in Node.js as:

You take a hash function and hash an empty message. Call the result iHash

Then create a buffer filled with zeroes of length modulus 0 (message length + hash length * 2 + 2). Call it ps .

. Concat iHash , ps , a byte with the value of 0, and the message. Call that db .

, , a byte with the value of 0, and the message. Call that . Generate random bytes the same length as the db and call that seed .

and call that . Call mgf(seed, db length) . Call the result dbMask .

. Call the result . XOR dbMask and db to create maskedDb .

and to create . Call mgf(maskedDB, hash length) and call that seedMask .

and call that . XOR seedMask to seed to create maskedSeed .

to to create . Concat a zero byte, maskedSeed and maskedDB to create the message

Note: Node doesn’t give any way to specify a digest algorithm here besides sha1.

To decrypt:

Calculate the hash of an empty string and call it iHash .

. Call the length of iHash hashLen .

. The first byte of the padded message is zero, so get a slice starting at the next byte of hashLen . This is maskedSeed .

. This is . The rest of the message is maskedDB .

. XOR maskedDB with mgf(maskedDB, hashLen) . This is seed .

with . This is . XOR maskedDB with mgf(seed, maskedDB length) . This is db .

with . This is . The first hashLen bytes of db should be equal to iHash . (You know your message has been tampered with if it isn’t.)

bytes of should be equal to . (You know your message has been tampered with if it isn’t.) iHash is followed by some number of zero bytes (possibly none) and then a byte with value of 1. (If it doesn’t, your message has been tampered with.)

is followed by some number of zero bytes (possibly none) and then a byte with value of 1. (If it doesn’t, your message has been tampered with.) The rest of db is your message.

Motherfraking PEM files

Now comes the hard part, getting the public or private key out of the file that the user provides. The file in question is likely going to be a .pem file. PEM files are the sole surviving part of a failed IETF proposal for encrypted email. They are files which look like:

-----BEGIN RSA PUBLIC KEY----- MIGGAn87CzBsWj+7ILyW0Z//IDUD6BXkgZ2cCA9tRIjcbNscID7H5Msb+0u9tHDe vWyamlj+OSSmJVbUStIy43S6LGnmBvvxn2sfVelZvlZaCndZpj/0QcyMx06RD/0t Vm9G+X8z8WLqjA/6r5qYkjUESMQJh9uEYveuaVV2ripdzjRDAgMBAAE= -----END RSA PUBLIC KEY-----

and consist of a header and footer around a bunch of base64 encoded data. The header and footer are of the form -----(BEGIN|END) :tag----- . The base64-encoded data is DER-encoded data.

What is DER encoding you ask? It is the Distinguished Encoding Rules for encoding data in the Abstract Syntax Notation One (ASN1) format. More details can be found in this article, but what you need to know is that DER/ASN1 encoding is a binary method for encoding data. The key above decodes as:

sequence of 2 elements integer value 161960552770399697736583593721290722658968249097500897481864131194167084781582467842371267641339824876964119703500297546233667881250293993668028296697189038155281236041428666387090163250593165185711140282087579809282559115211132751074088235101133730773838767046174614028835929718065895239952789579051709507 integer value 65537



You will note that this just has values, not the keys, meaning you have to find the scheme which defines how a particular key type is packed in.

The types we need to care about are:

integers

sequences

object identifier

bit strings

octet strings

There are 3 algorithms we can use to make keys:

RSA, which we covered earlier

DSA, which we didn’t cover, but what you need to know is that both public and private keys have p , q , and g parameters, and the public key has a y parameter and the private key an x .

, , and parameters, and the public key has a parameter and the private key an . Elliptical Curve DSA. Didn’t cover that one here, but what you need to know is both public and private keys need to know the curve ID (which is an object identifier) and there are private and public keys which are bit strings and octet strings respectively.

There are 5 types of keys that someone can give you (that I have found):

Algorithm specific private keys. These are keys that start with BEGIN (algorithm) PRIVATE KEY , e.g. -----BEGIN RSA PRIVATE KEY----- . The use of these keys looks to be a legacy feature and they aren’t recommended (especially for encrypted private keys, as they use their own half-assed encryption setup). Generic private keys that start with -----BEGIN PRIVATE KEY----- . This is a generic wrapper that gives you information on what algorithm and specific parameters for it and then the private key data as an octet string that you then decode to get the private key which is private key-specific data (as opposed to algorithm parameters). Encrypted private keys that start with -----BEGIN ENCRYPTED PRIVATE KEY----- . These contain an encrypted octet string and information to allow it to be decrypted if you supply a password. When decrypted, it is a generic private key. Algorithm-specific public keys. As far as I can tell, OPENSSL will only make them for RSA, so THEY all start with -----BEGIN RSA PUBLIC KEY----- . Deprecated. Generic public keys, which start with -----BEGIN PUBLIC KEY----- . These are similar to the generic private key files: they have an algorithm ID, parameters for the algorithm and then a bit string (not an octet string like private keys), which decodes into public key data.

Parts of these are defined in a variety of RFCs, with only part of any given RFC still being used. The rest of this isn’t defined anywhere, as far as I can tell.

Algorithm-specific private keys

While officially being deprecated, you can still see these keys being used in the wild. These keys are generally simpler in structure than the generic ones, but the encrypted ones are much less secure.

——-BEGIN RSA PRIVATE KEY——-

This is an RSA private key, which you can generate in OpenSSL with the genrsa command. The parameters it contains are all ints and are (with symbols which represent them in parentheses):

version (as in version of the file format)

modulus ( n )

) public exponent ( e )

) private exponent ( d )

) prime 1 ( p )

) prime 2 ( q )

) exponent 1 ( dp )

) exponent 2 ( dq )

) coefficient ( qinv )

Strictly speaking, all you need is n and d , but p , q , dp , dq , and qinv are used for the Chinese remainder theorem, and e is used for blinding, so you can create a public key out of the private one.

——-BEGIN EC PRIVATE KEY——-

This is an elliptical curve DSA private key:

version (an integer)

private key (an oct string)

named curve (an object id, but in some sort of over complicated arguments object as argument 0)

public key (a bit string, but also in the arguments object as argument 1).

The named curve and key parameters are part of a more complex structure defined in the ASN1 spec, but the EC private key is the only one which uses it, so if we took the time to learn it, that would be time we would never get back. We just need to thank Fedor for figuring it out for us.

——-BEGIN DSA PRIVATE KEY——-

For regular DSA keys, they have the following attributes (all ints):

version

p

q

g

y (public key)

(public key) x (private key)

Legacy encrypted private keys

I have alluded to how these keys do encryption badly. Here is how it works (most of which I learned from this Stack Exchange answer). If you want to protect your key with a password, then instead of

-----BEGIN RSA PRIVATE KEY----- MIICVAIBAAJ/OwswbFo/uyC8ltGf/yA1A+gV5IGdnAgPbUSI3GzbHCA+x+TLG/tL

your key will start with

-----BEGIN RSA PRIVATE KEY----- Proc-Type: 4,ENCRYPTED DEK-Info: AES-192-CBC,04D2D7882E0C474E07E542FE997D2A49 vfB5Gtm34n3SeI6JELjWiGw6O+j+tGR6Wbi3SNeAZkfSA8PTjei6PVHr+dGK5zMd

The first part of the DEK-Info is pretty easy to figure out: it’s the encryption algorithm and block mode. As for the second part, as you might remember from the post on ciphers, you can’t just use an English string as the key to a cipher, but it needs to be a binary sequence of a specific bit length and that to prevent the encryption from being deterministic you use a salt. (That is what the hex string that makes up the second part of the DEK-Info is: a salt.)

But what kind of salt is it – a salt for the key derivation function, or an initialization vector for the cipher? Trick question: it’s both. The string itself is the initialization vector for the cipher, and the first 8 characters are the salt for the key derivation function. For the record, don’t do this at home; a very nice attribute of key derivation functions is that they can make output of any length, so there is no reason not to use all of the string as the salt for the key derivation function and then use it to generate the key and the iv for the cipher.

Remember from part 1 that really really bad key derivation function EVP_BytesToKey which Node.js uses to create keys for ciphers? Guess what, that’s the key derivation function used here, and much like Node, it is used with a broken hashing algorithm (MD5 hard coded in, I might add), and a single iteration. On the one hand, it is nice that it’s using a salt, but on the other hand, the way it’s using the salt breaks a important rule of cryptography: Don’t reuse random numbers.

Long story short, if you had access to an encrypted key of somebody else’s, you could quite efficiently do a dictionary to decrypt it, so if you needed yet another reason to avoid these keys here it is.

Generic private keys

These are the keys that start with -----BEGIN PRIVATE KEY----- and define a wrapper around a private key from one of the algorithms mentioned above. The generic format is:

version (int)

algorithm info (sequence) algorithm id (object id) parameter (varies)

private key (oct string)

The algorithm parameters will vary depending on what it is and the private key is DER-encoded data as well, which must be decoded.

RSA

The algorithm ID is 1.2.840.113549.1.1.1 , the parameter is a null value, and the private key decodes to exactly the same 9 elements as the RSA specific key.

ECDSA

Note: I was never actually able to generate a generic elliptical curve key using the OpenSSL pkey command, but I was able to generate an encrypted one, which should decrypt to be the same thing.

The algorithm ID is 1.2.840.10045.2.1 , the parameter is an object id that lists the curve to use, and the private key decodes to the same 4 values as the EC specific key.

The only curve I dealt with was secp256k1 , and its ID is 1.3.132.0.10 .

DSA

The algorithm id is 1.2.840.10040.4.1 the parameter is a sequence of 3 integers which are p , q , and g , and the private key decodes to a sequence of a single integer which is x (the public key). Note that DSA is the only one where the generic version is not the same as the algorithm-specific one. This is because the p , q and g parameters are costly to generate and reusable, multiple parties in the same organization might use the same p , q , and g parameters, so they are more akin to the curve name in ECDSA, but unlike in ECDSA they are rather long (the size of p , g and y is the key length, g and x are about 1/10th of the key length), so including them twice would double the size of the PEM file. The public key ( y ) can be calculated with g pow x mod p , so omitting cuts the size down 30%.

Encrypted private keys

These keys are wrappers around encrypted generic private keys. They have the header -----BEGIN ENCRYPTED PRIVATE KEY----- and have the following format:

encryption info (sequence) encryption standard id (object id) parameters (sequence) key derivation info (sequence) key derivation id (object id) key derivation parameters salt (integer) iterations cipher information (sequence) cipher id (object id) initialization vector (oct string)

encrypted data (oct string)

The encryption standard is always going to be PBES2 (PBES1 is for working with older ciphers with key lengths of 60 bits and lower), so the encryption standard ID will always be 1.2.840.113549.1.5.13 , as far as I can tell the only key derivation algorithm that works with this is PBKDF2 with SHA1 with an object ID of 1.2.840.113549.1.5.12, PBKDF2 is an order of magnitude safer then EVP_BytesToKey, especially as by default more then 2000 iterations are specified by OpenSSL. SHA1 is likely safe for this now, but I’m not sure if that will continue to be the case. For ciphers, OpenSSL seems to support AES (in both 128, 192, and 256 variants) with ECB, CBC, OFB, and CFB modes (CRT mode isn’t supported and GCM doesn’t work as far as I can tell). You can view the IDs of the 12 working modes here.

Algorithm-specific public keys

Like their private cousins, these are depreciated, and as far as I can tell you can only make them for RSA keys in OpenSSL. They start with a header of -----BEGIN RSA PUBLIC KEY----- and contain a sequence of two integers: modulus ( n ) and public exponent ( e ). Don’t use this.

Generic public keys

These are like the generic private keys. In fact, they are almost exactly the same, except that they don’t include a version number as the first parameter, and the third parameter is a public key instead of a private key, which apparently means it needs to be a bit string instead of an octet string. Otherwise, the algorithm identifiers are the same and the algorithm parameters are also the same.

RSA

The public key bit string decodes to the same as the RSA-specific public key, a sequence of 2 integers n and e .

DSA

The public key bit string decodes to be an integer (not a sequence of a single integers like you’d expect, based on how every other thing here has done it). This is y (the public key).

EC

The public key bit string doesn’t contain anything DER-encoded, but is an encoded elliptical curve point (an uncompressed one to be specific).

Conclusions

When you serialize data, please choose a format that includes keys in addition to values. Nothing is worse than having to dig through the Internet tying to figure out what the values in a field represent and what order they are in. Also, those “well-known” OIDs for representing algorithms are never as well-known as you’d hope, so use human-readable ones. ASN1 has joined WKT-formatted projection info as one of my most-hated formats.

Package Size

On a more practical note, the pubic key aspects of the crypto module have caused it to start getting rather large, to deal with that we are breaking out the methods as much as possible into stand alone modules that like the inherits package uses the native crypto module on the server and the pure JavaScript version in the browser. So far we’ve got

create-hash which is identical to require('crypto').createHash

randombytes which is identical to require('crypto').randomBytes

create-hmac which is identical to require('crypto').createHmac

browserify-aes which provides an object with all 5 cipher related functions.

diffie-hellman which provides an object with getDiffieHellman and createDiffieHellman functions.

The rest should be done in the near future.

Thanks

I’d like to thank Nolan Lawson for editing this and all the other members of the crypto-browserify team Daniel Cousens, Dominic Tarr, Fedor Indutny, and JP Richardson.