Finding the American Black Chamber in New York

Herbert Yardley and the American Black Chamber

The U.S. Government's code-breaking operation was based in midtown Manhattan from 1918 until 1921. This was MI-8, or Military Intelligence, Section 8, the special bureau handling all aspects of cryptology. They created U.S. codes and ciphers and work to break those of other nations. The "American Black Chamber was run by Herbert O. Yardley, who based it in some brownstones on 37th Street and 38th Street. Let's visit New York and see what can be seen today of the American Black Chamber's operation.

Herbert Yardley was born in Worthington, Indiana in 1889. His father was a railroad station master and telegrapher, and he taught telegraphy to Herbert.

After graduating from high school in 1907, Yardley worked as a telegraph operator for the railroad while teaching himself poker, concentrating especially on the probability involved.

Now this was real poker and not the silly "Texas Hold-em" card game based on bluffing and guessing when the other players are bluffing. Yardley's game required serious understanding of probability.

He applied his poker winnings toward further education, and in 1912 he passed the civil service exam and was hired as a telegraph operator for the federal government. He began as a code clerk in the State Department, where he discovered that he was able to break the current U.S. government codes.

Yardley was startled when he easily broke the code used by President Woodrow Wilson and then was even more surprised to learn that the code had been in use for over ten years. This extreme cryptographic weakness while World War I raged in Europe (the U.S. would not get directly involved until April 1917) led Yardley to write a hundred-page paper "Solution of American Diplomatic Codes". He submitted it to his superior, who became quite angry as he had been the developer of most of these weak codes.

The U.S. entered the war in April 1917, and Yardley was sent to the War Department and then on to the U.S. Army Signal Corps. He was given a Reserve commission as Lieutenant, and named the head of MI-8, or Military Intelligence, Section 8. MI-8 was a special bureau dealing with cryptology, both the creation of U.S. codes and ciphers, and the cryptanalytic attack on those of other nations.

The small MI-8 under Yardley's leadership had broken almost all German diplomatic and military codes within a few months. One prominent success was the decryption of a message found on Lothar Witzke, a German saboteur who was arrested after entering the U.S. from Mexico. Witzke had a 424-letter cryptogram sewn into the left upper sleeve of his jacket. The resulting cleartext proved that he was a German saboteur, one of those responsible for sabotage on Mare Island Naval Shipyard near San Francisco, and probably the "Black Tom" explosion in New York harbor in June 1916. Witzke was tried and convicted, sentenced to death, the only German agent to be so sentenced, although President Calvin Coolidge issued a reprieve in November 1923 just before the sentence was to be carried out.

By then promoted to major, partly thanks to the Witzke case, Yardley went to Britain in August 1918. There he met Vernon Kell, head of the U.K.'s MI5, and Admiral William Reginald Hall, head of British naval intelligence and the director of the Room 40 group. Room 40 had broken the Zimmermann Telegram which finally brought the U.S. into the war. From there he traveled on to France and met with the French cryptography bureau, known as the Chambre Noire or Black Chamber.

The war ended 11 November 1918, and there was call for MI-8 to be disbanded. However, General Marlborough Churchill, head of U.S. Army Intelligence and a distant relative of Britain's Winston Churchill, had read Yardley's post-war report "Code and Cipher Investigation and Attack". General Churchill was determined that the United States must not foolishly throw away a valuable resource.

Churchill managed to persuade State Department officials to fund a covert and deniable cryptanalysis operation. It would analyze and attack foreign codes and ciphers, but for legal reasons it was to be located in New York, not Washington.

Or at least Yardley's explanation was that it was some legal constraint — Yardley himself later said it was get out of Washington as it was "overrun with spies", and it's also possible that Yardley chose New York because his close friend and head of MI-8's Shorthand Subsection, Franklin Allen, was familiar with New York from operating there during the war.

Whatever the reason for the New York location, MI-8 operated there covertly. Ties to the government were cut. Employees were paid in cash from a secret payroll and had no civil servant status. Official mail from Washington was received at a post office box, at least some of the time at box number 354, Grand Central Station. They established the Code Compiling Company as a cover, a producer of commercial code books. One of them, the "Universal Trade Code", sold in fairly large numbers.

Yardley organized a team of cryptanalysts largely based on his MI-8 staff. Yardley named his operation the American Black Chamber in honor of the French Chambre Noire. It was also known as the Cipher Bureau. This was the United States' first peacetime cryptanalytic organization, responsible for breaking the codes of other nations. It was a predecessor, although not a direct one for reasons explained below, of the National Security Agency.

The State Department and War Department had special arrangements with commercial telegraph companies, including at least the Western Union and Postal Telegraph companies. During the first year the work focused on Latin American countries (remember the Zimmermann Telegram and its Mexican destination) plus Germany, Japan, and Spain.

The Black Chamber soon broke the codes of the Чека (Cheka), the Bolshevik secret police that had replaced the Czar's Охрана (Okhrana) and which eventually evolved into the KGB and today's ФСБ or FSB. It was then asked to attack the Japanese diplomatic codes.

Given the quick break of the Japanese ciphers, the Black Chamber was able to contribute significantly to the partial containment of Japan at the 1921 Washington Naval Conference, often referenced as one of the Black Chamber's greatest accomplishments but perhaps the cause of its eventual disbanding.

This post-World-War-I international conference was intended to reduce tension and make major war less likely by limiting nations' total warship tonnage. Japan had won significant naval victories over Russia's Pacific Fleet in the Russo-Japanese War of 1904-1905.

The Black Chamber decrypted the intercepted messages between the Japanese emissaries and their home government. What was even more sensitive was that they were also breaking the messages of American allies. The U.S. negotiators had a huge advantage at that conference.

The U.S. position called for a 10:6 ratio, for the total tonnage of Japanese military warships to be limited to 60% of that of the U.S. fleet. The Japanese negotiators insisted on a 10:7 ratio, 70% rather than 60% of U.S. tonnage. However, the Black Chamber discovered that the Japanese government in Tokyo was telling its delegation to eventually compromise at 10:6 or 60% if needed to avoid a confrontation. Given that knowledge, the U.S. side simply stood firm until the Japanese delegation accepted 10:6.

Despite its successes and obvious utility, the State Department significantly reduced the Black Chamber's funding in 1924. Then, in 1929, Henry L. Stimson was named Secretary of State. Stimson was offended by espionage and any type of covert action, and he imperiously and naïvely issued the infamous directive "Gentlemen do not read each other's mail." The American Black Chamber was shut down on 1 November 1929.

Black Chamber locations

The American Black Chamber was first established at 3 East 38th Street, just off Fifth Avenue.

This is a telephoto view down Fifth Avenue from 38th Street late in the day. The Empire State Building, filling the block between 33rd and 34th Streets, is the building with the flags on the right side of the Avenue.

The location at 3 East 38th Street was a four-story brownstone building at the north-east corner of the intersection, just across the avenue from the Lord and Taylor department store.

The American Black Chamber was established here in the summer of 1919, but just operated out of this location for about a year.

The department store is still there and is still operated by Lord and Taylor, but the brownstone is gone. It has been replaced by the modern building seen in the second picture above. That view is toward the west along 38th Street to the intersection with 5th Avenue.

Presumably at least some of these buildings seen below were neighbors of the brownstone. The building housing the Butterfield 8 bar is the first building to the east of Fifth Avenue not built recently. Butterfield 8 now has the address 5 East 38th Street, so if the street numbering has not changed, it is the first address to the east of the original Black Chamber.

The lease to the brownstone was sold in 1920, and the American Black Chamber moved a short distance to another brownstone between Lexington and Third Avenues at 141 East 37th Street on 1 July 1920. That brownstone is seen above and below. By now the Japanese codes and ciphers were the main priority, including Japanese diplomatic, military, and naval attaché codes. The State Department passed along its 1924 budget cuts in the form of greatly reduced support to MI-8, and Yardley was forced to cut more than half his staff. MI-8 was down to just six employees including Yardley.

The greatly reduced MI-8 was moved in 1924 to an office building at 52 Vanderbilt Avenue. That is seen in the first picture below as the first building on the right side of the avenue with a visible near corner. It is the building beyond the white truck. Vanderbilt is a relatively short north-south avenue running along the west side of Grand Central Station.

They had access to less material, apparently because the telegraph companies were becoming more and more reluctant to pass along copies of their traffic. Yardley wrote, in a later book discussed below, of being "forced to adopt rather subtle methods" of obtaining traffic, methods not always supported by those to whom he reported, presumably illegal methods such as break-ins, theft, and bribery. Meanwhile Yardley received another Distinguished Service Medal in 1926, and through the mid and late 1920s MI-8 was successfully attacking and exploiting cryptosystems of Japan, China, Mexico, Nicaragua, and Peru. Yardley's operation basically ceased operation in June or July of 1929, when the State Department announced the coming end of funding. The office was not officially closed until the first of November 1929. William F Friedman arrived in New York in October to transfer the files and records to Washington, where they soon were used by the U.S. Army's newly formed Signal Intelligence Service. The SIS eventually led to the National Security Agency.

The government soon offered Yardley a temporary position at far less pay, and later another position at half his previous pay, apparently both times in the expectation that he would not accept the offers. In 1930 he sought a position as an instructor for the U.S. Navy's cryptanalytic research section, but was turned down, probably on the grounds that he had no experience with current Japanese naval cryptosystems. Yardley was out of a job and unable to find another. This wasn't surprising, given that the few organizations looking for his set of skills had mostly been targets of his cryptanalysis. He then wrote a book that became immensely popular and simultaneously brought him harsh criticism lasting to this day, The American Black Chamber. He published three articles in the Saturday Evening Post, then the leading mass circulation weekly. "Secret Inks" on April 4, 1931; "Codes" on April 18, and "Ciphers" on May 9. On June 1, 1931, The American Black Chamber was released. The second of those magazine articles included a photograph of a code book page, showing that U.S. military intelligence was working on British diplomatic codes in 1921. The book exposed much more.

The American Black Chamber was denounced by the U.S. government as having given away American secrets. Not the secrets of how to perform cryptanalysis, but the secrets about how and when it had been applied — not only against likely adversaries such as Japan, but also against U.S. allies. This led to Public Law 37, signed by Franklin D. Roosevelt in 1933, criminalizing any such future exposure. Further publication of The American Black Chamber was halted, although it was later allowed back into publication.

Yardley next wrote a book titled "Japanese Diplomatic Codes: 1921-1922", but the government cited Public Law 37 and seized the manuscript before it was printed. The manuscript was classified until 1979, when it was finally declassified.

By 1932 the U.S. Army had resumed cryptanalytic work, although only on a part-time and very limited scale.

Yardley went to China in 1938 and worked until late 1940 with Chiang Kai-shek's intelligence bureau. Another book of his memoirs, The Chinese Black Chamber, was declassified and published in 1983.

In early 1941 he was in Canada helping establish a cryptanalysis bureau. After the U.S. entry into World War II in December 1941, Yardley returned to the U.S. and was given a government job shuffling meat inspection forms for the Office of Price Administration. Meanwhile, William F Friedman and others were breaking German and Japanese ciphers, using Yardley's unpublished manuscript of "Japanese Diplomatic Secrets" at least for some initial guidance.

In 1942, Yardley ran a restaurant in Washington, DC. It came under unfounded suspicion of being a meeting place for Fascist agents. See my other page for the story about Yardley's restaurant and the FBI investigation.

He wrote and co-wrote several magazine articles and some thrillers, "The Blonde Countess", "Red Sun of Nippon", and "Crows Are Black Everywhere", featuring fictionalizations of his exploits overseas and some of the world of his Black Chamber, although it is still debated as to how much he actually wrote of some of these. See the journal article, "Who Wrote 'The Blonde Countess'? A Stylometric Analysis of Herbert O. Yardley's Fiction", Cryptologia, 33:2 (April 2009), pp 108-117. The movie Rendezvous was based very loosely on "The Blonde Countess".

Well after the war, in 1957, Yardley published "The Education of a Poker Player", describing a powerful system for playing real poker where an understanding of probability is required. Sales were strong, but Yardley died in 1958.

David Kahn has written a typically excellent biography of Yardley, The Reader of Gentlemen's Mail.

Another source of information on Yardley is "The Many Lives of Herbert O. Yardley", published in the NSA's "Cryptologic Spectrum" journal in 1981. As you might imagine, the tone of that article is disapproving at best.

The NSA has other full books available on-line in their collection of World War II history, among other areas of their on-line history library.

Also see my pictures of the site of Yardley's restaurant in Washington, DC, where he fell under baseless FBI suspicion in 1942.