Venona project
The Venona project was a United States counterintelligence program initiated during World War II by the United States Army's Signal Intelligence Service and later absorbed by the National Security Agency, that ran from February 1, 1943, until October 1, 1980. It was intended to decrypt messages transmitted by the intelligence agencies of the Soviet Union.
During the 37-year duration of the Venona project, the Signal Intelligence Service decrypted and translated approximately 3,000 messages. The signals intelligence yield included discovery of the Cambridge Five espionage ring in the United Kingdom, and also of Soviet espionage of the Manhattan Project in the US, known as Project Enormous. Some of the espionage was undertaken to support the Soviet atomic bomb project. The Venona project remained secret for more than 15 years after it concluded.
Background
During World War II and the early years of the Cold War, the Venona project was a source of information on Soviet intelligence-gathering directed at the Western military powers. Although unknown to the public, and even to Presidents Franklin D. Roosevelt and Harry S. Truman, these programs were of importance concerning crucial events of the early Cold War. These included the Julius and Ethel Rosenberg spying case and the defections of Donald Maclean and Guy Burgess to the Soviet Union.Most decipherable messages were transmitted and intercepted between 1942 and 1945, during World War II, when the Soviet Union was an ally of the US. Sometime in 1945, the existence of the Venona program was revealed to the Soviet Union by cryptologist-analyst Bill Weisband, an NKVD agent in the US Army's SIGINT. These messages were slowly and gradually decrypted beginning in 1946. This effort continued through 1980, when the Venona program was terminated. The analyst effort assigned to it was moved to more important projects.
To what extent the various individuals referred to in the messages were involved with Soviet intelligence is a topic of minor historical [|dispute]. Most academics and historians have established that most of the individuals mentioned in the Venona decrypts were probably either clandestine assets and/or contacts of Soviet intelligence agents, and very few argue that many of those people probably had no malicious intentions and committed no crimes.
Commencement
The VENONA Project was initiated on February 1, 1943, by Gene Grabeel, an American mathematician and cryptanalyst, under orders from Colonel Carter W. Clarke, Chief of Special Branch of the Military Intelligence Service at that time. Clarke distrusted Joseph Stalin, and feared that the Soviet Union would sign a separate peace with Nazi Germany, allowing Germany to focus its military forces against the United Kingdom and the United States. Cryptanalysts of the US Army's Signal Intelligence Service at Arlington Hall analyzed encrypted high-level Soviet diplomatic intelligence messages intercepted in large volumes during and immediately after World War II by American, British, and Australian listening posts. Frank Rowlett was one of the project leaders.Decryption
This message traffic, which was encrypted with a one-time pad system, was stored and analyzed in relative secrecy by hundreds of cryptanalysts over a 40-year period starting in the early 1940s. When used correctly, the one-time pad encryption system, which has been used for all the most-secret military and diplomatic communication since the 1930s, is unbreakable. However, due to a serious blunder on the part of the Soviets, some of this traffic was vulnerable to cryptanalysis. The Soviet company that manufactured the one-time pads produced around 35,000 pages of duplicate key numbers, as a result of pressures brought about by the German advance on Moscow during World War II. The duplication—which undermines the security of a one-time system—was discovered, and attempts to lessen its impact were made by sending the duplicates to widely separated users. Despite this, the reuse was detected by cryptanalysts in the US.Breakthrough
The Soviet systems in general used a code to convert words and letters into numbers, to which additive keys were added, encrypting the content. When used correctly one-time pad encryption is unbreakable. However, cryptanalysis by American code-breakers revealed that some of the one-time pad material had incorrectly been reused by the Soviets, which allowed decryption of a small part of the traffic.Generating the one-time pads was a slow and labor-intensive process, and the outbreak of war with Germany in June 1941 caused a sudden increase in the need for coded messages. It is probable that the Soviet code generators started duplicating cipher pages in order to keep up with demand.
It was Arlington Hall's Lieutenant Richard Hallock, working on Soviet "Trade" traffic, who first discovered that the Soviets were reusing pages. Hallock and his colleagues, amongst whom were Genevieve Feinstein, Cecil Phillips, Frank Lewis, Frank Wanat, and Lucille Campbell, went on to break into a significant amount of Trade traffic, recovering many one-time pad additive key tables in the process.
A young Meredith Gardner then used this material to break into what turned out to be NKVD traffic by reconstructing the code used to convert text to numbers. Gardner credits Marie Meyer, a linguist with the Signal Intelligence Service with making some of the initial recoveries of the Venona codebook. Samuel Chew and Cecil Phillips also made valuable contributions. On December 20, 1946, Gardner made the first break into the code, revealing the existence of Soviet espionage in the Manhattan Project. Venona messages also indicated that Soviet spies worked in Washington in the State Department, Treasury, Office of Strategic Services, and even the White House. Very slowly, using assorted techniques ranging from traffic analysis to defector information, more of the messages were decrypted.
Claims have been made that information from the physical recovery of code books to bugging embassy rooms in which text was entered into encrypting devices contributed to recovering much of the plaintext. These latter claims are less than fully supported in the open literature.
One significant aid in the early stages may have been work done in cooperation between the Japanese and Finnish cryptanalysis organizations; when the Americans broke into Japanese codes during World War II, they gained access to this information. There are also reports that copies of signals purloined from Soviet offices by the Federal Bureau of Investigation were helpful in the cryptanalysis. The Finnish radio intelligence sold much of its material concerning Soviet codes to the OSS in 1944 during Operation Stella Polaris, including the partially burned code book.
Results
The NSA reported that thousands of cables were sent, but only a fraction were available to the cryptanalysts. Approximately 2,200 messages were decrypted and translated; about half of the 1943 GRU-Naval Washington to Moscow messages were broken, but none for any other year, although several thousand were sent between 1941 and 1945. The decryption rate of the NKVD cables was as follows:- 1942: 1.8%
- 1943: 15.0%
- 1944: 49.0%
- 1945: 1.5%
The existence of Venona decryption became known to the Soviets within a few years of the first breaks. It is not clear whether the Soviets knew how much of the message traffic or which messages had been successfully decrypted. At least one Soviet penetration agent, British Secret Intelligence Service representative to the US Kim Philby, was told about the project in 1949, as part of his job as liaison between British and US intelligence. Since all of the duplicate one-time pad pages had been used by this time, the Soviets apparently did not make any changes to their cryptographic procedures after they learned of Venona. However, this information allowed them to alert those of their agents who might be at risk of exposure due to the decryption.
Significance
The decrypted messages gave important insights into Soviet behavior in the period during which duplicate one-time pads were used. With the first break into the code, Venona revealed the existence of Soviet espionage at the Manhattan Project's Site Y. Identities soon emerged of American, Canadian, Australian, and British spies in service to the Soviet government, including Klaus Fuchs, Alan Nunn May, and Donald Maclean. Others worked in Washington in the State Department, the Treasury, OSS, and even the White House.The messages show that the US and other nations were targeted in major espionage campaigns by the Soviet Union as early as 1942. Among those identified are Julius and Ethel Rosenberg, Alger Hiss, Harry Dexter White, Lauchlin Currie, and Maurice Halperin.
The identification of individuals mentioned in Venona transcripts is sometimes problematic, since people with a "covert relationship" with Soviet intelligence are referenced by cryptonyms. Further complicating matters is the fact the same person sometimes had different cryptonyms at different times, and the same cryptonym was sometimes reused for different individuals. In some cases, notably Hiss, the matching of a Venona cryptonym to an individual is disputed. In many other cases, a Venona cryptonym has not yet been linked to any person. According to authors John Earl Haynes and Harvey Klehr, the Venona transcripts identify approximately 349 Americans who they claim had a covert relationship with Soviet intelligence, though fewer than half of these have been matched to real-name identities. However, not every agent may have been communicating directly with Soviet intelligence. Each of those 349 persons may have had many others working for, and reporting only to, them.
The OSS, the predecessor to the Central Intelligence Agency, housed at one time or another between fifteen and twenty Soviet spies. Duncan Lee, Donald Wheeler, Jane Foster Zlatowski, and Maurice Halperin passed information to Moscow. The War Production Board, the Board of Economic Warfare, the Office of the Coordinator of Inter-American Affairs, and the Office of War Information, included at least half a dozen Soviet sources each among their employees.