Thursday, September 5, 2013

Widening the bell curve

My wife calls me a hero worshipper. My hero worshipping manifested itself at an early age by my love of He-Man and Superman. Thank goodness we did not have YouTube during my childhood and therefore will never again see 5-year-old-me running around in Superman underwear with a cape and boots. I grew out of that, but did not grow out of wanting to be a hero. In the years since my youthful admiration of fictional heroes I have learned that real heroes exist, but they often start out as normal people. Men such as Joshua Lawrence Chamberlain and Roger Bushell rose to the occasions into which they were placed. The key to becoming a hero is recognizing a need and then filling it.

Famous heroes manifest themselves in difficult situations when the need is great and filling it seems impossible.The horrors of war provide those circumstances in abundance. I like learning about wars for that reason. Heroes can appear anywhere and everywhere during a war. That includes anything from the battlefield heroics on Omaha beach to the decryption of Enigma at Bletchley Park. Considering those two scenarios as a computer geek, I can more easily picture myself assisting in the war effort alongside Turing and his bombes. Maybe that would not make a great Spielberg movie, but it could definitely fill a lengthy novel.

I recently finished Cryptonomicon, a historical/science fiction novel by Neal Stephenson that reviews the Enigma decryption effort in significant detail. A large portion of that story deals with a real practice that one character calls “widening the bell curve.” With regards to cracking Enigma, having it cracked only remained valuable so long as the Nazis did not know the Allies had broken it. Unbridled use of cracked information would yield short-term victories, but then Allies would tip their hand and reveal the weaknesses of Enigma. The Axis could then address those weaknesses and upgrade their encryption. On the other hand, not using the information rendered it equally useless. The trick became figuring out how use it without revealing knowledge of it. 

That brings us back to “widening the bell curve.” A normal distribution follows a bell curve: extremes happen much less often than the average. Something not following that probability, such as “magically” finding every Nazi transport, would appear as an extraneous bump on the bell curve. The information theory solution of widening the bell curve ahead of time tries to avoid exposing that bump. Widening occurs by adding additional data points to smooth out the distribution, and must be added in many places to ensure a smooth and symmetrical bell shape. Figuring out just what to do to maintain the normal bell curve became the job of one protagonist in Cryptonomicon.

The Allies, both in the book and in the real war, introduced additional data points through various means. Instead of magically showing up with a fleet to attack a transport, they sent out “spotter” submarines and aircraft to “find” the transport. Even better, they sent out additional spotters to other locations, providing false negatives. The Allies had to occasionally ignore information from the Enigma because acting on it would reveal the cypher’s weakness.. At one point the Allies transmitted a congratulation to a non-existent spy in Italy for the excellent information provided, which covered up an out-of-the-blue convoy attack. These and similar techniques resulting in the Enigma cracking remaining secret until publicly disclosed nearly three decades later. The most accurate word to describe the effort involved to maintain that secrecy is heroic.

This all has direct applications to today’s security landscape. We can look for irregularities in computer usage, or bumps on the bell curve, to indicate a threat. However, in order to find the bumps we must already know the shape of the curve. Bad actors will try to widen the bell curve in order to avoid detection. Organizations something to fill the role of identifying washed-out bumps. Bit9 can map the curve and identify the bumps so you can become a security hero!