Software powers social networks, controls vast supply chains, gets astronauts to the moon and back, and saves lives. It can also screw up horribly—causing programs to crash, systems to become vulnerable, and entire servers or data centers to go down.

And it doesn't necessarily take a lot of code to do this—just enough of exactly the wrong thing. Here's a salute to the truly terrible choices developers have made and, sadly, will make. Sometimes you only learn by making mistakes, but maybe some people can learn from others and spot the pitfalls ahead before they fall in.

System-breaking code

Just a little code can do a lot of damage. This is why you need to know what you're doing when the safety measures are off.

Delete the company

There may be times that you want to delete an old directory. But when you take out the entire drive, from the root down, in Unix, it's ugly. Better pray you have a recent backup.

The command:

rm -rf

In context:

sudo rm -rf --no-preserve-root /mnt/hetznerbackup /

The space before the last backslash is what unintentionally caused the self-destruct.

Lose all your customers

"[PHP] comes with a host of functions that are important to turn off unless you specifically need them," said Job Brown, web team leader of U.K.-based e-commerce vendor Wooden Blinds Direct. That includes the ability to eradicate your entire set of customers. "That semicolon is the difference between dropping your whole customers table rather than just the customer with an ID of 1," he said.

DROP FROM Customers; where id = 1

One more time

Another example of a potential disaster comes from a Quora answer by Colin Turner, a professor of engineering education at Ulster University. It's a fork bomb written in C that will repeatedly create other processes until the system completely runs out of resources because there is no condition for termination.

#include int main(void) { while(1) fork(); }

Watch those typos

Over at Stack Exchange, there was a discussion of bad programming mistakes in C. Some of the classics revolve around the theme of having typos in conditional statements that always cause the condition to evaluate to true. Here are a couple of variations. In the first example, variable c is assigned the value 1, while the second example always evaluates to a true condition, so any code always executes:

if (c = 1);



if (a == true);

Overlapping scheduled processes

John Chapin of Capital Technology Services remembers a project that called the Unix cron(). The original developer wrote a task to track user activity by the hour. "But there was also an odd condition that the server would inexplicably run out of memory and need to be rebooted every few weeks," he said. As it turned out, one round of analysis wouldn't be done before the next started, and eventually the overlapping processes took up all the system resources. His team found the problem and solved it by breaking the processing up into pieces.

Problem:

5 * * * * cd /var/www/myapplication && rake RAILS_ENV=production statistics:calculate

Fixed:

5 5 * * 0 cd /var/www/myapplication && rake RAILS_ENV=production statistics:annual

5 1 * * 0 cd /var/www/myapplication && rake RAILS_ENV=production statistics:monthly

5 1 * * * cd /var/www/myapplication && rake RAILS_ENV=production statistics:weekly

5 * * * * cd /var/www/myapplication && rake RAILS_ENV=production statistics:daily

Way too long

Chapin also had another good nomination, but one that can't be shown because it won't fit on a screen, which was the problem. It was in a Ruby on Rails controller that gathered all clients within a set number of days and then would do some processing.

"Every time the query ran, it would roll through an entire table of 250,000 records about 20 times," taking two minutes, he said. The problem was that the method was all on one line and it was so long that no one could parse it. Reformatting and changing the table's keys eventually dropped the load time to 10 seconds.

If you could read this line of code, then it wouldn't be a problem.

Error-ignoring routine

One Stack Exchange poster mentioned a piece of commercial code that kept crashing. The company wondered why. This error-handling (or error-ignoring) routine was the problem:

/* FIXME! */

while (TRUE)

;

Security vulnerabilities

One of the biggest dangers that emerge from coding flaws is making software more vulnerable to attack or misuse.

Opening the door to the system

Chen Levkovich, CEO of Zuznow, says that using the exec command in PHP is a mistake because it can give someone root access. "If you don't properly pass parameters without thinking about security, there's a chance someone [could break in]," he said.

Vulnerable:

exec($args, $output, $return_var);

Fixed:

$escaped_command = escapeshellcmd($args);

exec($escaped_command, $output, $return_var);

SQL statement passing

According to Oliver Lavery, vice president of research at IMMUNIO, a real-time web application security vendor, directly passing SQL statements opens a door for a SQL insertion attack. "The attacker can cause the program to confuse what it thinks is data and control statements," Lavery said. By sending a statement to end the data stream, the attacker can then send commands to take control of the system. Instead, pass parameters. Here's a Java example followed by a better way.

Vulnerable:

sqlQuery='SELECT * FROM custTable WHERE User=' + Username + ' AND Pass=' + password

Fixed:

sqlQuery='SELECT * FROM custTable WHERE User=? AND Pass=?'

parameters.add("User", username)

parameters.add("Pass", password)

The dangerous curl command

David Pellerin, a senior tech lead with custom software designer TWG, is amazed that the Unix curl command, used for testing or mirroring websites, allows an insecure mode that effectively bypasses SSL/TLS, trusting that all sites are on the up and up. "For local development and testing, this is fine, but my main point is that sometimes these tools get used in production scenarios, and I feel like it should be much harder to bypass these security features than by simply passing in a simple argument like that," Pellerin said.

$ curl –insecure

Bad hashing

Verifying a password can go wrong easily. Developers may assume that cryptography is enough. "But a lot of times that web application has a config file on the server with the password to the key," Pellerin said. Even hashing can go wrong, as with the Ruby code below that uses hashing algorithm SHA256 with no salt value. "This is dangerous because many programmers will think, 'Oh this must be secure—I'm using a one-way hashing function!' But in reality the hashed data can be cracked in under one second using 'rainbow tables.'"

require 'digest/sha2'

sha256 = Digest::SHA2.new(256)

sha256.hexdigest("password")

result = 5e884898da28047151d0e56f8dc6292773603d0d6aabbdd62a11ef721d1542d8

Now go to the following website:

https://crackstation.net/

Paste in "5e884898da28047151d0e56f8dc6292773603d0d6aabbdd62a11ef721d1542d8" (without the quotes), and click "Crack Hashes."

In one second you get a green message showing the decrypted phrase "password."

Taking any old filename

Even a simple line of code to upload a file can be a problem when it accepts whatever name a user provides. This is considered one of the top coding errors by the SANS Institute and Mitre. If the file has an extension such as .php rather than .gif or .jpg, the server might treat the file as an executable program (which is probably what it is). Without additional code to restrict the file type (or even do exception checking), the following C snippet is a danger:

fgets (filename_array; filename_length; stdin);

Deliberate sabotage

As Jeff Williams, CTO and co-founder of Contrast Security, wrote in a paper published at the 2009 Black Hat conference, someone might bribe a developer to insert code snippets to cause damage. Testing and review should consider such problems. Here's an example that would allow a third party to steal a file off a production server:

String x = req.getParameter( "x" ); BufferedReader r = new BufferedReader( new FileReader( x ) ); while ( ( x = r.readLine() ) != null ) resp.getWriter().println( x );

Real-world consequences

While many of the problems discussed above have been theoretical, the next few examples are bugs that actually happened. Perhaps you've seen some of those "worst bugs ever" lists. Just recently Delta Airlines' systems went down, although that was likely due to a power failure at its data center rather than a software bug. But it just illustrates how major software failures are affecting us all the time.

Buffer overflow and ATM dominoes

A C programmer mentioned on this Stack Exchange discussion a problem that appeared through the poorly advised combination of an assumption and use of the strcpy() function without checking the size of a target buffer. When writing the software for a router for credit card transactions, a programmer assumed that a nine-digit string would be long enough for the necessary bank identification number (BIN). Then the bank moved to a ten-digit BIN, and the transaction router crashed with a segmentation fault after the buffer overflow. The entire processing system that depended on the router followed suit, and as a result, all the ATMs of this large bank stopped working for a few hours.

Regex unbound

Little things can become a big problem. The Stack Exchange network had its own problem in July 2016 because of the following regular expression that was supposed to cut Unicode space from the start and end of a line. A "malformed post" with about 20,000 white space characters in a row played badly with the regex engine, which kept cycling through, one space at a time, eating up CPU time, as Stack Exchange explained. The code crashed the site for 34 minutes. Here's the responsible code:

^[\s\u200c]+|[\s\u200c]+$

Typo takes down site

Brown, from Wooden Blinds Direct, remembers a time he left a semicolon off the end of one line of PHP code. The company has a group of e-commerce sites with a common code base, and all of them crashed simultaneously. "It doesn’t have to be a complex error to take down your systems," Brown said.

function Oops($arg_1, $arg_2, $arg_3)

{

return $my_bad

}

Backup carousel

Brown saw another problem due to an external backup provider's scripting. Wooden Blinds runs hourly backups on "fairly hefty" databases, and the vendor hadn't finished one backup when another was supposed to start. "The provider hadn't accounted for [such a situation] in their code," he said. As a result, the backups kept looping until, by 6 am, there were "thousands of backups trying to run." Things ground to a decisive halt.

Meta madness

One of the biggest problems that Job van der Voort, vice president of product at GitLab, ever saw in practice was at a previous employer. A developer, who had already left the company, created a production system in Ruby using meta programming, with classes creating additional classes based on a few parameters.

"Initially the only reason they started looking at this was because the program was so slow," he said. A single page of code would make 2,000 database calls rather than the 1 to 25 that would be desired to optimize for speed. It took him a week to get the count down to 500, but eventually the company gave up and began a complete rewrite. Because so many of the classes were virtual and dependent on particular use scenarios, there was nothing permanent to even read, let alone test and debut. Here's a simple example that van der Voort provided:

a_new_class = Class.new(Object) do

attr_accessor :x

def initialize(x)

puts "Initialized #{self.class} with #{x}"

@x = x

end

end

When old methods go wild

One system that collapsed under the combined weight of a programming mistake and errors in deployment was the trading system of Knight Capital Group. In 2012 the company lost $440 million in 45 minutes "when it sold all the stocks it accidentally bought Wednesday morning because of a computer glitch," according to The New York Times.

As an SEC administrative proceeding noted, the company was replacing some years-old code that was no longer used but still callable in the application. It also repurposed a flag used in calling the old code. When a technician forgot to copy the new code to one server, the old code got activated and started sending child orders to some trading systems. There were no provisions for supervising these problems in general, and technicians took the new code off the other servers, putting the problem into overdrive as all the servers began to make the same error. The company eventually had to sell itself later in the year because of this incident.

Financial code forked over

Financial services can be a tricky area for coding. Lavery, of IMMUNIO, remembers in years past writing a data parser for stock exchanges. In his code he used a fork to create some additional processes. "I got an if/then condition wrong which was a test for when to stop forking," Lavery said. "It ended up forking endlessly. It took down our system and had hundreds of connections to the back end." Communications bogged down, which meant clients were losing a lot of money by not completing transactions quickly enough.

Have you ever seen or created a catastrophic bug? Tell us about it in the comments section.

Image credit: Flickr

Keep learning