As a known fact that the security of a computer system is only as strong as its weakest link. As a system gets more complicated, and more people get involved in designing and implementing, it is also getting easier for that weak link to slip into that system.
An investment company is interested in using our product, but as a paranoid government organisation, they got some security consultants to perform some auditing. Last Friday they claimed that they have successfully took over the entire system without going through authentication. When I first heard it, I was going, "Nah", it can't be possible. All the calls coming from the web server has to obtain the session instance to perform any operation on the application server. The system was designed to be like this when we first started, and tt is just not possible for someone to bypass session verification. But my firm confidence collapsed when I verified the logs from the hacked application server.
It was in one of the web server code, someone took a query parameter and wack it to the back of a path name on the file system to return that file. There was no checking whether the resulting file name sits inside the jailed directory, and it opened up a huge security hole in the system as an attacker can manipulate the web server to retrieve any file on the hosted box. Bad. Very very bad.
Well, over taking the whole system is a bit exaggerated. Moreover, this piece of dodgy code only existed in an older version of our software, as the newer version used a totally different mechanism to achieve the same thing, which is not vulnerable. However, I still felt furious and frustrated when someone's piece of broken hack can contaminate a multi-tier application server that was supposed to be "secure". While we have quickly patched that older version and made the patch available within a few hours of discovery, I still could not get over my mind that one can so carelessly code like that. Moreover, I am not sure how many of this kind of hack have somehow committed into the repository, under company's flexible project development model. I might need to do some code audit sometime - to find that weakest link.