Tag Archives: static analysis

Development Testing – Is It Worth It?

I delivered a webinar yesterday as part of my day job at Parasoft . The topic was “Development Testing – Is It Worth It?”.

I talked about the reasons why Development Testing is useful, how it relates to process, policy, and how you can move from a reactive process of finding bugs to a proactive process of writing code that is resistant to bugs. As the old saying goes, an ounce of prevention is worth a pound of cure, or a week of debugging in this case.

The presentation is general, not designed to push any specific tools, so you should find it helpful. I’ve made both the powerpoint slides and audio available below. You can usually access most of the past webinars and other video content on the Parasoft site.

Take a look for yourself. If you have any comments or suggestions, feel free to mention it in the comments, or email me or reach me on twitter.

(powerpoint) (mp3 audio)

Download (PPTX, 2.52MB)

The webinar invitation read:

Development Testing: Is it Worth It?

Development Testing is a lot like exercising and eating well: pretty much everyone agrees that it’s beneficial and should be done, but few actually achieve it in practice.

A rising number of organizations are flirting with Development Testing by giving developers a static analysis tool and hoping that they use it to prevent defects. This is not unlike packing some raw broccoli and spinach in your son’s lunch box and expecting his health to improve as a result. This approach to Development Testing inevitably fails to deliver the results that organizations have been able to achieve with a comprehensive, policy-driven Development Testing process: reduced risks while cutting development time and costs.

If you can’t bear the business risk associated with defects surfacing in your organization’s software, join our webinar—Development Testing: Is it Worth It?—to learn how to get the maximum risk reduction from your investment in Development Testing. After exploring the dangers of relying on static analysis alone and the top barriers to comprehensive Development Testing, you’ll learn how Parasoft’s comprehensive Development Testing platform can help you:

  • Consistently apply a broad set of complementary Development Testing practices—static analysis, unit testing, peer code review, coverage analysis, runtime error detection, etc.
  • Accurately and objectively measure productivity and application quality
  • Drive the development process in the context of business expectations—for what needs to be developed as well as how it should be developed
  • Gain real-time visibility into how the software is being developed and whether it is satisfying expectations
  • Reduce costs and risks across the entire SDLC

SQL Injection is So “2000-and-Late”

Planking p019 © by uvw916a

I’m kind of surprised, or at least disappointed that we are still talking about SQL injection breaches. About a year ago I wrote about SQL Injection and yet it’s still hitting major web sites. For example Hackmageddon has an interesting chart of cyber attacks in June (not just SQL injection). But for me writing about SQL injection feels a bit like writing about planking, it’s so 2000-and-late.

Companies that are losing passwords, especially plain-text passwords via SQL injection are passing from bad-luck to poor-quality to downright negligent. Adrian Lane of Securosis LLC suggest that hearing the same old headlines about it has made it uninteresting. In his words:

SQL injection? Not sexy, but it sure is effective.

The latest entry in the SQL Injection Hall of Shame is Yahoo (YHOO). I’m sure you’ve heard by now about the hundreds of thousands of user email addresses and plain-text password that were stolen from Yahoo using SQL injection. There is much to be learned from this event.

  • SQL injection attacks are completely preventable
  • Using static analysis flow-analysis tools will not catch all security errors
  • Storing plain-text passwords is inexcusable
  • Substandard security can lead to liability


Let’s make this really simple. SQL injection is preventable. I like the way Randy Abrams, research director at NSS Labs put it.

The only place we should be seeing SQL injection attacks today is in the classroom, as IT professionals are being trained to prevent such attacks

If SQL injection is getting through your system, you have a process/infrastructure problem. Or maybe a testing problem, but really that’s secondary. I’d go so far as to say if any testing or penetration tools find a SQL injection, see answer number one, you have a process/infrastructure problem. Seriously, if you’re running flow analysis for example and it finds a SQL injection error, don’t just fix that one, address the underlying root problem, IE you’re using unvalidated user input, and fix it everywhere, not just the instances that static flow analysis found.

As long as organizations play hunting games to find SQL injection they will continue to be vulnerable and pay the price. If nothing else learn this, SQL injection as well as other input validation errors are totally preventable, if you’re having problems then you’re doing it wrong.

Catching ALL the errors

Really this concept is just an extension of the previous one. Namely, you need to look at the root cause for SQL injection and put in place a development and testing methodology that requires 100% compliance. In this case as in many other security issues, the root cause is the use of user input that hasn’t been validated. You need to code in such a way that you don’t leave any room for unvalidated input to be used, ever.

Now the trendy thing to do these days is use flow-analysis tools to find “bugs” in software. Disclaimer for those who haven’t figure it out yet, my day job is at Parasoft who is a maker of static and flow analysis tools among other things. The problem is that people frequently misunderstand what flow analysis can do, and then either don’t follow through with what it’s told them, or have a false sense of security that they’ve done all they can.

Both behaviors are incorrect and will lead to vulnerabilities in your software. It’s important to understand that flow analysis isn’t a thorough methodology. It’s somewhat random in that it’s attempting to determine possible paths through your code without human intervention. By it’s nature it will miss things. On the upside it will tell you something you didn’t know about your code without any human effort.

The right response to finding security vulnerabilities in flow analysis isn’t to simply fix the ones found, but rather to ask yourself what the underlying cause is, then create a policy drive programming approach that will negate the possibility of such conditions surviving your development and test efforts.

One more honorable mention in this area is that sometimes people do validation in the client using technologies such as JavaScript. While this is a great way to speed things up and improve the user experience, you need to also validate on the back-end. Remember that the client-side can be subverted, and validation mechanisms bypassed. So use client-side validation as a convenience for the user and an extension of your security, but never in place of back-end validation.

Storing Passwords

STOP storing plain text passwords – immediately! If people forget their passwords, do a reset. They don’t need the old one back, they couldn’t remember it anyway. And make sure you’re salting and putting strong encryption on the passwords. Perhaps the biggest mistake that Yahoo made was
storing usernames and passwords in plain text.


As I hinted earlier, allowing something so preventable and so well understood to happen to you can get you in trouble. Liability and negligence issues often revolve around issues of common practices. If you’re not preventing SQL injection, someone could try and make the case that you were negligent, as is the case for the recent LinkedIn security breach. There is currently a lawsuit against LinkedIn which claims they didn’t meet their privacy policy of “industry-standard protocols and technology” because they didn’t properly deal with password storage, IE salting their passwords. Note that what they were doing is still better than Yahoo with their plain-text passwords.

When doing security, it’s not only safest to deal with the root case, it can save your bacon later on. Always promote best practices and you should be able to limit liability to actual loss at worst.

I’d like to say that hopefully this will be the last time I write about SQL injection, but I’m willing to bet that it’ll happen again to a big company before the end of the year. In fact, I wouldn’t even be surprised if it happened again before the end of this summer. Sometimes we never learn. Feel free to sound off with your observations in the comments section or you can reach my on Twitter, Facebook, Google+, etc.


Top 10 User Mistakes with Static Analysis

mistake © by doobybrain

I recently attended the Static Analysis Tool Exposition (SATE) IV Workshopsponsored by NIST. The goals of SATE are to:

  • Enable empirical research based on large test sets
  • Encourage improvement of tools
  • Speed adoption of tools by objectively demonstrating their use on real software

I find SATE interesting because it takes a couple of different approaches that are pretty useful to people trying to understand what static analysis can and cannot due. One approach is to have several full-fledged applications with known bugs, and versions of the application with those bugs fixed. These have the effect of showing what static analysis tools can do in the real world. Unfortunately, they don’t help much when trying to find out what kinds of issues static analysis can handle overall.

To do that, NIST has developed a test suite that has thousands of test cases with specific issues in them. Part of SATE is running various tools on the test applications and test suites, and then trying to analyze what they can find, how much noise they produce, etc. It’s an interesting exercise. You should check it out.

This year I was privileged give a presentation myself. I wanted to talk about some of the pragmatic aspects of actually trying to use static analysis in the real world. To that end, I created a slide show around the top 10 user mistakes, meaning things that prevent organizations from realizing the value they expected or needed from static analysis. These range from improper configuration to poor policy to dealing with noise.

Take a look for yourself. If you love or hate any of them, let me know. If you have others I missed, feel free to mention it in the comments, or email me or reach me on twitter.

(powerpoint) (pdf)

Download (PPT, 1.52MB)


Software Security Conference on Thursday

I’ll be speaking this Thursday at the SATE IV software security conference in McLean, VA. This is a free event open to the public and a great chance to learn more about static analysis at a day-long event. You can register at http://samate.nist.gov/SATE4Workshop.html

My talk is title “Top 10 User Mistakes with Static Analysis” and I think you’ll enjoy it. If possible I’ll post the slides here after the conference.