mistake © by doobybrain
I recently attended the Static Analysis Tool Exposition (SATE) IV Workshopsponsored by NIST. The goals of SATE are to:
- Enable empirical research based on large test sets
- Encourage improvement of tools
- Speed adoption of tools by objectively demonstrating their use on real software
I find SATE interesting because it takes a couple of different approaches that are pretty useful to people trying to understand what static analysis can and cannot due. One approach is to have several full-fledged applications with known bugs, and versions of the application with those bugs fixed. These have the effect of showing what static analysis tools can do in the real world. Unfortunately, they don’t help much when trying to find out what kinds of issues static analysis can handle overall.
To do that, NIST has developed a test suite that has thousands of test cases with specific issues in them. Part of SATE is running various tools on the test applications and test suites, and then trying to analyze what they can find, how much noise they produce, etc. It’s an interesting exercise. You should check it out.
This year I was privileged give a presentation myself. I wanted to talk about some of the pragmatic aspects of actually trying to use static analysis in the real world. To that end, I created a slide show around the top 10 user mistakes, meaning things that prevent organizations from realizing the value they expected or needed from static analysis. These range from improper configuration to poor policy to dealing with noise.
Take a look for yourself. If you love or hate any of them, let me know. If you have others I missed, feel free to mention it in the comments, or email me or reach me on twitter.
(powerpoint) (pdf)
Resources
