I gave a presentation and demo of NStatic to the .NET Developer Associations user group yesterday. There were at least two readers of my blog that showed up including Serge Baranovsky of VBCity. Serge actually sells a static analysis tool, so I was demonstrating to the competition. I received a good response and obtained much interesting feedback. People from the audience called it "intriguing."
Slide #1 - Introduction
I introduced myself as a former developer of a big software company, who set out on his own to develop his own AI software company, and that this was my first public demonstration of my software product.
Slide #2 - What is Static Analysis?
- Static analysis locates errors in source code without actually executing code.
- Why use static analysis?
- Bugs found early when cost to fix is low.
- No performance overhead.
- Covers all code uniformly, including those on rarely executed code paths.
Slide #3 - Existing Tools
I went through existing tools in the C++, Java, and .NET space. There are few tools available in the .NET space. Microsoft Team Suite has an enhanced version of FXCop (probably just called Code Analysis).
There are some tools (mostly for C++) listed in these links.
One site had a price list revealing that the advanced commercial static analysis tools had a per user cost of >$2000 up to as much as $10K. The prices have since disappeared the web page.
The Java.NET has some benchmark comparisons on two of the leading static analyzers both for C++.
I have been looking at some of these sites to get a feel for the typical find rates, false positive rates and analysis times. My goal is to beat them all.
Slide #4 - NStatic
This was an entire slide in which I included some bullet points in the blank document area.
NStatic: Original new approach
- Debugger-like experience
- Deep symbolic analysis
- Efficient, deterministic analysis
I included a disclaimer that this is a work in progress, and I don't know if any of these assertions are really true, since my product isn't out yet and I have not actually use most of the tools out in the marketplace.
I mentioned that there were two major changes that I made last November and December. One was IL Interpretation and the other was a change to my interprocedural analysis due to an Microsoft/Intrinsa patent on interprocedural analysis. The Microsoft patents show how one could avoid doing a full interprocedural analysis by storing function summaries in order to make analysis proceed quickly. I heard that Prefix can still take a day to run on large codebases; that's why they built a scaled down intraprocedural version called Prefast. Instead of taking shortcuts by summarizing each method, I just tried to figure out how to do full interprocedural analysis quickly; my new approach might even be faster than the earlier one.
I proceeded to go through different parts of the UI with examples.
Slide #5 - NStatic
I remarked about the similarity to Visual Studio with the various panes-- Solution Explorer, Error List, Call Stack, Locals window.
A few panes are missing in the first release, the Watch window and the Immediate window, which would allow the user to observe arbitrary values and execute arbitrary code symbolically. The Immediate Window, in particular, would be similar to the input prompt in Prolog that would take any query and provide an answer. The source code would in essence be a knowledgebase of facts, and you could ask arbitrary questions.
I have breakpoints, but don't currently allow stepping. The breakpoints are not integrated into the UI right now, but instead require a call to any Break method.
Slide #6 - UI: Scan Control
I mentioned that the previous screenshot was actually a scan across the Rotor codebase, but I was using a short timeout in which about 997 of 37K methods timed out. Some other options as well as IL Interpretation was also turned off, which is why the analysis time was just over 2 minutes.
Slide #7 - UI: Locals & Conditions
Slide #8 & 9 - UI: Error Highlighting
I used the same images as in my Effortless UI, II post, but I did include a couple of others.
There are still glitches in my highlighting. I haven't really spent much time thinking about this.
I didn't make it clear to the audience apparently that the arrows were produced by my software. It was until several slides later that an audience member asked, "Are those arrows produced by the software, since they look like drawings?" I said, "Yes! That's the point."
Another audience member noted that I could actually warn against the parameter, n, not being used in the TryCatch function. I told him that I do in fact check for unnecessary parameters. In fact, I check for a more general case, which is whether any parameter is unnecessary--ie, if its value must be a function of any of the other parameters or global state. This is an example of using general principles over specific rules. I talked about this later in my presentation, but here is an example.
I pulled out NStatic and showed two simple functions to make my point clear.
In the three parameter function above, the parameter c is unnecessary, since it is function of the other parameters. In other words, c is expressible in terms of a and b, as shown in the locals window. Similarly, in the first function, x is redundant, because it can only be the value 3.
This was another screenshot in which I referred to a bug in the codebase. This also shows off other views in the product including the ClassView.
In the second half of the slides, I went more into the technical description of how my product works.