« Missing the Inflection Point | Main | No Language Lower than C++ »

June 23, 2004

Whidbey May Miss the Next Coding Revolution

The Visual Studio.NET source code editing capabilities have consistently lagged the competition, mainly in the Java space. While Whidbey introduces some much needed enhancements to the C# (and VB) IDE such as refactoring and dynamic code formatting to catch, I predict that Whidbey will find itself once again behind the ball.

The truth is much of the new changes in Whidbey will be wasted effort, because I see a revolution brewing within the next three years in the way source code is written.

Text editors are going to go away (for source code, that is)! Don't get me wrong, source code will still be in text files. However, future code editors will parse the code directly from the text file and will be display in a concise, graphical and nicely presented view with each element in the view representing a parse tree node. (NOTE: I now the word graphical is misleading, so I explain it more in this newer post.)

Religious beliefs about key bindings will disappear, because most of those key combinations will become completely irrelevant in the new world. Yes, Don Box will finally kiss his Emacs goodbye. The new paradigm actually eliminates syntax errors (at least, they will be flagged immediately by the new graphical editor) and many semantic errors, because each edit operation directly alters, adds or deletes a node in the in-memory parse tree.

It also has good potential to eliminate hundreds of lines of coding by collapsing. Such an editor could introduce new keywords or nodes, that could map to hundreds of lines of textual code at save time, without requiring an improvements to the compiler or language. For example, the editor could be enhanced to support delegation, whereby a class inherits all the public methods and properties from a member variable. It could support default parameters in C#, which maps to multiple methods, when written out to a text file; the editor could recognize this pattern when reading back from a file. Backing store properties can be entered and displayed concisely. A lot of crud can simply disappear, with the editor automatically importing the appropriate namespaces when a class is reference or omitting the class name from an enum constant.

There are countless other benefits such as the ability to present multiple radically different views of code, all simultaneously synchronized; even within a view, code can be pivot, sorted, and filtered in different ways.

From a development standpoint, it is far easier to write an intelligent editor from a parse tree then from a sequences of lines of text, which is why it will happen. Hence, the effort put into dynamic code formatting, which may be thrown away, would probably have been better placed in edit-and-continue support. On the other hand, edit and continue may actually be easier to implement in the new model, because the parsing step is eliminated when the user edits code. In addition, implementing refactoring would have been significantly easier because code is stored in memory using a natural object tree representation rather than as streams of text. Changing the name of a function directly could instantly change all references elsewhere without the need to go through a dialog, because each reference in the view is not stored as text but as a direct link to the function object. The effort that Microsoft put into robust real-time parsing for Intellisense is completely wasted.

Part of my current entrepreneurial work is rethinking today's existing application and revitalizing them with artificial intelligence. One of these applications is text editing. I have been thinking about this upcoming revolution for some time, but just recently I have noticed that many other developers were coming to same conclusion, which usually means it is going to happen soon. Just in the past week, Lutz Roeder mentioned separation of code and text in his Reflector presentation; the presentation was actually dated in 2001. There was also many good examples of the potential of SCID (Source Code in Database), which refers to this shift in programming, in http://mindprod.com/projects/scid.html via secretgeek.com.

I think a major cause of the delay in this revolution is that both C/C++ relied on preprocessors and headers. Some historical languages like Smalltalk actually had this support. Fortunately, more modern languages like C#, Java and VB are standalone files, one class per file, with a little or no preprocessing support. This enables easy parsing.


The Reflector presentation (which is not about Reflector) you are mentioning is online for almost 3 years - you just discovered it last week. I had a conversation with Lutz once a few years ago and the guy is amazing. He can draw roadmaps 10 years out in the future and next tells you in all detail why these roadmaps will fail.

You might not have noticed it but your post is somewhat funny. You claim that Microsoft might miss the future and at the same time you point to a 3 year old presentation done by some Lutz who is now working at Microsoft :-)

I've been hearing this kind of statements forever.
IBM and others, try to make this happen, but the real power and control is allways in the code.

IMHO, we will keep adding more power per line of code in the future, just the same way is happening today. You write less code, that performs powerfull tasks (think XALM, Indigo)

I hear about this alot and I have to say there are some problems here.

1) Instead of language lock in, you could easily end up with ide lockin...which is far worse. Each IDE defines a feature in a different way, perhaps in a way that doesn't map directly to others. IDE's end up half their own code and half code dealing with other IDE's parse trees. Users have to learn that <= means delegate in VS.NET, but its (-<< in #Develop. This "see it the way you want to" is mostly BS. Its "see it the way the IDE wants you to" and thats all it is.

2) It locks you into one parse model...one logic model...is it even possible to express every language the same way? Or are we talking an IDE that does orders of magnitudes more processing to achieve langauge parity?

3) it forces the editors to work to define new language extensions for every language it want's to define at times. Imagine pointers in VB or trying to express the VB My statements in C#. This isn't possible with existing syntax without great complexity(and having the IDE totally lie to the user about whats going on). And lets not forget about roundtripping issues.

Those are all big issues, IMHO. It may be doable with a system that is pretty constrainted(java), but for a system that was designed to allow multiple languages and multiple ways of expressing the same thing, I fear such an approach would cost in semantic awareness of your program and significant loss of control over it(I'm thinking somewhere along the lines of going from assembly to C# without development speed benifits).

It has its benefits...and there may be a point where IDE's operate like that, however I don't believe it is the panacea that people have been making it out to be. Its really just XML-Fever, I think. Developers try XML and suddenly want everything in untyped trees so they can play with it, even if its a really bad idea.

Ok, after rereading your posts in the context of your last comment, I think I have a better understanding of what you are suggesting.

Although you haven't explicitly stated this, it seems you are envisioning basically a Model-View-Controller type approach to the code editor wherein the code (in the form of a parse tree) is the model, and the editor can show any number of different views of this model and can manipulate the model separately from the view.

I can certainly see the benefit if that's what you are going for. Am I understanding you better?

Doesn't Microsoft have rights to the intellectual property that Charles Simonyi and gang are developing for Intentional Programming?

Sean beat me to the punch. This sounds just like Intentional Programming, which used to be a research project here at Microsoft, run by Charles Simonyi. It seemed like a cool concept, but year after year, nothing seemed to come of it. Finally, Charles left to form his own company, presumably to continue the quest. Personally I'm skeptical - ever since the non-fruition of wearable displays, I've been wary of technologies that are always coming but never arriving.

Note that I have no knowledge of the current status or progress of Charles Simonyi's company or product.

Bruce, I'm still a believer in Intentional Programming. However, I also understand that we're in the _business_ of software development, whereas IP is part of the _science_ of software development.

I think Wesner has some great points here, but to conclude that this "new" way of programming will be "hot" before Whidbey gets here is a bit premature.

In fact, I believe it will be Whidbey that gets the ball rolling towards what Wesner envisions here.

I mostly agree with what's being presented here, but I must add that I diagree with your chosen implementation of the IDE visualizations.

Therein lies the core problem, as Daniel suggested. Honestly, I think we're much faster programmers with typing source code than we would be dragging that darned mouse around the desk for millions more miles per project.

Good post. Keep exploring and telling us what you find!

I don't think the "IP" on IP is worth very much. The patents are not very useful beyond extensible compilers and from what I hear none of the IP researchers continued working on the project except for Simonyi.

I can't imagine any place where a revolution like this would start other than Microsoft. No one else has any reason to spend billions of dollars and zillions of developers to make a shift like that happen. Notice today there is no business model for selling development tools anymore. Borland is just learning this the hard way.

Chris, there is a difference between business value and absolute value. IP is an awesome idea. However, not all awesome ideas make it in this capitalist world. I defend Simonyi's vision. I do, however, realize that the world isn't ready for it. Sadly, we may see the end of him before we see the beginning of the fruits of his labor. IP, or some newly renamed rendition of it, will see the light of day.

Michael, I agree. IP is an awesome idea and I would love to see stuff like that becoming real.

Actually IP is more a bag of ideas that a bunch of different people came up with during the 10 year lifetime of the IP project. Most of them have little to do with Simonyi's ecology of abstraction pitch and just ended up under the same umbrella. From what I heard there was lots of disagreement between Simonyi and other people working on the project and most of the people left over time in disagreement with Simonyi's leadership.

I also concur there is business value and idealist value which sums up to absolute value. The business value of IP is simply not there. There is not a single data point that IP ideas would be an improvement for any real scenario. Reflector might be an exception but the stuff Lutz Roeder is doing looks a whole lot more professional than anything the Simonyi crew ever showed at conferences. Reflector could have been by someone that never heard about IP before.

Chris, how about the business value of IP to the Mort's of the world? It is a type of usability that is beyond API usability.

I've been programming since 1970. And since 1970, I have heard every single year that "programming will go away", "programming will become simplified", "programming will be vastly different". Well, programming is not going away, and programming is not simpler. In fact, the trend is the other way.

It is so much more difficult to create SOLID and MAINTAINABLE systems in languages such as Java and C++ than ever before. What I see today is a mess of classes and derived classes; a mess of inheritance and polymorphism; a potpourri of private, public, friends access modifiers. Once the person who wrote this stuff is gone from your company, the usual company solution is to re-write the whole mess with the "new and improved" language of the year.
No, programming is not getting simpler, it has gotten more and more complex from 1970 to 2004. My job is threatened by outsourcing, not by simplifying programming tools.

Ares to the Ninth (to hell with war) Can you figure out why? Hint: Ares, Mars is war.


How to improve our coding style which requird to improvment?
where should we get this info.
please send path or artical on this or related.

The comments to this entry are closed.