Domain Specific Languages - Past, Future and Present

One of my career-long passions has been programming language development. I've written compilers and interpreters for Pascal, BASIC, C, and quite a number of what are now called "Domain Specific" languages. 

Some of them (like Pascal/MT+ and Digital Research C) were targeted (and reached) a wide audience of general-purpose developers. Many, however, were designed, by me, for me, to solve often a single problem and resulted in a significant increase in productivity (10x++) which I as sure is the result of being both the tool maker and tool consumer. They had great names like SPL/11, SPL/605, SPL/FLASH, (do you see a pattern? :) ).

I believe our industry must now evolve so that more of us can have that experience but in a shared, extensible and non-idiosyncratic fashion.

The Past:

I first started writing compilers in PDP-8 assembly language. My first compiler was an implementation of the Olivetti Programma 101 programming model (The P-101 was a late 1960's programmable calculator on which I clearly remember writing my very first program. I think it solved quadratic equations.). I am still amazed that I ever got the compiler finished as I had to translate the PDP-8 assembly language into octal by-hand and entered it numerically via a debugger as that was faster than using the assembler (using the assembler required punching the program onto paper tape and running it through the ASR-33 teletype 3 times to get a binary output... which then had to be fed back into the teletype in order to run the program!)

A couple of years later I moved on to using a technology called METAPI (written by John T. O'Neil) which allowed one to define a language using a modified-BNF description along with semantic generation statements. I used the IBM mainframe version to write 4 or 5 languages that targeted the PDP-11 for various research projects we were doing at Iowa State and, while at NCR, I re-implemented METAPI in Fortran (the only non-assembly language available on an NCR mainframe) so that I could do a few more languages for development of communication concentrator mini-computer systems. (This is all pre-Pascal/MT which was originally written in 8080 assembly language).

The Future:

As mentioned I noticed was that when I was developing a language for my own use (and for a very specific problem domain) I was able to multiply my development efforts by large amounts. I eventually realized that this was because when I had a very clear idea of my outcome I was motivated to frequently evolve the language to encompass a broader (and higher level of) abstraction whenever possible rather than writing more-and-more "application code". The tools available in those days (along with my own inexperience, I'm sure), made it substantially more difficult to incorporate language enhancements (and iterations) to extend the multiplying effect across a larger group without creating multiple incompatible versions.

I'm now convinced that as we try to raise the level of abstraction we must make it possible for the next generation of languages, both graphic and textual, to evolve and adapt as developers use them. If we are to make things more powerful we must move the language constructs fluidly back-and-forth iteratively honing in on the problem domain. We must make enable "mass customization" of these languages while at the same time retaining the ability for developers to move up-and-down the abstraction levels without having to stumble over one another's idiosyncratic implementations.

The Present:

I know we must move "up-the-stack" if we are to continue to propser as an industry. There are still a few very talented people who write assembly language today (bless 'em!) while most of us write in some 3rd generation (pick your favorite C-like, BASIC-like, script-like) language (3GL). 

I believe, in the future, in our continuing attempts to build more secure, more maintainable, more agile applications, the portion of the development community writing in 3GLs will be significantly reduced and the portion writing in DSLs will dramatically increase. I don't know what route we will take to get there but I firmly believe (personal opinion) that's where we're going.

Do you agree? I'd love to hear from others who have done and/or used DSLs and are willing to share their experiences and aspirations.