Thursday, June 12, 2008

Free and Flexible Development Tools

Often, when I'm talking about the shabbiness of Microsoft software I hear something to the effect that the development tools are great, especially Visual Studio. Really? Is that true or does it just appear that way since the environment is closed?

I used to agree with this assertion, but what did I know? Like many, if not most, Microsoft developers, I hadn't worked with any other development tools. As a matter of fact, there was a time when I couldn't even imagine any room for improvement. Something that I've come to notice is there is a Microsoft induced reality distortion field that is far stronger than that of any other vendor or government and it makes you lose your imagination.

Visual Studio actually does very little for you that you can't find elsewhere, and maybe even less.

One of the worst things about Visual Studio is it's opacity. It's very difficult to know what's really going on under the hood. References and components are visible only through tiny dialog boxes which in turn rely upon the registry to actually find and use the components and libraries that appear there. It's possible to see the references if you view the relevant project files as text, but the references contain elaborate numbers that map to the registry. This is a trip down the rabbit hole if ever there was one.

When using Visual Studio you have no choice about the compiler you can use or the debugger. If you want to use a real version control system you have to get third party software working on all the seats.

Intellisense is cool, but it exists elsewhere. Debugging, immediate windows and break points are standard fare in other integrated development environments.

But why belabor the point on a feature by feature comparison?

Let's take a look at a simple and useful task and see if it can be done in Visual Studio. Imagine you've got a directory filled with VB files (*.cls, *.bas, *.frm, etc.) and you want to find all the instances were a particular procedure is called, let's call it 'frmOSIssue.LoadData. How would you produce a list of all the places where this appears in the project?

The answer is you can't. Visual Studio has no facility to present you with this kind of time-saving information and Windows carries with it no tools that can accomplish this task either. In Unix/Linux/OS X, it would be accomplished with a single, simple command using software that is installed by default in every system.

grep -i frmOSIssue.loaddata

This might seem arcane, but it's no more arcane than any other weird stuff software developers have to know to do their jobs. This command calls grep, which searches text files. The -i argument tells grep to be case insensitive and the last argument is the string you want to find. Once you know what grep does and are familiar with a few of the options you're set to go. You can certainly benefit from knowing regular expressions, but you don't need to know very much to start using the command to good effect. Here's what the result looks like after it searched over 600 files in less than one second:

OSIssue.frm:    strProcedureName = "frmOSIssue.LoadData"
TLIssue.frm:    strProcedureName = "frmOSIssue.LoadData"
WizOS_P01.frm:    strProcedureName = "frmOSIssue.LoadData"
WizOS_P07.frm:    strProcedureName = "frmOSIssue.LoadData"

Similarly, I can do this in a file filled with SQL scripts to show dependencies. Basically the same command does a lot of heavy lifting that SQL Server Enterprise Manager and Visual Studio can't do.

But, but, but.... what about Visio? How can you diagram this? Many believe there is no replacement for Visio, especially since Visio's file format is so strictly proprietary that no other software can produce or read VSD files, as far as I know. First, there are other tools that provide UML and ERD. These are not as hard to come by as some would have you believe. Second, these alternatives often produce better output. Third, they're free and use simple text files, so you'll always have them and you can do other things with them too.

One diagramming package that I love, Graphviz, draws diagrams from a text file. For some really impressive examples of what this can do, check out http://graphviz.org or Google for grpahiz examples to see the possibilities. These files are simple text, so they are easy to make, easy to share, small and they will work with any software package that can read this standard format, .dot. Because they are text, it is also simple to use diff to compare one version of your diagram to another. Try doing that with Visio. Also, these packages can produce a number of graphics formats, most usefully PDF. Producing PDF is typically a built-in, globally available capability in operating systems other than Windows.

This is a little Perl script that reads the files in a directory and finds all the interrelations among them and writes a .dot file that can be processed into a graph.

#!/usr/bin/perl -w
use File::Basename;

# -- Take the list of file names from the directory
# -- Loop through the list of function names and search all files in the directory for it
# -- Put the result (filename:functionname) into an array

opendir(DIR, ".");
@files = grep(/\.sql$/,readdir(DIR));
closedir(DIR);

foreach $file (@files)
{
        $funcname = basename($file, ".sql");
        @relatedfunctions = qx{grep -l -w -m 1 $funcname *.sql};

        foreach (@relatedfunctions)
        {
        # The mapping shows that the first function listed is called by the second function listed
                $_ = basename($_, ".sql");
                if ($_ ne "$funcname\n")
                {
                chomp($_);
                print "$funcname" . " -> "  . "$_" . ";\n";
                }
        }
}

It then writes a simple file, a piece of which is here:

digraph FunctionMap {

subgraph cluster_Functions {
label=Functions;
}
AGGR_PPU_PooledAmount -> AGGR_PPU_AssignedStatus;
AGGR_PPU_PooledAmount -> AGGR_PPU_Name;
AGGR_PPU_PooledAmount -> Bond_OutStanding_AGGR_PPU;
}

After putting a little business at the top of the file it can be opened by a piece of software that reads .dot files and creates high quality diagrams. I typically use Graphviz for this, but there are other software packages that do this too.

Here is the nice, clean data-driven diagram. 


One thing I like about this is that there is no need to rearrange the boxes, it's all done for me. My staff sometimes spent the better part of a day trying to arrange the elements of a large schema by hand so that it was intelligable - no more. There are variety of styles to choose from and I'm not finished yet plumbing the depths of this format.

This is actually super efficient, light weight, platform independent and produces really great output. Is it Visio? No, it's not. But I don't miss Visio either; it's too cumbersome and expensive. The benefits of these tools outweigh the capabilities of Microsoft's IDEs. I still have to use Microsoft's IDEs sometimes when supporting some old applications because the compilers aren't readily available otherwise, but I chafe under their limitations now and use alternatives when ever possible. Also, when I need a capability I'm free to put one together myself or find it on the Internet. So far I've written scripts to convert TSQL into PL/pgsql, convert .dot diagrams into code stubs and to globally modify comments and to outline training documents.

Another example of how some basic tools can beat Visual Studio comes when you have to compare entire directories. If your directory consists of all the project files from Visual Studio or a bunch of SQL scripts and you want to compare it to another directory Visual Studio isn't going to be much help, but diff will be. The followig commands will tell you what's different between two directories of files:

diff -rq dirA dirB > dir_diffs.txt

or from within one of the directories you want to compare you can use 'find'.

find . -type f -not -exec cmp {} /path/to/dir2/{} ";" -print

Software developers should start programming new languages in their rawest form so they know what's happening, but they should move to an IDE soon after that so they can be more productive. The question is, what is the IDE? Is it something Microsoft provides for you - take it or leave it - or is it something you put together yourself based on your needs, tastes and budget? I prefer the later. Having control of my environment makes me free and independent. When I'm free and independent, I'm most productive.

No comments: