The Year We've Been Waiting For Has Arrived On Cats' Paws

For years we've heard that this year would be the year of Linux on the desktop, or this will the be year when Open Source really takes off. But each year we find ourselves sitting in front ofcki soul-sucking Windows machines at work, helping our friends and family with their Windows viruses, and listening to more FUD and vapor hype from Redmond than we cant stand. Waiting for Linux and Open Source to take over the world has left us so fatigued that we may have missed its actual arrival.

Maybe that's because we've been looking for the wrong thing. Instead of concentrating specifically on Linux, we should be looking for the family of operating systems that Linux is a part of.

That immediate family of software are the Open Source Unix/Unix work-alikes, including Linux, taken together. I think this is meaningful because these system share a certain philosophy, a culture and a very similar toolset. Tribalism aside, if I had to choose between working on a Windows box all day long or working on a flavor of Unix/Linux, I would choose the Unix/Linux option every time - even if I couldn't have my first choices, OSX and FreeBSD. My guess is, so would many of you. The extended family of operating systems would include the non-free unix operating systems as extended family.

Instead of measuring the popularity by counting desktop or server installations, why not use another yardstick? Most people who use a computer don't have a choice in operating systems, someone at a factory or in a corporate IT department made a choice for them, so installations don't mean much as the decisions are made by a few for the many. What then can we use as a guage?

In our capitalist system we use money as a measurement of what's important to us, rightly or wrongly. We think nothing of saying that people value their cars because collectively we spend an enormous amount of money on them. We will work ourselves ragged chasing the almighty dollar and the most shallow among us go so far as to make moral judgements of our fellow man based upon wealth. And although most computer users don't have to underwrite their critical operating system choices with money, because most of them aren't making critical operating system choices in the first place, what does the world look like when we look at where those who choose operating systems put their money?

What do our outlays on software tell us about what is truly most popular, most productive and most successful in the software field? The way we spend our software money tells us that the Open Source Unix and Unix-workalike systems are actually more popular than Microsoft Windows. The victory of Unix/Unix workalike systems has arrived, on cat's paws.

But don't take my word for it, look at the market capitlizations for the following firms as of April 30, 2010 (in billions):

Apple: 237.58

Red Hat: 5.61

IBM: 165.12

Novell: 1.96

Sun: 5.60

Why did I pick these companies? Because they are all vendors of Unix or Unix-like systems, including Linux. Each of them has a significant reliance upon and/or makes significant contributions to Open Source as well.

Apple's OSX is a certified Unix, as is Novell's software and IBM's AIX, to name just a few. Apple's OSX is derived substantially from Freebsd and Apple contributed Darwin to the open source community. Apple is also the world's largest seller of Unix units, I believe. While IBM's AIX is propietary, it is a Unix and IBM has a major business in Linux too. Novell has been a long time Unix vendor and for several years has been selling Suse Linux. Red Hat is a pure Open Source Linux company. Sun, while no longer independent, had a certified Unix, Solaris, which shares ancestry with the BSDs and they contributed quite a bit to Open Source projects.

So how do the market capitalization numbers for these firms support my suspicion that the year of Linux/Unix/Unix work-alikes has arrived? Well, if I take the market capitalizations of just these firms, and discount them to account for the fact that not all of their value comes from Linx/Unix and Unix work-alikes, I get a set of numbers that look something like this (in billions):

Apple: 237.58

Red Hat: 5.61

IBM: 50.00

Novell: 1.00

Sun: 3.00

Again, this is just me scratching some numbers on a napkin to reflect what I think their Open Source business, their Linux/Unix and Unix work-alike businesses are worth. I didn't discount Apple because OSX was instrumental in the company's revival and it's the operating system in their latest iPods, iPhones and iPad. For Apple, there is no meaningful separation of Unix, it's advantages and philosophy from the success, versatility and reliability of their products. Red Hat also gets a pass because its whole business is Open Source and Linux. That much of its revenue comes from support is not a stike against it as support fees is a central business proposition of Open Source software. Since IBM's business is heavily reliant on hardware and support, I discounted it rather steeply, maybe too steeply, but rather more aggressive in this case. In the case of Sun, I took only 3 billion of the 5.6 billion Oracle paid for them. These were the easiest firms to get my head around, but I didn't include HP or other specialized vendors. Together I think it's reasonable to say that the Open Source/Linux/Unix aspects of these companies are worth over 297 billion dollars.

By comparison, Microsoft's entire market capitalization is: 267.60 B

And that's for everything at Microsoft - Windows, Office, SQL Server, SharePoint, XBox, Zune, patents, real estate, the whole thing, no discounting there to simply compare their Windows value so my figures match operating system to operatnig system. This isn't an apples to oranges comparison, this is apples to an orange grove comparison. And the apples win. Sorry, couldn't resist.

It's clear to me that even when generously estimated, I have to conclude that the world has already voted with its dollars that the Open Source Unix/Linux operating system is the better system. When people who actually make choices about operating systems, and whether or not to spend a dollar on Open Source/Unix/Linux or Windows, it chooses Open Source/Unix/Linux. I haven't even taken into consideration the value of all the free installations of Open Source/Unix/Linux.

So when do we start saying that Microsoft has lost the operating system the battle and Open Source/Unix/Linux has won and 2010 is the year of Open Source/Unix/Linux?

Free and Flexible Development Tools

Often, when I'm talking about the shabbiness of Microsoft software I hear something to the effect that the development tools are great, especially Visual Studio. Really? Is that true or does it just appear that way since the environment is closed?

I used to agree with this assertion, but what did I know? Like many, if not most, Microsoft developers, I hadn't worked with any other development tools. As a matter of fact, there was a time when I couldn't even imagine any room for improvement. Something that I've come to notice is there is a Microsoft induced reality distortion field that is far stronger than that of any other vendor or government and it makes you lose your imagination.

Visual Studio actually does very little for you that you can't find elsewhere, and maybe even less.

One of the worst things about Visual Studio is it's opacity. It's very difficult to know what's really going on under the hood. References and components are visible only through tiny dialog boxes which in turn rely upon the registry to actually find and use the components and libraries that appear there. It's possible to see the references if you view the relevant project files as text, but the references contain elaborate numbers that map to the registry. This is a trip down the rabbit hole if ever there was one.

When using Visual Studio you have no choice about the compiler you can use or the debugger. If you want to use a real version control system you have to get third party software working on all the seats.

Intellisense is cool, but it exists elsewhere. Debugging, immediate windows and break points are standard fare in other integrated development environments.

But why belabor the point on a feature by feature comparison?

Let's take a look at a simple and useful task and see if it can be done in Visual Studio. Imagine you've got a directory filled with VB files (*.cls, *.bas, *.frm, etc.) and you want to find all the instances were a particular procedure is called, let's call it 'frmOSIssue.LoadData. How would you produce a list of all the places where this appears in the project?

The answer is you can't. Visual Studio has no facility to present you with this kind of time-saving information and Windows carries with it no tools that can accomplish this task either. In Unix/Linux/OS X, it would be accomplished with a single, simple command using software that is installed by default in every system.

grep -i frmOSIssue.loaddata

This might seem arcane, but it's no more arcane than any other weird stuff software developers have to know to do their jobs. This command calls grep, which searches text files. The -i argument tells grep to be case insensitive and the last argument is the string you want to find. Once you know what grep does and are familiar with a few of the options you're set to go. You can certainly benefit from knowing regular expressions, but you don't need to know very much to start using the command to good effect. Here's what the result looks like after it searched over 600 files in less than one second:

OSIssue.frm:    strProcedureName = "frmOSIssue.LoadData"

TLIssue.frm:    strProcedureName = "frmOSIssue.LoadData"

WizOS_P01.frm:    strProcedureName = "frmOSIssue.LoadData"

WizOS_P07.frm:    strProcedureName = "frmOSIssue.LoadData"

Similarly, I can do this in a file filled with SQL scripts to show dependencies. Basically the same command does a lot of heavy lifting that SQL Server Enterprise Manager and Visual Studio can't do.

But, but, but.... what about Visio? How can you diagram this? Many believe there is no replacement for Visio, especially since Visio's file format is so strictly proprietary that no other software can produce or read VSD files, as far as I know. First, there are other tools that provide UML and ERD. These are not as hard to come by as some would have you believe. Second, these alternatives often produce better output. Third, they're free and use simple text files, so you'll always have them and you can do other things with them too.

One diagramming package that I love, Graphviz, draws diagrams from a text file. For some really impressive examples of what this can do, check out http://graphviz.org or Google for grpahiz examples to see the possibilities. These files are simple text, so they are easy to make, easy to share, small and they will work with any software package that can read this standard format, .dot. Because they are text, it is also simple to use diff to compare one version of your diagram to another. Try doing that with Visio. Also, these packages can produce a number of graphics formats, most usefully PDF. Producing PDF is typically a built-in, globally available capability in operating systems other than Windows.

This is a little Perl script that reads the files in a directory and finds all the interrelations among them and writes a .dot file that can be processed into a graph.

#!/usr/bin/perl -w

use File::Basename;

# -- Take the list of file names from the directory

# -- Loop through the list of function names and search all files in the directory for it

# -- Put the result (filename:functionname) into an array

opendir(DIR, ".");

@files = grep(/\.sql$/,readdir(DIR));

closedir(DIR);

foreach $file (@files)

{

        $funcname = basename($file, ".sql");

        @relatedfunctions = qx{grep -l -w -m 1 $funcname *.sql};

        foreach (@relatedfunctions)

        {

        # The mapping shows that the first function listed is called by the second function listed

                $_ = basename($_, ".sql");

                if ($_ ne "$funcname\n")

                {

                chomp($_);

                print "$funcname" . " -> "  . "$_" . ";\n";

                }

        }

}

It then writes a simple file, a piece of which is here:

digraph FunctionMap {

subgraph cluster_Functions {

label=Functions;

}

AGGR_PPU_PooledAmount -> AGGR_PPU_AssignedStatus;

AGGR_PPU_PooledAmount -> AGGR_PPU_Name;

AGGR_PPU_PooledAmount -> Bond_OutStanding_AGGR_PPU;

}

After putting a little business at the top of the file it can be opened by a piece of software that reads .dot files and creates high quality diagrams. I typically use Graphviz for this, but there are other software packages that do this too.

Here is the nice, clean data-driven diagram. 

One thing I like about this is that there is no need to rearrange the boxes, it's all done for me. My staff sometimes spent the better part of a day trying to arrange the elements of a large schema by hand so that it was intelligable - no more. There are variety of styles to choose from and I'm not finished yet plumbing the depths of this format.

This is actually super efficient, light weight, platform independent and produces really great output. Is it Visio? No, it's not. But I don't miss Visio either; it's too cumbersome and expensive. The benefits of these tools outweigh the capabilities of Microsoft's IDEs. I still have to use Microsoft's IDEs sometimes when supporting some old applications because the compilers aren't readily available otherwise, but I chafe under their limitations now and use alternatives when ever possible. Also, when I need a capability I'm free to put one together myself or find it on the Internet. So far I've written scripts to convert TSQL into PL/pgsql, convert .dot diagrams into code stubs and to globally modify comments and to outline training documents.

Another example of how some basic tools can beat Visual Studio comes when you have to compare entire directories. If your directory consists of all the project files from Visual Studio or a bunch of SQL scripts and you want to compare it to another directory Visual Studio isn't going to be much help, but diff will be. The followig commands will tell you what's different between two directories of files:

diff -rq dirA dirB > dir_diffs.txt

or from within one of the directories you want to compare you can use 'find'.

find . -type f -not -exec cmp {} /path/to/dir2/{} ";" -print

Software developers should start programming new languages in their rawest form so they know what's happening, but they should move to an IDE soon after that so they can be more productive. The question is, what is the IDE? Is it something Microsoft provides for you - take it or leave it - or is it something you put together yourself based on your needs, tastes and budget? I prefer the later. Having control of my environment makes me free and independent. When I'm free and independent, I'm most productive.

What Happens in Vegas...

Some of my past speaking engagements came up in conversation recently and out of curiosity I paid a visit to the web site of the publication that held the conferences, Advisor Media. I gathered from the site that they have largely switched their focus away from technology and no longer hold the kind of conferences I used to speak at and attend.

While I was reviewing the site a blog by the publisher, John L. Hawkins, caught my eye. It was titled, "How I Escaped From Windows Vista" and it expressed his frustration and disappointment with Vista and recounted his fall back to XP. It was a good read. The few comments expressed quite similar complaints, so I just had to chime in myself.

My comment can be found at the blog, but I thought I would fill in a couple of gaps. The Advisor Media's tech conferences I was involved with were largely Microsoft affairs, though once I spoke at an interoperation conference in Maui's Ritz Carleton (very, very nice). While I certainly did a lot of Microsoft stuff at the conferences, I also specialized in Oracle and did a day long project management post conference session.

As I mention in my comment to John's post, I last spoke for Advisor Media in Vegas in '05. I had a talk on Oracle, gSOAP and I did my post conference project management session. I showed up with my PowerBook and did my thing. When I needed Windows, I ran it under Virtual PC. I was a misfit for sure and some of my old conference acquaintances did not know what to make of me. How had I gone so wrong? I had some explaining to do, but I think I was seen more as a misguided curiosity than anything else. Besides, the talks were good and showing gSOAP's performance advantage over .NET SOAP services was a hit.

Trouble loomed at the end of the conference when the speakers and representatives from Microsoft formed an expert panel to take questions from the attendees. There were a lot of questions about Share Point, which was pretty new then, and Longhorn, which was delayed again. I really couldn't speak to either of these things. I had never used Share Point, still haven't and I think it would have been pointless, and bad form, to bring up Unix/Linux/OS X in response to a Windows operating system question. After all, these folks are probably pretty much stuck with the OS whether they like it or not. So I had very little to say.

Then an attendee asked the Microsoft representatives about the state of Access' Jet engine which hadn't been updated in years. Now, regardless of what you think about Access, or what I used to think about it, Access was very important to a lot of people. Rightly or wrongly, it put food on the table for a lot of software folks, myself included, and they were genuinely concerned by Microsoft's neglect of this product.

I don't recall exactly what the Microsoft reps said about updates, but I remember that the answer ticked me off because it wasn't really an answer and I felt I had to address this attendee's question about getting his hands on an adequate database. Panelists share a microphone and pass it around, so I called down for it. I got a couple of looks from the other panelists who thought the crazy uncle was about to recite his medical history over Thanksgiving dinner, maybe it was my imagination.

I said quite simply that there was no reason to wait for Microsoft to take care of a product that is obviously so important to their work, but was essentially a commodity, there were options. I mentioned Postgresql and MySQL and how they were more than adequate and were always being updated. Anyone who had worked with databases would have no problem working with these and they would be glad they did. I think I briefly related some of my own experiences. Essentially, I said that Microsoft had turned its back on developers and they should return the favor if they weren't happy.

The other panelists were not happy about what I had to say and they were quick to get the microphone away from me if I recall correctly. Since I was up on the stage I couldn't get a real read on the audience's reaction, so my thanks go to John for letting me know that it wasn't well received. I didn't say this to piss people off, I just felt that they deserved a better answer than the one given to them by Microsoft.

It was remarkable to me that things had come to this. This was around the time when a sweaty, chair chucking Steve Balmer stormed across a stage shouting "Developers! Developers! Developers!" At the time we were supposed to believe that he was telling developers how important they were to Microsoft, but I think he was aiming those chairs at them to shut them up for asking for better tools. To Microsoft, developers exist to sell licenses.

When they take away your cheap, though effective, tools like Access or start charging for Visual Studio, when they leave the operating system bereft of compilers, languages, interpreters, debuggers, build tools, version control or file management tools so that even the simplest things require the purchase of new software or enormous effort on your part, and when the hardware requirements continue to outpace the useful life of a computer then it should be plain for all to see that they don't give a shit about developers unless you're selling licenses and feeding beast.

If Microsoft really cared about its developers in the trenches, you'd be able to start programming real software as soon as the OS booted for the first time. A technology ecosystem that gives good tools to developers is the technology ecosystem that will be advanced by those developers. New and imaginative software will be created there. Think about that for a moment. When you've got a fresh install of any version of Windows, what can you do with it? Nothing. There's no shell, Perl, Python, C++ compiler, web server, web languages or application server. There's no way to simply manage directories full of *.bas or *.sql files. You can't diagram anything, you can't really edit anything but the crudest documents in the crudest way.

The dirth of tools for developers is probably the main reason that advances in computing have never occurred on Windows and likely never will.

On the other hand, the open source environment is built by and for developers. That's why all the things I mentioned above are available to you when you boot into a clean install, you may only need to issue a simple one line command to obtain them. The availability of these tools is the reason why open source operating systems such as Linux, FreeBSD and OS X have eclipsed Windows in performance, features and security for several years. All the interesting work is being done there and has been for a very long time. There have been no significant developments coming out of Windows. Multi-user systems? Unix. Protected memory? Unix. True multiprocessing? Unix. Hardware abstraction? Virtualization? Unix. Application servers? Unix. World Wide Web? NeXT, Web Servers? Unix. Security? Started on and is still strongest in Unix. Film production? OS X. User interface advances? OS X and KDE. File management systems? Solaris and Linux. Garbage collection and managed code? Solaris. I could go on, but you get the idea.

I'd like to think that I spoke truth to power, but I didn't really. I certainly spoke the truth, but Microsoft really isn't powerful anymore. No one is really afraid of them. And those to whom software is important don't really think of Microsoft very often. Between the competition from Googel, Apple, Linux, sanctions from the EU, dismissal from standards bodies and their own incompetence they really don't matter very much at all. When I read that John has left everything behind except for Widows itself then I know something has changed - for the better.

Microsoft is like that drunken sorority sister who's had too much to drink and doesn't realize the party is over. She spent the evening dissing everyone, but now she's shouting for affection and looks as though she's about to pee on the rug. The only difference is that the girl is dressed and Microsoft has no clothes.

A Tale of Two Developments, Microsoft versus Open Source

Long ago I wrote that I would relate my experience converting a fledgling system from Windows/IIS/SQL Server/.NET to an open source stack, but I never did. This is one of those situations where I've told the story so often that I feel as though I've already told it here, but that's not true, so here it is.

Back in the 1990s I created an application banks used to check their customers' names against a special list of terrorists, narco-traffickers and sanctioned governments kept by Treasury. This list, the list of Specially Designated Nationals, is one of the weapons at the disposal of the Office of Foreign Assets Control in their offense and defense of economic warfare.

Briefly, the government updates the list whenever necessary and publishes text files on a website. The files contain related lists of information such as addresses and aliases. Critically, the names have been translated from other languages and transliterated from other alphabets, so searching presents some challenges.

Early on, this application was a simple desktop application located in a bank's compliance office. It was written in VB with an Access database. Some commercial FTP tool was included to handle the download and VB code downloaded the text files, parse them, create the database codes and handled the transliteration issues, pumped the data into the data tables, cleaned up and presented a user interface. Access provided reports, which by their nature crude, user authentication and logging.

This solution was put together pretty quickly, but left much to be desired. Access was notoriously easy to corrupt, difficult to secure and hard to transport. If you've ever programmed text manipulation in VBA or VB, you know what a painful and resource intense task that can be. As a binary hairball requiring a purchased license, this solution could not necessarily be shared freely. Also, the solution required a particular environment, Windows and Office.

To extend the reach of the application we decided to make it web-based. Naturally, knowing no better, the software stack would be all Microsoft products, Windows, IIS, SQL Server and at the time the brand new .NET driving glittering ASP-delivered content to the obviously superior IE browser. Further, just to demonstrate how great the solution was, we would provide the data interactions through SOAP calls for interoperability.

Microsoft's .NET was quite amorphous at the time, more a strange brand than a development platform. While as a development platform it was interesting though new to me and the folks in my shop. We were too busy with some other things at the time to engage in this experiment of rebuilding a solution on a new platform. Instead I enlisted some colleagues who were interested in seeing how this would all work out. These fellow Microsofties had written with me and had spoken at conferences along side me and had already a couple of years of .Net under their belts by being beta testers, early adopters and evangelists for the platform.

What could go wrong?

That all depends upon your standards and expectations.

In keeping with Microsoft's recommendation at the time that every significant service should run on a separate physical machine, the new prototyped, web-enabled solution straddled 3 different servers. One contained the database, one handled the web server and another handled the ftp monitoring, downloading, parsing and logging. That's three boxes taking up rack space, drinking electricity, demanding administration, repair, patches and licenses. If I remember correctly, IIS wasn't free back then, you had to buy it or buy a special version of Windows Server to get it included and SQL Server certainly wasn't free. Moving from a desktop platform to a server platform still didn't provide adequate, programmable FTP capabilities, so we had to purchase that too. Did I mention the cost of .NET Visual Studio? Adding to the brittleness of the solution, there was no useful built-in logging capability and Window's scheduler was known to be broken, so we had to get a third party application schedule events like checking the government's FTP site for updates. What kind of server doesn't have a functioning scheduling ability? I think Windows still lacks an independent, reliable clock.

Following Microsoft's standards, and in light of the considerable hype from Microsoft about its offerings and .NET, my expectations were that this would be a robust application providing gazelle-like performance and the kind of reliability that would let me sleep through the night. Based upon the cost of this solution and the obvious horsepower being brought to bear on it, I thought these were reasonable expectations.

I was wrong.

Microsoft's suggestion to have only one significant service running on a server at a time wasn't intended to improve stability or performance, it was intended to sell licenses - they don't really care about stability. Each of the three servers is as likely to fail as any of the others, just at different times. Having three boxes running Windows in production meant that we had constant crashes, hang-ups, compromises, memory bust-ups, disk corruptions and reboots. Just about every problem that came up with this infrastructure required a hands-on solution because remote administration of Windows was practically unheard of at the time. Instead of sleeping through the night, it was as if I had incontinent triplets sleeping in cribs in the next room.

This was obviously the price to pay for performance though, right? After all, Formula One race cars are fussy, but you wouldn't race the Grand Prix in anything else, would you? I wouldn't have minded a little grease under my fingernails if I had performance I could have been proud of, but I didn't. Windows and .NET were so inefficient at handling strings and so bogged down with unnecessary interprocess calls within and across the servers, and gobbled up so many cycles and so much memory that the performance was abominable. A simple search to see if a customer's name appeared on the SDN list took a round trip of 7 seconds.

Read this slowly:

1 Mississippi

2 Mississippi

3 Mississippi

4 Mississippi

5 Mississippi

6 Mississippi

7 Mississippi

A human being flipping through an alphabetized list of the SDNs could almost match this performance. The solution's only efficiency came from maintaining the list, keeping a record of all the searches and ensuring that the first aid kit didn't run out of band-aids due to paper cuts. All these benefits were derived from non-Windows software. Natively, Windows provided non of these capabilities. Other than that, the system was a worthless and expensive failure.

I left the project behind and moved on to other things. Among the things I moved on to was the development of a new product, Agelis, and the search for a different default database to use with that product. As you may recall from my earlier blogs, it was that search that introduced me to open source software and defenestrated me. As my search through Unix-based, open source software carried me away I was getting resistance from some of my staff on my new view of the world. They wanted to see something concrete. They wanted proof that open source software could actually deliver.They were still picking crow out of their teeth after following me blindly into the morass of .NET. Their position was that this new fangled, web-based, SOAP stuff was just a fad and essentially untenable.

What they wanted me to prove was that an open source operating system, database, programming languages, web server, development and administrative tools would work better than the Microsoft's stack they knew.

Since I've already discussed FreeBSD and PostgreSQL you can see that the operating system and database are covered. Naturally, the FTP comes built-in on an operating system that's designed to work with other machines and Apache is the standard web server - everywhere, not just on Unix. Also, unlike Windows, FreeBSD comes loaded with language interpreters and compilers. So Perl, PHP, Python, C/C++ and Java are readily available, among others.

One of the real performance bottlenecks with the Microsoft stack was the SOAP service, .NET was simply not up to the task. Creating a SOAP service to provide XML output from a database query should be a simple thing. Since Unix handles text as a matter of course, it's been optimized over the decades to handle text and SOAP and XML are all about text. The only task I had before me was to find a really fast SOAP server. There were several choices. In the end I chose the gSOAP library and tool kit.

The new stack was FreeBSD, PostgreSQL, Apache, Perl, JavaScript and a SOAP server built using gSOAP. Logging was controlled by FreeBSD's logging ability and Cron took care of all the scheduling. Since each of these tools was so light-weight, I ran them all on the same server for the prototype with processing power and memory to spare. There were no memory leaks, no race conditions, no memory gobbling-oh-my-gd-we're-going-to-have-to-reboot -- again -- moments. Ever. Everything that took place on the system did so because we told it to take place and there were no mysterious processes running and absolutely no black magic.

Everything about this solution was better than the Microsoft version. First, it flew. The Microsoft version took 7 seconds to accept a search criteria, transform that criteria into a code, search the database, create the outgoing XML, deliver it to the web server and have the page rendered and sent to the browser. And it took 3 machines all running at the same time to make this happen.

The open source implementation, running on a single machine, did all this while using a small fraction of the system resources and returning results almost too quickly to measure. In production, we have been able to conduct millions of searches a minute without raising any concerns on the main server at all. I can administer every aspect of this solution from an ssh client on my phone.

And the software cost absolutely nothing. Zero. All the money I saved by not paying Microsoft I was able to sink into hardware, hosting and marketing instead.

What started out as an exercise to demonstrate the viability of a particular software stack became a profit making product. The product was always a compelling one, but the platform it needed changed. In an ever more competitive software environment, it's important that our raw material is as cheap as it can be and be of high quality too. Open source provides those benefits. I've seen for myself first hand how a product idea can be killed by Microsoft's lousy technology, but also by its lousy, expensive business model in which it charges for something when there is a better alternative available for less, or nothing.

The Claustrophobia of Microsoft Windows

Microsoft wants us to believe that "Windows" stands for something beautiful and ethereal, a view into our work, a way to see the future, something that lets light into our world so we can do everything we want. But Micorosoft Windows is no such thing. Microsoft Windows is nothing more today than it was when it was first introduced: boxes for programs to run in. That's it, boxes on your screen. Empty boxes.

That became abundantly clear to me when, after years of not using Windows, I made a fresh installation of XP on a VMWare virtual machine on my new Intel MacBook Pro after installing Freebsd 6.2 and Debian earlier in the day. I have to say that if you haven't done that in a while, please do it because the contrast is stark and experiencing it back-to-back is the best way to grok it.

After the installation of Debian and FreeBSD I had full powered, responsive computers which were both ready to accept thousands of software packages without cost or complaint. Both of these systems easily connected to my network, even the Windows domain that still remains, and found their way onto the Internet. I could run them remotely and even forward their desktop GUIs to my MacBook. Nothing surprising there.

Then I installed Windows XP. When the installation was finished I was at the desktop where I found.... nothing. Nothing at all. Windows was completely impotent out of the box. There's no decent software to be found when the installation CD stops spinning. No, Solitaire does not count as an application - especially if I can't even find something with which to write a decent memo. Clicking Start, Run and typing 'cmd' gave me a command line which I thought might help (it's been a while), but I might as well have been scratching my name in the ground with a stick as use the command line with its anemic command set and only about 20 viewable lines. Windows had trouble finding the network, even the other Windows boxes. It's downright claustrophobic to use Windows because there's no way out.

I know none of this is news to anyone, but I was so struck by how empty the view was out of Windows' windows.

My step-son has a computer in his room that runs XP. The box is only a couple of years old but was running like a dead dog. It had obviously been compromised by spyware, but I kept my distance out of respect for his privacy. Then it tanked and wouldn't boot. Now, since he had gotten a very cool Alienware laptop for his gaming, the old XP box wasn't doing much except running tax returns and iTunes. My wife asked me to see if I could fix it. I said I would install FreeBSD on it and it would be like new.

First, I used FreeSBIE and Knoppix to boot the computer from CD. Either of these would do, but I wanted to try both. (They're both great tools for rescuing other systems, so if you haven't tried these, give them a whirl.) Once under control of a Unix operating system, I was able to mount a 4GB USB memory stick to hold my wife's files. Rather than go through the muscle cramps of clicking and dragging the files and directories, I just wrote a quick script, ran it, watched some TV and checked it out later. It worked. I yanked the USB stick and sat back to figure out what to do with the now utterly unimportant and compromised box.

Long story short, I decided I wanted to try PC-BSD for this machine. I love FreeBSD and have set up some very nice servers and even a couple of zippy desktop machines using it, but I'd read some great things about PC-BSD and DesktopBSD and thought this would be a good time to get my hands on one of them. I chose PC-BSD because I thought my wife and step-son might like KDE more than Gnome.

The installation was a piece of cake. There are plenty of step-by-step accounts of how this gets done, so I won't bore you here with the details, but I will bore you by repeating that this computer probably never ran faster than it does now. I actually compared the CPU load on this machine running some software with my step-son's new Vista loaded laptop running nothing. The Vista laptop was breathing harder sitting still than the FreeBSD box was at a long, light jog.

So, it looks like Windows is a dog. It's anemic, triggers claustrophobia, catches more diseases than a tarted-up street-walking crack-whore and doesn't have enough energy to get out of its own way.

The reasons for this sorry state of affairs is because Microsoft doesn't know what it's doing. And it was made abundantly clear to me last night.

Before leaving on a business trip for the week I moved some Word files off the PC-BSD box to one of my PowerBooks for my wife to use. Copying was easy, but when my wife tried to open the Word files, Microsoft Word on my Mac complained that it couldn't read the file because the file format was no good. It was late, so I took the bait and tried to figure out what the problem was with the file formats. Geez. It turns out that I had simply forgotten to change the owner of the files once I had moved them into my wife's account on the Powerbook. So Microsoft Word couldn't distinguish between a file permissions error and an unknown file format. There's something deeply distrubing about an important piece of software that can't properly interpret fundamental information from the operating system.

Microsoft really doesn't know what it's doing, does it?

First Steps

In 2002 I wanted to create a software product to sell in the Public Finance space, Agelis. Agelis requires a relational database with solid capabilities, but large systems like Oracle or DB2 were overkill.

As I explored the problem of developing a solution to bring to market and checked out the stack of software that would be used to build the system I realized something important. I realized that my customers were going to pay only so much for our solution - for any such solution - and that amount was going to be based upon its value to them. Nothing more.

We knew that our solution was a distinct offering and was the embodiment of some special understanding we had attained in this problem domain. I felt that whatever money we could get a client to part with for that solution should be ours and not get shipped off to Redmond or Redwood Shores for no good reason.

I started a search for a cheaper, yet technically capable database server to use as the default implementation of Agelis. In the process I reviewed a number of products that were cheaper than SQL Server, Oracle and DB2 and quite capable.

In October of 2002 I was home recovering from a bee sting (I'm allergic, so it was a big deal) and watching my TiVO. As I drifted in and out of my Benedryl-overdose induced naps I thought about that trusty little box that simply never - never - gave me a hard time. And it occurred to me that if that ran on Linux, how bad could open source software really be? I thought to myself that maybe I owed it to myself to check into this.

The tree outside my picture window was alive with color and I drifted off to sleep.

But I didn't forget.

Before going any further I must remind you that at the time I knew absolutely nothing about open source software. Like many developers enveloped in the smothering embrace of the Microsoft ecosystem my view of open source software was as dismissive as it was uninformed. While I had heard of Linux, my impression was that it was a hobbyist's toy and nearly impossible to wrestle into submission. In the back of my mind I knew there had been shareware once upon a time and I knew there were some wild-eyed radicals running around in Cambridge, MA saying that all software should be free and I knew that I knew nothing more than that.

I had no idea what I would find and I'm still astonished at how my decision to look a little further for a database that was a little cheaper would so utterly change my understanding of what software was, what it could be and how it would change just about everything for me and my little company.

The first thing I came across was SAPdb - a bit off-beat, but that's what happens when you open yourself to new experiences. I downloaded SAPdb and really liked its performance and stability. Being able to use the little "Powered by SAP" logo would have been cool too. However, SAPdb was a strange database that presented a pretty steep learning curve.

As I looked further, I came across the usual suspects, MySQL and PostgreSQL. I'm actually embarrassed to admit that I knew nothing about these projects at the time. I'll cut to the chase and simply explain that we decided that PostgreSQL suited our needs best and I started to test it out.

During this time my Dell laptop decided it had better things to do than work for me and I was tired of trying to recover Windows yet again. So I slicked it and installed Red Hat Linux (and then Suse for the sake of comparison) and started running SAPdb and PostgreSQL. This was a huge step for me.

It had to have been at least 15 years since I stared down a command line and I had really no experience with Unix. While I certainly struggled with it - after all, I had a business to run, it's not as though I had all day to tinker with this stuff - I was able to get a pretty good handle on things. I was able to install software, connect to the internet, connect my network, do some x-forwarding, remote administration, run OpenOffice. In a couple of months I slicked my desktop computer and installed FreeBSD to see what that was like. This was a lot of software installing for someone like me. Things like this did not usually go well for me. I had wrecked so many Windows boxes in my life to conclude that either I hated computers or they hated me.

I preferred FreeBSD. I thought it was cleaner and very well thought out. I also liked that it was complete, but that's just me. I recognize that there are some very good Linux distributions out there and I have noticed that Suse runs like a gazelle and Ubuntu is a beautiful thing that tugs at my heart strings, but this is really about choice, right?

This whole expedition into open source started because I wanted to create a stack for our software product that didn't entail sending undeserved rents to others. But in the end I found so much more than that. I found that we had the ability to service the entire stack, from the base operating system code to the finest detail of the user interface. With open source software as the default implementation of our solution, we could look a client in the eye and know that we could give him unconditional support while knowing that the foundation of the system was rock solid. It really doesn't get better than that.

But could we really? Could we really create an entire product from open source software? My next post will show you that we did exactly that by converting an existing, faltering and failed product from its Windows.NET/SQLServer/IIS implementation to an all open source success.

Leaving Microsoft Behind

In life most relationships don't suddenly end. In reality they degrade over time moving from love and respect to indifference and then to contempt. Eventually there comes a point where the only thing left to do is to say good-bye. I came to that point with Microsoft in 2002.

Personally, I've made a complete break with all their products. However, my software company, The Oasis Group, Ltd. still uses and develops software for Windows. This situation is like having my ex's stuff still in my house - awkward and uncomfortable reminders of an unclean break. Windows still exists at Oasis only because we have one last project to finish before we can be free of it. Despite that, we haven't purchased a license for Windows or any Microsoft product in years and we have mostly moved to open source software instead.

This was not a casual thing. I was not an anonymous software developer banging out code for a living. I had a thriving boutique company that was doing interesting work around the world for the bluest of blue chip firms. We were in deep as a Certified Solution Provider, Certified Solution Developers, authors of magazine articles and several books on developing software with Visual Basic, SQL Server, Oracle and Access and I spoke at numerous conferences, some sponsored by Microsoft. All of my business associates, co-authors and fellow conference speakers were also Microsoft adherents. Until I made the break, alternatives were not discussed or even contemplated.

The fact that I made the break says less about any brilliance on my part than it does about the smothering effect Microsoft has on customers and on the developers they have brought into their ecosystem. I rarely found Microsoft developers who were not also apologists. I know because I was one of them. "Hello, my name's James and I'm a Microsoftaholic."

I guess I got here in some ways like a lot of people. After years of putting up with crappy software I was confronted with two realities by Redmond: endure having to get my software developers to learn what was at the time an untried and rather amorphous new development environment called .NET and start paying what I thought were exceedingly high new licensing costs for software that was proving to be increasingly unstable, obstinate, insecure and late.

But I had a choice. I could have chosen to go along in order to get along or I could make a break for it. I chose to make a daring daylight escape. I chose to run and I'm glad I did.

In other posts I'll explain in greater detail how this all happened, what I was seeking, what I've found and how my little company is going to continue to explore - and thrive - with the freedom it has claimed for itself. It's been an interesting trip so far and I think the scenery is just going to get better and better.