*Words of Wisdom From The Elder

Joe Stagner posted on his blog about the 10 people who don’t matter that was spurred by an article on CNN.  Joe’s post got me to thinking about open and commercial software and how the two mix.  It also got me to thinking about why one fails and another succeeds.  I’m not sure where this is going to wind up but here are some thoughts.

Apple has already found a way to use open source in a commercial business. I think they have a great blend of commercial and open source software.  For example, Mac OSX was built out of the open source operating system BSD.  We know this effort today as the open source project Darwin.  So yes, if you are using a Mac, your operating system is open source based.  Apple chose BSD over Linux for the OS primarily because the license that Linux used wasn’t condusive for a business.  If you read the GPL license and compare it to the BSD license you’ll understand.

Apple also took the foundation for Konqueror, the web browser integrated with the KDE environment, and turned it into Safari.  Well, sorta took, let me clarify that.  Apple primarily took the rendering engine just to be clear.  There have been some rumblings in the KDE community about Apple “not giving back” and the Firefox guys have had some issues as well , just checkout some of the newsgroups.  Time will tell if this relationship will work or not.

There are tons of other examples as to how Apple has embraced open source software.  Two more examples come to mind that I think other companies could learn from are:  OpenLDAP and Samba.  OpenLDAP is the open source equilvalent to Active Directory if you want to think of it in those terms.  Samba is the tool used to allow Unix operating systems to connect to Windows computer shares or printers.  The problem with these two packages is it would take a novice 6 months or an expert the good part of a day or maybe even more reading howtos to figure out how they work.  It’s not sometehing you are going to install by clicking next, next, next, next.  LDAP would probably take a month, it’s a beast.  What Apple did though is take these two projects and incorporate them into the OS so users could easily connect to a Windows PC or join and Active Directory domain.   Kinda smart if you think about it because they didn’t have to write it from scratch.

Now stop and think about it for a second.  Apple built their Mac OS X on an already existing open source operating system, built their browser on an already existing engine, and utilized standard protocols that were built by the open source community and incorporated those into their OS.  Hmmm.  Do you see a pattern?  I do.  What Apple is doing is taking the best of breed open source projects in certain areas and then providing a much needed feature to those projects so the average end-user can use them which is an Interface.  This is important and I’ll give some more insight to this down the road.

I’ve said this over and over again when I taught Linux certification classes at the college level.

“The open source community is great at building things based on a protocol or an RFC.  If you don’t give them a spec, you never know what you’ll get.”

That’s a bold statement I know, but allow me to give a few samples.  The main reason this occurs is the developers don’t have to think about what they are building.  The spec is already written so all they have to do is write their code to the spec.  Anyone that’s a developer will tell you this is some one of the easiest programming tasks in the world to do.  If all of the hard “engineering” has already been done for you, you are simply converting english sentences into code.  I like to call it translation programming.  If you have ever wondered what the difference is between a “Developer” and a “Software Engineer”, this is it.

This is why Apache, Sendmail, OpenLDAP, IMAP, POP3, OpenSSL, OpenSSH and other programs based on standard protocols and specs work very well in the open source world.  When you move away from these documented protocols and into something less tangible that businesses really want, like the ability to have a single application that does calendering, scheduling, email, etc (you know.. Outlook?), the open source community fails to produce anything.  To this day there is no equilvalent to Exchange in the open source community.  Why is that?  More on why later.  The only thing out there currently is Evolution which is an Outlook client knock off written for the Gnome Desktop that looks 15 years old (yes I am not a fan of the GTK interface).   Suse Linux has tried to “piece the puzzle” together by offering a web based version of Exchange using open source tools derived from standard protocols such as LDAP, IMAP, ICAL, etc.  I’ve demoed the product and it doesn’t work.  Sorry but: Microsoft 1, Suse 0. 

Let me answer my own question as to why the open source community hasn’t produced an Exchange equilvalent.    The simple answer:  There isn’t a spec.  Tell me how  the open source community is going to do user testing, gather wants and needs from businesses around the world and role that into an open source product?  The answer is they are not!  This is where things unravel for the open source advocate that preaches all software should be open source.  Open source projects (normally run by geeks) don’t specialize in interacting with businesses, end users, user testing, and usability.   Aha!  So now you see why Apple providing the missing piece to the open source products they’ve added into their OS works.  Notice I haven’t said one bad thing about open source projects.  Honestly, the ones built around protocols and existings RFCs are great but they lack that vital last piece, Interface.  Apple and Micrsoft for this matter shine in this area.  Microsoft to take this one step further shines at the collecting of business wants and needs and then rolling it into a product.  Businesses need to do word processing so they created Office.  Businesses want to do calendaring and email so they created Exchange.  Businesses wanted to share information across different boundaries in the Enterprise so they created Sharepoint.  Where is Apple in all of this?  Well they are hardware focused and simply use software to sell their hardware so they could care less what the Enterprise wants or needs.  But they’ll sell all of your employees an Ipod though!

Something else you should take note of is the correlation of successful open source projects that have a true “leader” to those that don’t.  For example:  linux, perl, php, python, ruby just to name a few.  I bet most open source advocates could name you the person behind each of those projects.  Perl has been around for years and if you don’t know of Larry Wall then I can’t help you.  The reason these projects are successful is they didn’t adhere to the full “bazaar” concept as outlined by Eric Raymond.  Instead, there was one guy making the final decision, not the bazaar.  Even if that decision was wrong, one person ultimately said what was going to happen or be in the release of the product.  What about Steve Jobs or the Program Manager for Exchange? This is no different.  At some point, you have to have someone at the end of the Lifecycle of development to say, “Yes” or “No”.  Whether this is your PM or the guy who started the project in the first place it doesn’t matter.  If you let people program wildly on your project you are going to wind up with extreme inconsistencies.  Some great examples of this can be found in an open source prorgramming language that I’ve used extensively called PHP.  Even PHP programmers are disgruntled because of inconsistencies.  Is it str_replace() or strreplace() just as a quick example.

Everyone looks at Linux as the open source defacto standard but honestly after having used it for years the whole thing is just a mess in my opinion.  There are over 10 prominent versions of Linux in the wild each with their own way of doing things and their own packages.  Why is that?  Well, Linus just focuses on the kernel, but who is the “go to guy” that says Yes or No for the rest of it?  The answer is he/she doesn’t exist.  If you install the various versions of Linux you’ll find files in different places, different versions of software bundled with each system and different packaging systems.  From a developer standpoint developers can’t even write to a standard interface API.  Should they write their app so it works in KDE, or Gnome, or Enlighenment, or FVWM, or whatever?  How are vendors suppose to write programs to all of this?  Most of the time they don’t.  They choose one vendor’s project like Redhat and write to it.  This fact has made me mad on one or more occassions when I had a server running something other than Redhat that I needed to install a commercial product on.  After all wouldn’t you think the open source community would want vendors to write stuff for their platform to give it more market share?  It appears they don’t because there is so much effort going into creating new ways of doing things.  Release early and often they say and if we break something in the process for someone else who cares. 

My point to all of this is Linux is a mess because of the “bazaar”.  Sure Linux is great for a web server and I use it for that to this day.  Why is it such a great web server?  Easy.  Because there was a spec on how to build the web server.  Anything other than a web server and I’m not interested.  I want to dabble in open source software that’s been polished I’ll turn on my Powerbook and play.  This probably explains why a lot of fellow linux advocates that were disgruntled like myself switched to Mac OS X.

So you see there are problems with open source software.   The open source world isn’t a perfect little bubble where everything is all dandy with penguins jumping around hither and tither.  To get the results from the open source community into the hands of end users  the open source projects need commericial interaction.  The project needs the “guy” to say whether or not something is going to get put into the product.  The project needs a lot of TLC spent on it to polish documentation, make things “Forrest Gump” and easy to use.  Look at MySQL as another example of a company who’s product is open source and the difference between it and PostGreSQL.  If you didn’t know anything about either one, which would you rather use?  More than likely MySQL because it has better documentation is easier to setup and configure, has a better interface etc. 

Companies like Microsoft, Apple, Sun, IBM and others can get ahead of the competition if they choose to add that missing piece and use open source effectively.  The question now is, which one besides Apple is going to do it?

For a long time I’ve been wanting to put this together.  A lot of my friends who were open source advocates like myself are perplexed with my transition to what they call the “Dark Side” of development.  Referring of course to Microsoft as the evil empire.  I took some time this morning to finish how and why I migrated from programming mainly in PHP to .Net.  I didn’t want to put this information into a blog post because as I was writing it, it just didn’t feel right so I created an article out of it.  In order to give you the full picture I give some information on my background to cement how firmly planted I was in open source technologies.  Then I give you some of my thoughts on the conversion process.  In the end, I give some advice for others that might be curious as how to make this conversion.

NOTE: This is the original article below (thank you wayback machine!)

Intro
Let me be the first to point out that I haven’t always done .Net. No one really has since it has been around for only a few years, let’s be honest.  For all practical purposes the technology is still fairly new.  Before .Net I was primarily developing web-based applications using LAMP (linux, apache, php, mysql).  The $1,000,000.00 question is how does one transition from LAMP to .Net?  And more important is why?  After all, aren’t those technologies left and right brian technologies?  These are the questions I hope to answer in this article.

My Background
Let’s first start out back in 1993 when I was in college.  In 1993 most computers were  Windows 3.1 and a few people were holding onto their Amigas while others were running a Mac.  486 computers reigned supreme and Doom 2 was the big hit.  I was bored with my Windows 3.1 486 computer though.  I had read just about every help file in the whole system and knew all the command line switches to programs that the average computer user didn’t even know existed.  I then got introduced to Linux by the primary Unix Administrator at the college.  I remember the day vividly because he called me into his office because I had taken over the X11 terminals in the lab and he wanted to know how I did it.  Sitting in his office he had a Gateway P133 machine running X11 (the primary graphics front-end to unix operating systems).  I of course questioned how he was running it on a standard PC because up until this time, standard PC equipment couldn’t be used.  Even though I was in trouble I was still curious.  He explained it was a new unix like operating system that was free.  He then showed me how he was using it and I was hooked.  The thought of running multiple programs at the same time on my PC and using the same tools the college was using (mainly so I could hack my friends shell accounts) was appealing.  I immediately went out and bought 50 floppy discs, download Slackware and started compiling.  Trying to get things to compile and work were just fun.  I spent 6 months just getting my Trident 1MB video card to work with XWindows.  I have no idea why at the time but looking back on it I guess this more challenging than the assignments I was getting from the teachers.

By 1994 I was working for the college IT department and we setup one of the first known Linux labs.  I think it was the first Linux lab in the country but I don’t have any proof, let’s just say we were an early adopter.  In the lab students could browse the web, write papers, all within their shell accounts on the cluster, or they could dual boot the computer into Windows.  Pretty cool stuff for 1994.  After I graduated college with a music degree and a computer science minior, I continued playing with Linux and open source software.  When my wife and I moved to Ann Arbor, MI, I joined a local Linux User Group.  It wasn’t long after joining I started presenting at the group with topics ranging from how to get your TV Tuner card to work in Linux to “Migrating Business Processes to the Web Using Open Source Technologies”.  I spent a lot of time coding and doing system administration in the open source world and in 1997 started my own web hosting company which I still have today.  Most everything I did after 1997 was based on what the community calls the LAMP model (Linux, Apache, Mysql, PHP).  I didn’t even have a Windows PC, except for my wife’s computer.  After doing so many Linux presentations and becoming somewhat of a go to guy based on my professional experience I was invited to teach at a local college.  There I taught four classes on Linux which led students to a Linux Certification.  The classes included an introduction, system administration, networking and security.  I also taught the first PHP class at the college and continued to do so until it got taken over by another professor.

After using Linux for years, I had taken the OS just about as far as I could take it.  I was growing tired of reading Howto’s, compiling code, posting to newsgroups, loading untested software and drivers and bringing down my PC for days at a time.  For me the experience was about learning.  In other words, why break a production server when I can break my own?  I was growing tired of this routine nevertheless.  Using a computer with Linux on it was like, well, work.  Using Linux on servers I was happy as a lark.  But, for my personal day to day machine, it was work.  Why couldn’t I just plug something into my personal PC  and it work?  Why did I have to touch the command line all the time?  Why did I have to compile or hack a driver just to plugin a digital camera and transfer a picture to the PC?

Once Apple announced Mac OS X I started watching it pretty closely.  I was waiting on the moment their new operating system (which was unix based) was ready for prime time.  In 2002 I decided to switch from Linux as my primary OS to OS X.  I acquired a Powerbook and an Ipod and was very pleased.  OS X was such a blessing over Linux.  I was a true convert even though my attempts to get into the switch commercials at the time failed.  I was shouting OS X from the roof tops.  I no longer had to fight Linux on a laptop to get work done and was able to do so much more.  During my Mac OS X switch I was working for a consulting company who focused on building web applications using the LAMP model. We held PHP User Group meetings at the company and several other members were moving to OS X as well.  It was a great combination.

In 2004 I was hired as a PHP developer for Quicken Loans.  When I joined the company I was given a dual processor Mac running OS X and told to write PHP code.  This was great!  I was working for one of the best companies in the country and getting to do what I loved.

As you can see I was an open source developer.  It’s the only thing I really knew and the only thing I had ever really done.

The Conversion Begins
After working at Quicken for several months I started to change.  Call it whatever you want but working for a larger company changed how I started to view things.  I can’t explain it but I started to somehow think differently about the technology I was so passionate about.  Was I maturing?  I don’t know.  In retrospect I think once you absorb a technology so deep, two things happen.  One is you get bored.  The otherthing that happens is you know all of the strengths and weaknesses of the technology and you find yourself looking at other technologies to plug into the holes.

Early in 2004, I started playing with .Net while researching Sharepoint Portal Server for the company’s intranet.  The company intranet was written in PHP and I was assigned to the intranet.  We were having lots of problems with single sign on and other things like integrating with Active Directory.   Without giving too many details away, let me just say that our intranet is huge.  Thousands and thousands of pages, applications, content, data, etc.  Basically if you want to know anything, you go to the intranet.  I started researching Sharepoint because I was asked to look at it.  It wasn’t something I would have started looking at on my own given my background that’s for sure.  As I researched Sharepoint and Asp.Net, the more I read, the more I became intrigued by it.  As I mentioned earlier I had become bored and disgruntled with PHP.  I felt I had done just about everything there was to do with the language and the platform.  And, let’s face it, PHP is just for the web.  I wanted something more.  Something that challenged me mentally while also making my life easier.  Something that I could learn that would provide me a one stop shop for whatever I wanted to write as a developer.  I was looking for a Swiss Army Knife. I wanted to be able to learn the ins and outs of one technology and then write either a web application, desktop application,  mobile application, smart phone application or anything in between.

While investigating .Net I decided I was going to take my open source advocate hat off and place it in the floor. I then approached .Net as if I was an unbiased “architect” who had to make a decision which technology he was going to recommend.  Once I made this conscious decision I started to notice how well thought out the platform was.  .Net wasn’t just a scripting language like PHP, this an entire platform full of swiss army knives!  I finally managed to get my hands on Visual Studio and this is where I became even more impressed!  But wait, this is a Microsoft technology!  This can’t be good.  I mean after all Macs rule supreme and open source rocks.  What was going on?  Could it be I was turning to the “dark side” after all of these years of avoiding it?

All the while I was researching .Net I was still writing PHP applications.  This is where I started to differentiate the technologies and this is important.  I started to catch myself saying things like, “I could have done this a lot easier in Asp.Net”, or “Man I wish I could debug this better”, or “Why do I have to build my own session object or cache object and waste my time?”.  I could feel the dark side coming over me as I added more features to our intranet and built additional one off applications.  Writing code in VIM or Zend Studio on my Mac started to become more of a chore or a task and I found myself fighting the tools I had used for years.

Defending the “Dark Side”
Obviously by this point I was at a crossroad.  Then one day I got into a discussion at the water cooler with a coworker.  I found myself in the middle of a .Net vs PHP dual.  My coworker had his PHP pistol drawn and was firing away at me.  Every shot my coworker hurled at me I was able to deflect.  This surprised me.  Speed he mentioned for example.  Asp.Net code is complied I said unlike PHP which is just a scripting language.  I can compile my PHP code to byte code he added, and then I said yes, but you have to purchase Zend’s compiler.  I don’t have to purchase anything I continued adding.  He then drew back his open source pistol and hurled another bullet at me and said, well, PHP is free.  I then said, so is .Net.  Anyone can download the .Net framework, install it and start writing .Net apps.  I then added there was even an open source project called Mono that is taking the .Net framework and building it cross platform.  My coworker had fired two shots and my .Net shield had deflected them.  What was going on?  Why was I defending the technology I had used for years?  He then had one shot left in his open source barrell.  He pulled the trigger and “security” was fired at me.  This was the one I knew he was going to bring up and  I had researched this very topic and was shocked at what I found.  So he states, “well, that doesn’t matter, PHP is still more secure than .Net”.  By this point in the dual he was getting frustrated and had starting moving from fact based arguing to emotional based arguing.  I could just tell by his tone.  Then I said, “That’s interesting you say that because I did some research on it and comparing the .Net framework to PHP since .Net’s first release, it has only had 3 security holes.  PHP has had 43.”.  My coworker then dropped his open source gun and said, “hmm, really?”.

Conversation Complete
At this point I knew my conversion to the “dark side” was complete.  I had evaluated both technologies.  One I knew extremely well, the other I was just scratching the surface but it didn’t matter because the more I dug into .Net the more I moved away from PHP.  By this point my love affair with .Net was starting to spread internally.  When one of your coworkers on the Unix Team starts calling you “Syth Lord Elder”, you know you have turned (true story, and he still calls me this to this day but he’s trying to find a word higher than Syth) to the “dark side”.  To my coworker’s credit who was at the water cooler the day I defended the “dark side” he started researching .Net. Today I am proud to report he and I were founding  members of our internal .Net team.

Since my “re-birth” as a programmer as I like to call it, I’ve done quite a bit with .Net.   I spend most of my time building a Smart Client at work along with a lot of web services and web applications.  I am also building mobile applications and working on a mobile CRM product for all of our bankers in my spare time.  I also spend a lot of time training and mentoring to other programmers who are moving from various languages to .Net too.  I enjoy the training and mentoring since I have an education degree.  I need to use it for something right?

Word of Advice
That’s it.  Now you know how an open source LAMP developer made the transition to .Net.  It has been a long road for me needless to say.  Getting out of your comfort zone and learning new things can be frustrating.  Trust me I know. Things that would take you two minutes, now take you twenty minutes because you have to read how to do it within the new platform.  My recommendation for anyone that wants to learn .Net that is coming from another background as I did is this:

1. First and foremost throw yourself into a really good C# book.  Don’t bother reading anything else.  Learn the basics of the language first, then learn what the .Net framework provides you as a developer.  Remember that the language and the framework are two seperate things.
2. Second, dive head first into the framework on MSDN.  Dig into the namespaces of System.Data, System.Collections, System.XML and those type of things that you are going to use all the time.
3. After that, download Visual Studio Express for free and learn how to use the IDE.  This is where you’ll really start to see how complex things turn into simple drag and drop operations.
4. Lastly, don’t get discouraged or overwhelmed at the amount of information on .Net.  There is tons of information about .Net on the Internet and it is easy to get overwhelmed.  Remember that Microsoft is a big company and they have 1,000’s of developers pumping out code everyday.  Don’t let it discourage you.

Drop me a Line
I was speaking to one of the Microsoft reps in Jackson, MS at the launch event and he says I’m the only person that he knows of doing .Net programming in Hattiesburg, MS.  That doesn’t surprise me, but by the slim chance that someone from Hattiesburg reads this for the love of a good framework contact me!  Just take my first name and this domain and you’ve got my email address (decrypt that spam engines!).

I found a really good article which outlines some nice tips to scaling PHP. The article is written by George Schlossnagle. In his article he explains why you do not need large DB abstraction layers like PEAR and ADODB and why you should NEVER use relative paths to images. The article can be found here.

The following was taken from Linux Magazine from an article written by Jeremy Zawodny. It talks about a litlte known feature of the MyISAM tables in MySQL which allows you to create raid tables. The snippet of the article is publised below.

RAID Tables The final variety of MyISAM tables isn’t widely known, but can be useful in some circumstances. RAID tables are simply MyISAM tables whose data (.MYD) files have been broken into multiple files. CREATE TABLE mytable … TYPE=MyISAM RAID_TYPE=STRIPED RAID_CHUNKS=4 RAID_CHUNKSIZE=8 Running that SQL command breaks the MyISAM table into 4 separate files (chunks) that are written to in a round-robin fashion in 8 KB stripes. Why do that? If you’re using a filesystem that places a limit on how large a file can be (2 GB or 4 GB), RAID tables work around that limitation. (However, file size limits are becoming more and more rare, as recent Linux kernels don’t have size limits anywhere near that low.) Performance is another reason for RAID tables. By putting each chunk on a separate physical disk, you can spread the I/O work out more evenly. In most MySQL installations, the main bottleneck is disk I/O, so this isn’t very far fetched. However, if you have the option of using hardware RAID or even software RAID at the OS level, you’re probably better off doing so.

Reference: LAMP Post, by Jeremy Zawodny

The LogiCreate Applicaton Server built with PHP was released as open source on June 5th, 2003. LogiCreate is a framework to build portals, CMS, CRM or custom web applications. It is extremely fast to start developing applications with, very little code for the developer to start writing and is extremely different from pretty much anything else out there right now. You can download the PDF presentation on the LogiCreate site as well as register on the site to download the system. We’ve been using it for several years now and will start offering hosting accounts pre-packaged with it soon.