You have reached the blog of Keith Elder. Thank you for visiting! Feel free to click the twitter icon to the right and follow me on twitter.

Erlang – Hot Code Loading on Nodes For Dummies

Posted by Keith Elder | Posted in Erlang, XBOX 360 | Posted on 20-02-2014


Erlang – Hot Code Loading on Nodes For Dummies

Hot code loading across nodes in Erlang is a fantastic feature. After hearing about this feature it peaked my interest early on about Erlang and got me to dig in more.  But, I couldn’t get it to work because of my newbyness to Erlang! Hopefully this blog post will help others. 

Imagine You Are a .NET Windows Service

To understand how hot code loading in Erlang works let’s picture for a moment a Console application or Windows Service or Web Service deployed to three different machines. As an Engineer we need to do a simple fix and redeploy code. What do we have to do? Well we have to make our change locally then build the solution, run unit tests, etc. Now it is time to deploy our simple change. This is where things start diverging fast between Erlang and other technologies.  In .NET/Java we have to copy the entire library to each server and then cycle through stopping and starting each process on each server so the new code will get picked up. Whether this is Java or .NET doesn’t matter the same thing applies. Even if your language of choice is a dynamic language like PHP/Ruby/Python you still have to copy the file you changed to all servers. The server or process doesn’t have to be restarted in dynamic languages but you still have to copy the file(s) to the various servers either way. In Erlang we can simply connect to a node and then network load the new code across all nodes in the cluster! 

See this is the big difference. Erlang is distributed out of the box. It was built to be distributable and continue running, even during code pushes. Erlang was designed with the purpose in mind that a system had to keep running. Think about it for a second. What if the phone company said you couldn’t make a call between 9:00 PM and 1:00 PM on a Saturday night because of maintenance window? Yeah that wouldn’t go over very well. There are numerous reports of uptime in systems written in Erlang. Many fans of Erlang use the terms of 9 9’s (and there is even a company called Nine Nines) which his 99.9999999% uptime. If you do the math that is about 30 milliseconds of downtime a year. What? Yeah. That. 

Here’s a quick walk through on how to do this. I tried several times to get it to work and kept running into errors and getting errors so I’ll cover them so you won’t have to struggle at first as much as I did. Here we go.

Install Erlang 

I assume you wear big boy pants and know how to install software but here are a few quick ways to get Erlang installed and up and running. 
  • Mac – use brew and “brew install erlang”
  • Windows – Download Erlang from Erlang.Org
  • Debian Linux – sudo apt-get install erlang
  • CentOs Linux – sudo yum install erlang

Open a terminal / cmd prompt window after it is installed and type “erl”.

If you’ve gotten this far, good, you have Erlang installed. By the way to exit type “ctrl-g” and then “q” (for quit). 

Now let’s write a simple program that will add two numbers. 

Our Program – demo.erl



add(A, B) ->
   A + B.
Copy the above code into a file named demo.erl. Note: it has to be named demo.erl because the first line declares the module name as demo. This is based on a convention as the name of the module and file have to match. 
While Erlang is a dynamic language it is still compiled. To compile this from the cmd line type: “erlc demo.erl”. If there are no errors the command prompt should just return. If there are errors they will print to the screen. If you are typing by hand be sure each line ends with a dot (period). 
The above file isn’t magical, it simply adds to numbers. For those coming from a .NET/OOP background think of the export line as making the function add public. Since Erlang is a functional language there are no classes! This is a good thing as Erlang programs are a lot smaller than languages such as C++/.NET/Java. 

Setting Up Our Nodes

Because Erlang is distributed we can create multiple nodes and run our module demo across three nodes. Typically you’ll do this on completely different servers but because Erlang runs within a VM we can start three instances of the VM locally, connect them, and we have a distributed program. To do this open three terminal windows or command prompts. I’m using a Mac so I will be using tmux in iTerm2 for these examples. 
If you want to follow along with my screen shots be sure to install tmux (brew install tmux). I’ve created three cmd prompts inside of tmux and each one I’m going to run the following command to start an Erlang instance in each one:

erl -name {flintstonecharacter}@ -setcookie dino

As you type each one in your command prompt change the name of the flint stone character. I’m using the following: fred (node1), bambam (node2), betty (node3). After doing so you should have three Erlang prompts that look like this if you are using tmux. 

So what did we just do? Well the -name gives each instance their name of course. The -setcookie option is super important as this is what really connects all three Erlang VM’s together. That cookie has to be named the same.

Running Our Code
Now that we have al three started let’s load our demo program into the top one and run it. 
To do that we are going to type the following:


The result of running that command should be: {ok,demo}. That means our code was compiled. Again be sure the demo.erl is in the folder of the first Erlang shell when you launch it. To run it simply type:


Press enter and you’ll get back those numbers added together. 
So far our other two instances know nothing about our demo module. But let’s clue them in so they can run it too! To do this we are going to ping the nodes so all three are communicating. In your first Erlang shell type: nodes(). This should return [] which means there are no other known nodes. Let’s add them though.
To do this we are going to type the following into the first instance (fred):


Now type nodes(). and you should get back a list containing both bambam and betty. 

At this point all three are connected. We can now hot load some code for them all to run. Let’s push our demo.erl program to all of the nodes (this works if you have 3 or 30). To do this we are going to use the function nl() which means network load. Run this command in your node1 instance (mine is fred). 

Notice we get back “abcast”. Now we can go to the other nodes and run demo:add(4,5). and it works! We just hot loaded code to all the nodes in our cluster. 
If you aren’t sitting back in your chair right now scratching your head going *holy cow* you should be. This is amazing. Take it from me. To do this in other technologies would be mountains and mountains of work. Trust me, I’ve done it. 
In the case above we basically deployed new code to new instances. What if we want to change it will it is running? Basically the same thing is just as simple. Edit the demo.erl file and add the following line above the A + B. line. 

io:format(“Calculating like a boss!\n”).

This is the equivalent of doing “Console.WriteLine()” for those .NET folks. We can test this out on our first node by running c(demo). again. Run demo:add(1,2). in the shell and you should get something like this:


Great it worked. Now let’s push our change to all our nodes! Simple. Just re-run nl(demo). and then re-run a calculation on each node. Boom! Hot code loading. 

Hopefully this little walkthrough will give you some encouragement to look into Erlang. I left out a lot of details about how things worked intentionally as if I didn’t the article would be just too long. Thank for reading and more Erlang articles to come. 

Our Xbox Avatars

Posted by Keith Elder | Posted in XBOX 360 | Posted on 20-11-2008


The new Xbox Experience launched yesterday.  I got up early in the morning to update the system and play before walking to work.  Overall I like the new experience, it definitely has a more social appeal to it and things are more discoverable.   

Last night, Ellen and I sat down to make our avatars.  This turned out to be tougher than expected because, honestly, there weren’t enough basic options.  For example, there was only one page of noses and only a few mouths.  Ellen and I went through every option on the menus and this is the best we could come up with for me.  My picture is at the top of this blog, so I’ll let you be the judge as to how close we were able to get.


We  then started working on Ellen’s and I think we were able to get a better resemblance with her.  Here is hers. 

image image

If you are curious as how to get your full body avatar here is the secret.  You go to this link:


By the way, my Xbox handle is “Z0rka” (that is a zero not an oh) and Ellen’s is “Sonata109”.  Friend up with us if you like.

Guitar Hero – The New Drum Kit

Posted by Keith Elder | Posted in XBOX 360 | Posted on 02-07-2008


News has been floating around that EA/Harmonix/MTV is going to release a new version of their hit Guitar Hero this year entitled……(drum roll)……. (pun intended)…….. Guitar Hero World Tour! (cymbal crash!).  One of the changes is they are adding a drum kit.  Being an avid Rock Band drummer I was happy to hear they are adding crash cymbals to it as shown here.


So what’s not to like?  Personally, and this is just me, I think it is missing an extra tom tom to the right of the green pad.  If they really wanted to provide a real feel to the drum set there should be another tom next to the green pad.  Of course that would give 7 total inputs but still manageable.  Actually thinking about it I’m not even sure why I’m asking for more pads to be placed onto it, I can’t play a lot of songs on expert now as it is!

I just hope they make the bass pedal more resilient so it doesn’t break like Rock Band’s.  I hope to be standing in line to pickup the latest version when it hits the shelves.

DreamSpark – Fantastic Idea But Only The First Step

Posted by Keith Elder | Posted in .Net, Smart Clients, SQL Server, Windows, XBOX 360 | Posted on 27-02-2008


When I was in high school one of my math teachers took it upon himself to teach a few students about computer programming.  It wasn’t a real class it was just something he put together during our free period.  Instead of attending study hall we’d go to this computer programming class.  Basically other students in the school called the few of us taking the class the “Star Trek” club.  Honestly I didn’t watch Star Trek and never have.  It just never interested me.  I know that I just lost thousands of geek points by stating I am not a Star Trek fan publicly but when you grow up on a small farm in Mississippi things need fed, watered, and slopped.  Coming home from school and sitting down to vegetate to watch TV wasn’t in the cards of our household.  Anyway, I digress.  Even though Mr. Foley didn’t have a lot of support from the school in putting together this class, it was the one thing that really got me hooked on computers.  Mr. Foley was only one man though.  It takes a much larger effort to get students interested in technology.  To hook the next generation of students on programming Bill Gates recently announced DreamSpark.  It is a new project that will provide thousands of dollars of free software to students.  Students like me who didn’t have access to the tools companies were using while they learned can now install and leverage the same products for free.   As an educator I’m really excited to see this announcement.   Here are some thoughts as to why I think this is important and what should spark within the community.

What is DreamSpark?

DreamSpark is simple, it’s all about giving students Microsoft professional-level developer and design tools at no charge so you can chase your dreams and create the next big breakthrough in technology – or just get a head start on your career.

DreamSpark makes the following software available to students who register on the site.   The program is available in 11 countries giving millions of students access to professional developer tools.


All a student needs is a computer to get started.  They can load Windows Server 2003 onto the machine along with Visual Studio 2008, SQL Server 2005 and Expression Studio and have the same tools we are using today in enterprises across America.  If the student wants to build games for Windows or the XBox then they can install XNA Game Studio.  The amount of things that can be built with these technologies is amazing, especially when you think about WPF Windows Applications and Silverlight.

Ok students, Bill has done his part to give you the software.  What are you going to do with it now?  Someone has to replace a lot of the Elder generation as we age.  Even right now there is a huge demand for developers that know these technologies.  Here are some numbers from the popular online job engines.

Searching for .Net

  • Dice.Com – 11,131 jobs available
  • Monster.Com – Over 5,000 (max return value)
  • CareerBuilder.Com – 3,376

Searching for SQL Server

  • Dice.Com – 14,588
  • Monster.Com – Over 5,000 (max return value)
  • CareerBuilder.Com – 8357

Searching for Windows

  • Dice.Com – 15,480
  • Monster.Com – Over 5,000 (max return value)
  • CareerBuilder.Com – 16,933

Positives About DreamSpark

I Couldn’t Even Buy A Vowel

As I said earlier, the DreamSpark program is a great idea I just wish it was around when I was younger.  When I was in college I was still doing off to the side programming while pursuing my music degree.  I knew there were tools, IDEs, databases and other things that people in large companies used but I was broke.  I was financially embarrassed as my father used to say.  I was so broke I couldn’t even afford to buy a vowel on Wheel of Fortune.  The only option I really had was to learn open source tools.  Back then the open source tools were, well, hard.  There weren’t web pages full of documentation, books, and fancy editors.  You basically first had to learn the VI editor and then try to dig through the man pages and the source code.  To say the experience was painful would honestly be an understatement.  It was slow going.  The sad fact today is that a lot of students are in the same boat I was back then.  DreamSpark fills this gap and puts the latest technology in the student’s reach.  This is wonderful, I can’t express that enough. 

Challenges DreamSpark Faces

It is important that students be offered the opportunity to learn the tools we are using today but the sad fact is only a few will take advantage of it without the community getting in behind this effort.  If 1% of the students downloaded these tools, I would consider that a success.  A perfect benchmark to go against is every student enrolled in a Computer Science program and every student in high school taking computer programing.  If 100% of those students download and use the software then that is a perfect world. 

In order to achieve this goal, it is going to take a lot of effort from those of us in the community.  The following is an open letter to Microsoft, the community and educators who can effect change and help create the next generation of programmers.

Give It Away And They’ll Come

The first thing we must overcome is thinking free equals a lot of use.  Some have heard the line before “Build it, and they will come.”.  The reality is just because something is free doesn’t mean anyone will use it.  If this were the case, we wouldn’t need ads, marketing departments, or ad agencies.  This is the first thing we need to do, spread the word.

Ask yourself this question.  How many students will seriously go download this software?  How many of them will actually learn about it?  There are numbers in the industry I’ve heard repeated of the 5% geek rule.  The premise is take 100 computer science students and only 5% of them are truly dedicated to their profession.   This top 5% are the ones that will more than likely take advantage of DreamSpark, the rest will never know.

We (Microsoft, the community, and educators) need to make a concerted effort to spread the word about DreamSpark.   We need to establish DreamSpark Day events at schools and Universities in our local areas and get this information out.  By information I am not just talking about the bits, that isn’t enough. 

How Long Will It Take?

Here is a pondering question.  How long will it take the education institutions to realize they have thousands of dollars of software available they can build an entire curriculum around?  Unless a concerted effort is made the answer is years.  Why?  Well for starters very few of the professors know the technology.  Sad but true.  When I say “few”, I mean a very small percent. 

There are a few professors out there that are teaching .Net.  Those educators should have no problem, but the majority of professors I’ve encountered love to tell you how their first computer was the size of a house and how they punched cards to program it.  What really saddens me is the “dinosaur” professors as I like to call them are still doing things the way they have for years.  Thankfully this isn’t every professor out there, but there are professors at colleges that still rest on their laurels and just learn enough to teach out of a book.  The majority of these professors have no real world experience in technology except in academia because they have spent their entire lives in academia.  I know these types of professors exist because I had several of these “dinosaur” teachers in college.  I used to call them the “Punchcardasaurus”. 

“Punchcardasaurus – A professor who loves to tell you about the stone age of computers but doesn’t know anything about today’s current technology.”

I can’t begin to tell you how much of a waste of time it is for someone like myself to have to sit through a lecture from a Punchcardasaurus only to correct him or her about how the Internet really works and the Mosaic browser is not really the Internet (true story).  I’m starting to digress but hopefully the point is driven home.  A concerted effort needs to be made at the institutions of learning.

Educational Institutions Already Have Free Software – Open Source

I firmly believe that higher educational institutions should focus more of their efforts putting students in a position to get a job and succeed.  Sadly the curriculum of colleges don’t.  Someone graduating from college today should have experience with all sorts of technologies and languages.  Colleges that just teach C++ are doing their students a grave injustice.  Students should learn C, C++, C#, Obj-C, Java, and a variety of other languages including dynamic languages.  Each of these should expose students to the various types of platforms such as Windows, Unix and OS X. 

The majority of colleges use a lot of open source software to teach their students today because, well, the software doesn’t cost the college nor the students anything.  If a college is just focusing on open source software, they are really doing their students a major disservice.  Students should be educated on both sides of the fence and taught multiple platforms.  In the end it is the students who should decide which platform they feel will be the most valuable to their career or interest, not the educational institution.  The more this is done, the more the students  know and the better chance they have at getting a job.  DreamSpark of course helps to fight the cost factor argument with open source software but it is going to take time for the value to sink in.  Especially in institutions where open source software has a very strong hold with a lot of zealots to play devils advocate.  Yes, these zealots exist, to ignore this fact would not prove wise.

Train The Trainer

The Punchcardasaurus and other professors need help.  It has to start with those teaching others.  Here is another sad but true fact.  Even if a school wanted to build a curriculum around DreamSpark who’d teach it?  If it was offered the odds of it being taught with real world business experience on the platform is very small.  We need to do something to train the current educators how to use these new tools.  There are a variety of ways this could be done.

  1. Use the current Microsoft training curriculum and certification courses to offer classes to faculty and staff.  Schools are off in the summer and it would be a great time for re-educating.
  2. Help the teachers by putting together a standardized curriculum around the DreamSpark project so there is consistency to how the platform is taught.

I personally would be more than willing to spend my time during the summer to put together a train the trainer workshop for those educators willing to participate.  This could be run at night so not to interfere with work duties. 

Already Lagging Behind

Another sad but true fact is educational institutions are years behind in terms of the technology being taught.  I know when I was in college this was true.  For example the language called Java was started in 1991 and released in 1994/1995.   While this was a newly released technology it wouldn’t be taught in the classroom until many years later.  Back then they were teaching us Pascal.  The progression worked something like this.  Once you mastered Pascal, which at the time was already considered a dead language, you could learn C, and then after that C++ and if you stayed around to get your PhD they’d eventually mention Java as an experimental language.  Basic skills are an absolute must have but what chance does a student have at actually getting a job today in a company that uses a technology like .Net where the student has never used it?  Very few companies have a need for engineers that are highly skilled in assembly and compilers.  However, a company would be willing to hire someone who knows how to write applications using Windows Presentation Foundation, Silverlight, Windows Communication Foundation and Workflow Foundation.  All of which have been released in the last year.  I truly feel sorry for students graduating today because the industry changes so quickly.  They need our help.

What Can We Do To Help

There are a lot of challenges that need to be overcome no doubt.  All of them are challenges, not road blocks though.  They can be fixed.  To restate some of the ideas above here are the things we (Microsoft, the community, educators) need to start with.

  1. The Community / Microsoft – Organize some type of event at educational institutions to help spread the word.
  2. The Community / Microsoft – Help train the current educators.
  3. Microsoft – Help the educators standardize a curriculum using existing training information, labs, etc.
  4. Microsoft – Put together a program whereby students build something using the tools and award them with free trips to TechEd or Mix or other conferences.


XBox With Blu-Ray, 2 Quad Processors, 2GB, HDMI, 7.1 Surround, 1080P DVD Upscaling, IPTV, Bluetooth, Cable Card, 500GB HDD, Home Server Integration and Home Automation

Posted by Keith Elder | Posted in XBOX 360 | Posted on 17-02-2008


Now that Blu-Ray has driven the nail in the coffin of HD DVD, Microsoft is about to release the ultimate console that will take over your living room for good.  The Blu-Ray / HD DVD format war has held them back from releasing a true home media machine that compliments other products in the line up like Windows Home Server.  Many may not know it but the format war put a wrinkle in Microsoft’s plans to own your living room.  This is why Microsoft chose to ship the Xbox 360 with a standard DVD player a few years ago.  Not knowing what the standard would be has held them back from truly embracing the potential of the XBox and XBox Live’s success.  That wait is now over.  Ladies and gentleman, I give you the XBox 1080.

XBox 1080 Specs


The specs of the 1080 are truly remarkable as it is a culmination of industry standards finally coming together in one solid machine.  The good news for gamers is nothing has changed in the architecture to make your current games obsolete, they only get more powerful.  The XBox 1080 has two quad core processors and 2GB’s of memory to bring a great 3D experience to games and also provide enough horse power to run other tasks in the background while playing the most intensive of games.  With all of these cores it is no wonder why the marketing reads “Have your cake and eat it too.”.   The XBox 1080 will truly rock your Guitar Hero / Halo 3 experience and provide developers more room to breathe allowing the XBox 1080 to devote processor cores to games while other ones are keeping an eye on your home (see home automation section). 


image At the heart of the graphics engine of the XBox 1080 lies NVidia’s latest acquisition, the PhysX Tech card by Ageia.  This newly added addition to the NVidia family allows for physics to be more realistic within games.  Ageia has already been apart of the Xbox, Wii and Playstation but more is to come.  New things are now possible that never were before like sheets flying in the wind, fully destructible buildings in first person shooters and more. 

Home Theater

The XBox 1080 ships complete with a Blu-Ray High Definition drive that supports standard CDs as well as standard DVDs.  Finally home owners can place their separate DVD players and Blu-Ray players on Ebay and consolidate their entertainment centers into one device to rule them all.  The XBox supports 1080P upscaling of traditional DVDs providing a stunning picture to home owners that waited out the Blu-Ray / HD DVD format wars by not supporting either side.  The 1080 also supports high definition video at 1080P (where it gets its name) using the Blu-Ray disc drive.  High definition without great sound would be show stopper for home theater enthusiasts but the XBox 1080 doesn’t spoil the fun.  It supports true 7.1 Dolby surround sound and can be auto configured to support a variety of options like THX for movies and 5.1 for games.  To carry the high def data the XBox 1080 comes with an HDMI output.  It also support component video and digital sound connectors.  For those of you that don’t have either option, sorry, the XBox 1080 can’t help you.  Microsoft wanted the machine to be completely cutting edge and that meant letting go of yesterday’s technology.  For some, it may be time to upgrade.

HD DVR and Media Center

image Besides replacing your old DVD and Blu-Ray player, the XBox 1080 can replace your legacy DVR or digital cable box.  There are two options the 1080 supports:

  1. Cable Card – Slide in a cable card from your cable provider into your XBox and turn your XBox 1080 into a 500GB HD DVR.  With the Cable Card the XBox also supports features like OnDemand from Comcast.
  2. IPTV – If you are fortunate enough to live in an area where you can get IPTV from a provider, just plug into your fiber network and configure the XBox for IPTV.  Within minutes you’ll be watching TV in 720P streamed right from the Internet.

The 1080 takes advantage of years of work that have gone into Media Center to bring a full blown High Def DVR into the home.

Home Server Integration

image Integrating with Windows Home Server has never been easier.  Simply point to your home server and start streaming videos, photos, music and much more directly from Home Server.  One of the best features about the home server integration is the ability to take advantage of your home server’s storage capacity.  While the XBox 1080 ships with a 500GB hard drive, it can quickly fill up, especially with high def movies and games either recorded in media center or ones rented/purchased from XBox Live. 

For movies you’ve recorded or downloads you’ve purchased that you want to keep simply select it in the menu and press “Send to home server”.  In the background the data is then moved to your home server (with DRM rights still intact).  The transfer and viewing of items from Home Server and the XBox 1080 is seamless.  For users that have purchased games or downloads from XBox Live, this solves one of the problems of having to repurchase data if you want to upgrade to a larger hard drive or in case your hard drive fails.  What better place to store data than on the server.  For people that are at friends houses and have purchased items from XBox Live that configured and synced home server with their Live accounts, any purchases made from your friends house will be automatically added to your home server.  This new upgrade to XBox Live allows friends to visit another friend who may not have the same add on or mod and still make it accessible as long as the person that owns the item is logged in.  The added bonus of synchronizing these downloads or purchases made from other XBox with home server will be a big win.

Home Automation

One of the most anticipated features of the XBox 1080 has been home automation.  By purchasing the $399 home automation kit you can now control your own home right from your XBox.  Home automation features automatic timers, security integration and monitoring, and much more.  See the website for more details about what is possible with the home automation feature.

With all of these features it is no wonder the XBox 1080 is one of the most anticipated console launches to date.