UPDATED on 30 June 2009

I spoke at the CodeStock 2009 conference and I thought it would be helpful for the attendees and others to be able to download my code and slides. The title of my presentation was "How I Learned to Love Metaprogramming" and it concerns Dynamic Language Runtime architecture, performance of dynamic typing and Python to C# integration. The slides and source code are linked below. I will be giving this talk again in September at the Charlottesville .NET User Group meeting. Both of the demos require C# 4.0 which is available in Visual Studio 2010.

• Demo One - shows how to do XML parsing using a fluent interface based on a DynamicObject derivation in C# 4.0
• Demo Two - shows how the Level 0, 1 and 2 CallSite and ActionBinder caches perform. UPDATED: I added a demo on 30 June 2009 that shows how the DLR 0.9 compares by invoking dynamic code through the DLR hosting APIs, thereby bypassing the CallSite caching mechanisms. The results are very instructive, showing that the DLR's polymorphic inline caching can yield a 250000% increase in performance. You read that correctly: a two hundrend fifty thousand percent increase in performance.

Slides in PDF (Acrobat) format (688.49 kb) 
Slides in PPTX (PowerPoint 2007) format (639.23 kb) 
Demo One Source Code - MetaObjectPlay200905.zip (5.17 kb) 
Demo Two Source Code - PythonIntegration200906.zip (5.33 kb)

I admit that I've been a geek for a good long while now. 30 years ago, I was taking computers apart and soldering new things into them to experiment or to force them to suit my needs. Those were fascinating times in the computer industry to be sure. Operating systems were simple enough to be understood in their entirety by a single person. Software applications were also very simplistic. Kids in 2009 would LOL at our total lack of sophistication but we were enthralled to be able to move circle and squares sloppily about the screen to simulate bombs or gnomes or whatever we could imagine. We had to use our imagination more in those early days and it wasn't a bad thing per se. Poor graphics and poor sound weren't an impediment to us because we didn't know that they were bad. But we knew they could be better and that kept us going.

At the halfway point on my journey thus far, I bought a Sony Magic Link PIC-1000 handheld computer. It was 1994. I think I paid about $800 for it at the time. I remember the excitement I felt as I opened the package and powered it up. It ran the Magic CAP operating system from General Magic and it was a thing of pure joy, I'll tell you. The screen sporting 16 shades of gray was organized into a virtual desktop complete with drawers that you could open and close by tapping them. You could drag objects into and out of the drawers. It had a filing cabinet, an on screen keyboard and a graphical toolbar of contextual tool icons at the bottom of the screen. The built-in modem would allow you to connect an e-mail or fax provider to exchange messages and documents with others.

I used my Sony Magic Link every day for a couple of years and it taught me a lot about the value of good software design. Some of the applications were clunky but, for the most part, it was a very functional device. Sony positioned the Magic Link as an upscale alternative to Apple's Newton. Having used both of them, I believed the Magic Link to be a far superior device for getting things done. The screen was bigger. It did decent animations. It had a built-in modem and a PCMCIA card slot. And the sound was good for the period. The Newton was cool but it was a glorified notepad, in my opinion. There's no doubt that Apple studied the Magic Link with great care to learn from it. As an iPhone user today, I often think about how similar my iPhone is, in so many ways, to the seminal PDA from Sony. Many of the Magic Link's gestures and graphical metaphors live on in the iPhone today, with or without attribution from Apple. Microsoft didn't seem to learn much from it though as Windows Mobile 6.5 still feels like a desktop OS crammed into a device not so well designed to run it. The truth hurts, I know.

I'm pretty sure that I still have that old Magic Link lying around in the attic somewhere. As I sit here in the White Hawk Music Café waiting for my son to complete his guitar lesson, I bristle with some of that old excitement I felt in 1994, wondering if I find that old device in the attic, will it still work? Will it be as fun today as it was then? Is there something I can solder into it to make it do something fun and new? Some things never change, I suppose. And that's good, too. Never stop learning. That goes for you, too, Mr. Ballmer. Watch and learn.

In part 1 of this article, I showed you how to create a fluent parser for XML documents as a dynamic type in C# 4. With that dynamic class, you can use the dot (.) and index ([]) operators to traverse any XML document with a very natural syntax. Being able to read XML fluently is great. But to be really useful, our dynamic class should allow the programmer to modify the XML document fluently as well. To do this, we'll need to modify the behavior of the overridden TryGetMember method to create missing XElements whenever they are referenced by the code. Here's the modified code for the TryGetMember method:

Notice near the bottom of the TryGetMember method that when the object named in the binder is not found, a new XElement is created and added as the first child of the current element. Then, a DynamicXml wrapper is created to continue the fluent chain to the next call as may be necessary. Since the specially handled "Value" member is used to get the value of a DynamicXml element, we also need to handle the "set" case for the times when the programmer attempts to assign a Value to an element. To do that, we need to override the TrySetMember method in the DynamicXml class like this:

With these two changes in place, we can write code to modify the underlying XML document, too. In the following code that exercises both of the code changes, notice that the first author of the second book has no middle name element in the source XML document. With the changes we've made to TryGetMember and TrySetMember, however, this isn't a problem at all. Look at the code that assigns the middle name value of "Lisa" to that author below.

When information about the first author of the second book is dumped to the console, we see that it contains the middle name that was assigned in the fluent mutation of the XML document:

---- Begin Author ----
EmailAddress='tfreefall@jump.com'
FirstName='Trudy'
MiddleName='Lisa'
LastName='Freefall'
----- End Author -----

Nice and easy, right? That's enough for this installment. In the next article in this series, I'll add some error handling code and deal with "the XML attribute problem" as it will come to be known. Enjoy.

As a follow-up to my last post on How SQL Server Sorts the UNIQUEIDENTIFIER Type, I thought it would be useful to have a function that would reorder the bytes of UNIQUEIDENTIFIERS whenever I need to show them in numerically correct order. Here's the User-Defined Function (UDF) I wrote to do this:

Invoking the UDF is easy. Here's a little script that demonstrates how to do it:

This outputs the following text in the Messages window of SQL Server Management Studio:

EBC23DE8-DC16-4A8B-8E11-6B1509B0DAED
6B1509B0-DAED-8E11-8B4A-16DCE83DC2EB

This matches the octet ordering that I showed in the previous blog post. Of course, if you attempt to create a new UNIQUEIDENTIFER in SQL using the reordered string, the new value will not be equal to the orginal. So be careful to use the reordered string only for reporting or whenever you need to see the UNIQUEIDENTIFIER values in numerically correct order. Enjoy.

When you read sorted UNIQUEIDENTIFIER values in the query results from Microsoft SQL Server, it's not immediately clear how they may be ordered. Alberto Ferrari wrote a blog post about this subject a while ago. However, I didn't find that Alberto's article highlighted the answer to my fundamental questions about UNIQUEIDENTIFIER sorting. Numbering the octets in the UNIQUEIDENTIFIER from left to right as 0..9 then A..F, what is the Most Significant Byte (MSB) to Least Significant Byte (LSB) ordering of the type from SQL Server's perspective?

You can really think of a UNIQUEIDENTIFIER like a really big integer: 128 bits or 16 bytes wide. However, when you look at the 32 hexadecimal digits of a UNIQUEIDENTIFIER expressed as a hyphenated string, it's not clear that SQL is treating this type like a number that we would read from left to right. For example, look at the following two UNIQUEIDENTIFIERs generated by SQL Server's NEWID() function:

• EBC23DE8-DC16-4A8B-8E11-6B1509B0DAED
• 4F899E16-9D3E-4EA6-8A32-749A3FCAD865

Do these look like large integers? Maybe if you took the hyphens out. If they were numbers, which of them would be the larger one? Reading left to right, anyone with a cursory understanding of hexadecimal notation might assume that the first value beginning with EB is larger than the one beginning with 4F. But that's not correct from SQL Server's perspective. The second one is the larger UNIQUEIDENTIFIER to SQL Server. To prove this, run the following query in SQL Server Management Studio:

WITH [UIDs] AS ( --                        0 1 2 3  4 5  6 7  8 9  A B C D E F
          SELECT [ID] = '0', [UID] = CAST('01000000-0000-0000-0000-000000000000' AS UNIQUEIDENTIFIER)
    UNION SELECT [ID] = '1', [UID] = CAST('00010000-0000-0000-0000-000000000000' AS UNIQUEIDENTIFIER)
    UNION SELECT [ID] = '2', [UID] = CAST('00000100-0000-0000-0000-000000000000' AS UNIQUEIDENTIFIER)
    UNION SELECT [ID] = '3', [UID] = CAST('00000001-0000-0000-0000-000000000000' AS UNIQUEIDENTIFIER)
    UNION SELECT [ID] = '4', [UID] = CAST('00000000-0100-0000-0000-000000000000' AS UNIQUEIDENTIFIER)
    UNION SELECT [ID] = '5', [UID] = CAST('00000000-0001-0000-0000-000000000000' AS UNIQUEIDENTIFIER)
    UNION SELECT [ID] = '6', [UID] = CAST('00000000-0000-0100-0000-000000000000' AS UNIQUEIDENTIFIER)
    UNION SELECT [ID] = '7', [UID] = CAST('00000000-0000-0001-0000-000000000000' AS UNIQUEIDENTIFIER)
    UNION SELECT [ID] = '8', [UID] = CAST('00000000-0000-0000-0100-000000000000' AS UNIQUEIDENTIFIER)
    UNION SELECT [ID] = '9', [UID] = CAST('00000000-0000-0000-0001-000000000000' AS UNIQUEIDENTIFIER)
    UNION SELECT [ID] = 'A', [UID] = CAST('00000000-0000-0000-0000-010000000000' AS UNIQUEIDENTIFIER)
    UNION SELECT [ID] = 'B', [UID] = CAST('00000000-0000-0000-0000-000100000000' AS UNIQUEIDENTIFIER)
    UNION SELECT [ID] = 'C', [UID] = CAST('00000000-0000-0000-0000-000001000000' AS UNIQUEIDENTIFIER)
    UNION SELECT [ID] = 'D', [UID] = CAST('00000000-0000-0000-0000-000000010000' AS UNIQUEIDENTIFIER)
    UNION SELECT [ID] = 'E', [UID] = CAST('00000000-0000-0000-0000-000000000100' AS UNIQUEIDENTIFIER)
    UNION SELECT [ID] = 'F', [UID] = CAST('00000000-0000-0000-0000-000000000001' AS UNIQUEIDENTIFIER)
)
SELECT [ID], [UID] FROM [UIDs] ORDER BY [UID] DESC

This query is based on the one that Alberto Ferrari showed in his blog. But this one goes after the answer that I'm interested in a bit more directly. The query creates a rowset that associates each distinct octet in UNIQUEIDENTIFIERs with an order identifier, 0..9 then A..F. Then, by ordering the results of the query by the the UNIQUEIDENTIFIER values, we should be able to tell which octets are more significant numerically than the others according to SQL Server. Here's what the results look like:

A 00000000-0000-0000-0000-010000000000
B 00000000-0000-0000-0000-000100000000
C 00000000-0000-0000-0000-000001000000
D 00000000-0000-0000-0000-000000010000
E 00000000-0000-0000-0000-000000000100
F 00000000-0000-0000-0000-000000000001
8 00000000-0000-0000-0100-000000000000
9 00000000-0000-0000-0001-000000000000
7 00000000-0000-0001-0000-000000000000
6 00000000-0000-0100-0000-000000000000
5 00000000-0001-0000-0000-000000000000
4 00000000-0100-0000-0000-000000000000
3 00000001-0000-0000-0000-000000000000
2 00000100-0000-0000-0000-000000000000
1 00010000-0000-0000-0000-000000000000
0 01000000-0000-0000-0000-000000000000

Do you see the pattern? Reading the IDs in the left column from top to bottom, we can see that the most significant octet is in the A position and the least significant one is at position 0. The pattern that represents the most significant to the least significant bytes reads as ABCDEF8976543210 which is not exactly what you might expect. Looking back at the new UNIQUEIDENTIFIER values shown above, now we know why EBC23DE8-DC16-4A8B-8E11-6B1509B0DAED is considered to be less than 4F899E16-9D3E-4EA6-8A32-749A3FCAD865 numerically by SQL Server. The 6B octet at position A in the first UNIQUEIDENTIFER is less than the 74 octet at the same position in the second value. In fact, if we were to reorder the all of the bytes in those two UNIQUEIDENTIFIERs left to right according to the way that SQL Server really "sees" them numerically, they would need to undergo these transformations:

• EBC23DE8-DC16-4A8B-8E11-6B1509B0DAED would be seen as 6B1509B0DAED-8E11-8B4A-16DC-E83DC2EB
• 4F899E16-9D3E-4EA6-8A32-749A3FCAD865 would be seen as 749A3FCAD865-8A32-A6E4-3E9D-169E894F

So, now we can read SQL Server UNIQUEIDENTIFIERs numerically. That will come in handy one day so tuck it into the back of your brain. Enjoy.