I presented a talk called "Enterprise Data Validation" at the Hampton Road .NET User Group this evening. The premise was simple. Data validation needs to happen in all the tiers of a modern application but the validation rules should be defined only once to avoid synchronization errors. In this talk, I showed how to extend SQL Server using extended properties to store regular expressions for data validation as column metadata. I also showed how to add a regular expression matcher to SQL Server using the SQL CLR and how to add check constraints to invoke the regular expression parser. Then I built a WCF service to query the validation metadata to make it available in other application tiers. I quickly assembled WCF service host and client showed how you could bring all of the elements together to create a working Enterprise Validation solution.

Download the SQL Scripts (20.06 kb)

Get the Slides

Attendees can Rate My Talk

I've been working with SQL Azure for some time now and I really like it from a technology perspective. For the uninitiated, SQL Azure, formerly SQL Data Services (SDS), is Microsoft's foray into relational databases in "the cloud". Microsoft runs a special build of SQL Server that operates in a highly-available configuration on the servers in its Azure data centers. From a system administrator's standpoint, there are some radical differences between SQL Server and SQL Azure. How in the world are we going to live without the BACKUP command or the KILL STATS JOB command, after all? When designing SQL Azure, Microsoft took a long look at the list of features that had grown into SQL Server over time and realized that there were a lot of physicality features that had become baked into the T-SQL language that make no sense whatsoever running in a grid type configuration with thousands of other active databases. In my mind, this is a good thing because it forces Microsoft to think critically about what pyhsical and logical assets really make up the database from an administrator's perspective and from a developer's perspective. Companies like Teradata and Netezza have been thinking this way for some time now in an effort to make their database appliances much simpler to manage. I sincerely hope that some of what Microsoft is learning with SQL Azure creeps back into SQL Server 2011 (or whatever it will be called). If so, it will be a great thing for Microsoft and its customers. In the short term, these tradeoffs will make traditional database and network administrators feel off balance, though.

Developers, on the other hand, will find SQL Azure quite comfortable. Much to my surprise, even my NHibernate-based applications work in SQL Azure without modification. One caveat there: If you use the NHibernate Hbm2Ddl utility as part of a Domain-Driven Design (DDD) process, just watch out for the fact that SQL Azure does not support heap tables. Because of this, every SQL Azure table must have a clustered index so make sure that all the tables in your model have a primary key or at least have a clustered index. (I know, I know. Primary keys don't have to be implemented as clustered indexes in SQL Server but you get the point.) A full list of SQL Server unsupported T-SQL can be found on MSDN. And while the list might look really long, the average application is not likely to encounter a lot of problems running against a SQL Azure database. The T-SQL statements that I'll personally miss the most in SQL Azure? Those would be OPENQUERY, OPENXML, SELECT INTO and NEWSEQUENTIALID.

Database Administration

Let's begin by looking at a bit of administration. The SQL Azure portal is the best place to start. You must log in with your Windows Live ID and click on the link for your project page. The screen for managing your SQL Azure databases is rather sparse. I was expecting to see something like SQL Server Management Studio on the web. Perhaps the web interface will evolve to support more features over time. For now, it presents something like this:

I've masked parts of the screen shot to protect my privacy but you get the idea. The section at the top shows basic information about you SQL Azure "server" with an option to reset the administrator password. The section below shows two tabs labeled "Databases" and "Firewall Settings". The databases tab shown above allows you to create databases, see the connection strings required to connect to them or to drop them. The list shows the current size and size limit for each database. As of this writing, SQL Azure limits databases to 1GB or 10 GB maximum sizes. Hopefully, that will change in the future to allow much larger databases to exist in the cloud. I mean, why build a mesh or grid infrastructure for massive database scaling and limit it to 10 gigabytes? The size of the 3 databases shows them to be zero bytes in size because I truncated them before writing this article. After clicking on one of the radio buttons beside a database name and clicking the "Connection Strings" button, you'll see an AJAX popup that looks something like this:

The popup shows what the ADO.NET and ODBC connections strings would look like in an application configuration file. Notice in both connection strings that there's nothing special about SQL Azure. We can use the plain, old SQL Server Native Client 10.0 over TCP/IP to connect to the Azure database. But can just anyone across the Internet connect to your data? Of course not. Microsoft allows you to restrict connections by IP address ranges or from Windows Azure tasks that you may be running in the MicrosoftServices cloud. The Firewall Settings tab on the main screen is where we can do that. The Firewall Settings screen looks something like this:

Again, I've masked out my IP address in the Record Name that I created called VZW shown here. Since I work on the go using my EVDO card a lot, I need to change the IP Address Range every day, sometimes several times a day. There's a way to change the firewall rules through DML but I've yet to try that. I was thinking of writing a Windows Azure or .NET Services service that would have access via the MicrosoftServices checkbox shown above. I could call the service with another form of authentication to have it update my SQL Azure firewall rules automatically. Until I write that, I'll have to use this web-based console interface to set up the firewall rules. One thing I've noticed during the CTP is that the firewall rules don't take effect immediately. When my IP address changes and I make a rule change in the SQL Azure Firewall Settings, it may take up to 10 minutes to push that change to the firewall and execute it. When I'm ready to make a change, I simply press the "Edit Record" button and an AJAX popup that looks like this is rendered:

Notice that the firewall rule for SQL Azure allows you to specify a range of IP addresses, not just one. That would be handy to use if, for example, all of the addresses within a Class C IPv4 Address block were allowed to connect to the SQL Azure databases you manage. I didn't see any support for IPv6 addresses in the Firewall Settings but I'm supposing that Microsoft will have to support IPv6 in the rules in the future. Additionally, I'd expect to see some richer firewall rules type, e.g. the use of subnet masks to further refine the grant or denial rules and rules based on IPSEC/VPN configuration. For now, IP ranges are enough to get started. I could show you the screens for creating and dropping databases in the web-based console but it's really not all that interesting. Besides, we can do that using the SQLCMD command-line tool as shown in the Database Access section below.

Database Access

I admit it. I'm a UNIX hacker from pre-history so whenever I have the chance to master something at the command line, I jump at it. So when I heard that SQL Azure worked well with the SQLCMD line tool, it brought a grin to my face. Here's a screen shot of a Windows Powershell-based exchange between me and my SQL Azure server. Once again, I've blanked out some of my personal information but this time, it's color coded to help you understand what's important.

The first thing to take note of is that I've issued 4 commands here, number 2# through 5#. Command 2# connects to the master database: notice the -d master parameter at the end of the 2# command line? The 1> prompt means SQLCMD is waiting for input from me. I typed "CREATE DATABASE Blog" followed by Return then "GO" followed by return. Until I provide the "GO" statement, all of my commands would be batched on the client side. When the "GO" command is received, it's not sent to SQL Azure. Instead, it's a signal to the SQLCMD client to send the current batch to the remote server. When the results returned from command 2# show no error, the 1> prompt shows to indicate the start of a new batch. The exit command takes me back to PowerShell.

At this point, our new Blog database has been created in the cloud and we're ready to use it. So in command 3#, I typed in a somewhat lengthy table definition. When I used the "GO" command to execute the batch, however, I got an error saying that the remote host (SQL Azure) has closed the connection. I took too long to type the command so SQL Azure, being a good steward of resources like IP connections, dropped the connection. The fact is that the connection was probably already dropped after I entered the "CREATE TABLE" command at prompt 1> and before I entered the "GO" directive at prompt 2>. But because SQLCMD was batching my commands, it didn't sense that the connection had been dropped until it tried to send the batch to SQL Azure. I haven't found a way to make SQL Azure keep the connection open longer and I probably would advise against using that feature if it exists. IP connections are precious to any kind of server that needs to scale to large number of users. Forcing SQL Azure to keep the connection open longer so you can type is just a bad idea because it would severely impede scalability.

Fortunately, I don't have to type that long "CREATE TABLE" command in again. When I ran SQLCMD again in step 4#, I simply pressed the up arrow on my keyboard and it "remembered" the command. Tapping return and issuing the "GO" command, the table is successfully created in the new Blog database. In the same session, I then started to type an "INSERT" command to put some data into the new Article table. Again, I took too long to do it so you can see that the connection was again closed by SQL Azure before committing the batch. No worries, though. Rerunning SQLCMD and using the up arrow trick saved the day again. The values were inserted successfully on the second attempt.

Now let's talk a bit about those masked out values that I color-coded in the screen shot. The yellow masks are where my user ID is inserted. You can create users and logins using the "CREATE USER" and "CREATE LOGIN" commands in SQL Azure just as you can in SQL Server. Once you've done that, you can use the user IDs with access to a given database to do your work. The @ sign trailing the user ID in each SQLCMD is significant. For whatever reason, you must connect to SQL Azure with the user ID (-U) qualified at (@) your server name. The server name shows in the pink or salmon masks. Also notice that the pink/salmon masks show up again in each command in the server name (-S) section. Just remember to specify your SQL Azure server name after the @ in the -U parameter and again in the -S parameter. Also, the -S parameter must contain the Fully Qualified Domain Name (FQDN) of the server so that the SQLCMD tool can resolve to your SQL Azure server's IP address. Now, let's query the data.

Notice that in command 10# that I used SQLCMD's -q parameter to pass a query string to my SQL Azure Blog database. In this case, it's a SELECT statement that dumps the data that I inserted with command 4# earlier. The output format isn't so pretty but you can tell that the data matches what I inserted before. Since I used the -q (lowercase) parameter, I still have to use the "exit" command to leave the SQLCMD interpreter. In command 11#, I used the -q parameter again to DROP the Article table from the Blog database. And in command 12#, I used the DROP DATABASE statement to drop the Blog database from SQL Azure altogether. In this case, I had to specify the master database using SQLCMD's -d parameter. Also note that by using the -Q (uppercase) parameter in command 12#, the exit statement is implied so I didn't have to exit manually as I did in commands 10# and 11#. That's handy.

Coding to SQL Azure

If you are using ADO.NET or ODBC, the connection strings to your SQL Azure database can be obtained from the SQL Azure web console as shown earlier. Let's take a moment to dissect the ADO.NET connection string while we're on the subject. I'll only address the parts that need some special attention below:

Server=tcp:<server name>.database.windows.net; Database=<database name>; User ID=<user name>; Password=<password>; Trusted_Connection=false;

• Server=tcp:<server name>.database.windows.net - this is the fully qualified domain name of your server, prefixed with the tcp: directive. This tells the SQL Server Native Client to use the TCP/IP protocol to connect to the FQDN that you specify. If your client is configured to prefer named pipes or some protocol over TCP/IP, the tcp: directive in the connection string tells it to skip directly to TCP/IP instead.
• User ID=<user name> - unlike SQLCMD, when using ADO.NET, the user name does not have to include @<server name> as the suffix. Just the user name part will do.
• Trusted_Connection=false - this may not be what you think. This directive doesn't mean that the connection won't be secure. Every SQL Azure Tabular Data Stream (TDS) connection is tunnelled through the Secure Sockets Layer (SSL). Instead, this directive means that we won't be using OS-driven authentication like NTLM or Kerberos.

You can construct a connection string in C# quite simply by using the SqlConnectionStringBuilder class as follows:

Notice that the Encrypt property is set to true in the connection string builder. This isn't strictly required because SQL Azure will force this value to true even if the client does not specify it. You should also note that SQL Azure does not accept connections on any TCP port other than 1433 at this time. So don't try to use a different port in the connection string builder or the connections using it will fail. When you're ready to use the SqlConnectionStringBuilder, invoke the ToString() method to get the full connection string back for use in your code. I'd show you some ADO.NET code here to do INSERT, UPDATE and DELETE but, to be honest, it would be pretty boring. Your ADO.NET code most likely won't have to be changed when moving from SQL Server to SQL Azure.

Product Availability

As of this writing, SQL Azure is still in CTP (Community Technology Preview) and not available for commercial use. The Microsoft Professional Developer Conference (PDC) coming up in mid-November 2009 is the time that's expected for commercial launch of the product. Right now, it appears as though Microsoft is going to limit databases in the SQL Azure cloud to 1GB or 10GB, so many larger-scale commercial applications may have to wait for a time when 100GB or larger databases may be ported. There's no guarantee that will ever happen but one has to assume that Microsoft, once it has gotten some commercial experience serving real customers, will open SQL Azure up to databases that can really show its capabilities.

Pricing

Check Microsoft's SQL Azure Pricing information page for details about cost and measurement.

Closing Thoughts

Microsoft's first attempt at putting SQL Server into the cloud is fairly impressive. And although SQL Server Management Studio can be used with SQL Azure, there are some known compatibility issues that make using command line tools safer for the time being. I'm betting on the fact that Microsoft will make some rich, GUI-based management tools available in due time. After all, Microsoft differentiated itself in the database space years ago by making network and database administrator jobs much easier through the use of great tools. Why wouldn't they continue that trend with SQL Azure? With respect to the query engine and the storage engine in SQL Azure, this first release is fairly strong. The fact that my NHibernate-based applications run without modification is impressive to say the least. If you're accustomed to writing lots of rich stored procedures that use every trick in SQL Server 2008's book, you may encounter some problems when using SQL Azure, though. There are many subtle changes and omissions in the implementation concerning those features that many of us consider to be on the periphery.

Will SQL Azure be a hit in the marketplace? Who knows? That's the big question now. There's little uptake on relational cloud databases in general so it remains to be seen if the popularity of SQL Server will translate into the cloud well. That will have a lot to do with pricing and Microsoft's target market which isn't fully understood just yet. Imagine a medium-sized company that would have to pay for server hardware and SQL Server 2008 Standard Edition plus the Client Access Licenses to make the system available. Then there are the environmental factors like power and cooling to consider. There's also hardware and software maintenance to add in and the people to manage it all at three nines of uptime per month. What's that worth per year? If Microsoft can convince business managers that it's a safe thing to do, that the development experience is great and that the pricing's right, SQL Azure could be quite popular in the marketplace. Only time will tell. Personally, I'm already thinking of clients who could benefit by shedding their servers in favor of cloud databases. I'm definitely going to start small, though, and work my way up. My clients who are spending between $2,000 and $3,500 US per server (Total Cost of Ownership) with less than 10GB of storage are the ones who could benefit the most by considering the move to SQL Azure.

When iterating over a sequence of numbers in Python, the range() function is commonly used. However, the implementation of the range() function in Python 2.x instantiates each element in the sequence before the iteration begins. This is really costly from both memory and CPU perspectives when the desired range of numbers is large. Consider using the xrange() function instead which implements a Python generator to yield each number in the sequence as needed. Using xrange() instead of range() for large iterations can have a big, positive impact on your code. For example, in an application I was working on recently, replacing range() calls with xrange() boosted my performance from ~900,000 transactions per second to over 3,000,000. In Python 3.x, the range() function is supposed to be implemented as a generator but I haven't tested that to be true yet. Let me know if you have.

I spoke at the Charlottesville .NET User Group this week and at the Raleigh Code Camp. I cheated and did the same presentation to both groups. Call me lazy but, in the middle of planning our own Code Camp in Richmond, I really didn't want to prepare two separate talks. I did a talk back at CodeStock 2009 on a similar topic back in June 2009 but it's evolved a lot since then based on my own growth and understanding. You can find the code and slides below.

This talk takes about an hour and covers the basic concepts of metaprogramming and how the DynamicMetaObject class, the IDynamicMetaObjectProvider interface and the DynamicObject helper class work in the .NET Framework 4.0. The talk is very demo-centric, focusing on the code that support the concepts outlined in a few slides. The first demo focuses on helping you to understand the dispatch mechanism in the DLR and how you can implement your own classes that participate in the call binding process. It's a fairly simple fluent XML parser implemented using the DynamicObject helper class. While it's simplistic, it helps to introduce developers who are new to the DLR to the concepts of dispatch and binding in C# before diving into things Pythonic.

The second demo focuses on performance measurement. Having learned about the delegate and rule caching that goes on in the DLR's CallSite<T> and ActionBinder classes in the presentation, these tests show the effect of all that awesome work that Microsoft has done for us to enhance the performance of dynamic code. The result is that we see the caching yields about 5,000 times the performance of traditional static to dynamic language interoperability. Rather than the hundreds of transactions per second that we are accustomed to when doing dynamic language invocation from a context like C#, we now get millions of transactions per second. To put it all in context, the same transactions performed in pure C# are still about 4 times as fast as the DLR with delegate and rule caching. However, you have to admit that what the DLR provides today is pretty amazing. For many, this level of performance makes the use of languages like Python for scripting a large application feasible for the first time.

The question I most often get about this is, "When will the dynamic language performance be just as good as 'real' .NET code?" OK, so I take exception to the term 'real' there but I know what you all mean. I don't know the answer and it doesn't really matter that Ruby and Python performance be just as good as C#. Different languages have their own strengths and sometimes that includes performance. But when pure C# generates 6 million transactions per second, traditional thunking across to dynamic languages performs at a rate of several hundred per second, the fact that adaptive Inline Caching in the DLR helps me to yield close to 2 million transactions per second is pretty darned good. We're well on the way to my other favorite language, Python, becoming a first-class citizen in the .NET family. It takes education, of course, for people to know just how wonderful the DLR is which is the real purpose of my evangelism. I just love the DLR, if you can't tell already.

I am doing a Channel 9 geekSpeak next week on this topic and I'll refer to some of this material. And my talk at the October 2009 NoVA Code Camp will use an even further evolved version of this talk. Drop me a note and let me know what you think about this topic. Do you have success stories? Horror stories regarding dynamic language integration?

Slides for DLR Performance Talk September 2009 [PPTX] (184 kB)

Slides for DLR Performance Talk September 2009 [PDF] (680 kB)

Code for DLR Performance talk September 2009 [Updated 21 Sept 2009] (16 kB)

UPDATED on 30 June 2009

I spoke at the CodeStock 2009 conference and I thought it would be helpful for the attendees and others to be able to download my code and slides. The title of my presentation was "How I Learned to Love Metaprogramming" and it concerns Dynamic Language Runtime architecture, performance of dynamic typing and Python to C# integration. The slides and source code are linked below. I will be giving this talk again in September at the Charlottesville .NET User Group meeting. Both of the demos require C# 4.0 which is available in Visual Studio 2010.

• Demo One - shows how to do XML parsing using a fluent interface based on a DynamicObject derivation in C# 4.0
• Demo Two - shows how the Level 0, 1 and 2 CallSite and ActionBinder caches perform. UPDATED: I added a demo on 30 June 2009 that shows how the DLR 0.9 compares by invoking dynamic code through the DLR hosting APIs, thereby bypassing the CallSite caching mechanisms. The results are very instructive, showing that the DLR's polymorphic inline caching can yield a 250000% increase in performance. You read that correctly: a two hundrend fifty thousand percent increase in performance.

Slides in PDF (Acrobat) format (688.49 kb) 
Slides in PPTX (PowerPoint 2007) format (639.23 kb) 
Demo One Source Code - MetaObjectPlay200905.zip (5.17 kb) 
Demo Two Source Code - PythonIntegration200906.zip (5.33 kb)

Welcome to Part 4 of a multi-part series on JavaScript Object Notation (JSON) support in the Microsoft .NET Framework. In this article, we'll focus on the JavaScriptConverter class from the System.Web.Script.Serialization namespace. You can find the other parts of this series at these locations:

• Part 1 - An exploration of the DataContractJsonSerializer class
• Part 2 - An exploration of the JavaScriptSerializer class
• Part 3 - JSON serialization from WCF (including a REST primer)
• Part 4 - Using the JavaScriptConverter class to customize JSON serialization

Welcome back. We've been exploring the differences in .NET's two competing JSON serialization classes for a while now. This time, let's dig into ways that we can customize the JavaScriptSerializer to solve some difficult problems. In some circumstances, the services provided by the JavaScriptSerializer for dehydrating and rehydrating of POCOs just isn't sufficient. As discussed in Part 2 of this series, the JavaScriptSerializer has the interesting capability of handling plain .NET classes that aren't marked up with serialization-related metadata in any way. Most of the competing serialization and deserialization classes in the .NET Framework Class Library require some kind of meadata, typically in the form of attributes, to assist the serializer in doing it's job. And there's good reason for this. Sometimes, simply iterating over the properties and fields of an object doesn't provide enough information to properly put it into a format that's suitable for storage or for transmission. For example, look at the following class definition which uses two properties to expose a single DateTime object in different formats:

When an instance of this class is serialized to JSON using the JavaScriptSerializer, it may look something like this:

{"AsRFC1123":"Sun, 17 May 2009 01:00:00 GMT","AsSortable":"2009-05-17T01:00:00"}

As you can see, the serializer decided to include both string properties, AsRFC1123 and AsSortable which is redundant. There's no need to include the DateTime information twice. However, the JavaScriptSerializer can't know what should and should not be serialized because the definition of the Redundant class has no markup to give it clues about what should be included. The JavaScriptSerializer doesn't depend on attributes as some other .NET serializers do. Instead, if you want to customize the serialization process when using the JavaScriptSerializer, you must provide an instance of a class derived from JavaScriptConverter. For the Redundant class, the following converter can be used to fix the problem:

First of all, notice that there are 3 members from the abstract base class that need to be overridden. These are:

• IEnumerable<Type> SupportedTypes { get; }
  
• IDictionary<string, object> Serialize( object obj, JavaScriptSerializer serializer );
• object Deserialize( IDictionary<string, object> map, Type type, JavaScriptSerializer serializer );

The SupportedTypes property tell the serializer which types the converter supports. In this case, only the Redundant class is handled. The Serialize method accepts an instance of the invoking serializer and the object to be converted into JSON. However, the converter doesn't actually produce JSON output. Instead, the Serialize method on the converter must return a dictionary that maps strings, the names of the members, to objects, the actual objects that will be included. In the RedundantConverter class, you can see that were only including one property to be serialized named "ts". Remember, the whole point of attaching custom serialization is because the AsRFC1123 and AsSortable are redundant. We don't need both of them. And since, we'll only be including one of them (AsSortable), we took the opportunity to shorten the name to "ts". This will make the resulting JSON more compact which is almost always a good idea. The Deserialize method works in reverse, accepting a dictionary of name value pairs and constructing a new Redundant object from it. In this case, the Type parameter of the Deserialize method isn't used because this converter only assists with the Redundant type. However, if multiple types were converted by this class, you could include a switch statement to instantiate the appropriate type based on the Type parameter. Here's a bit of code that registers a RedundantConverter and tests the serialization and deserialization processes:

The output of this code will look something like this:

Redundant serialized without conversion support:
"{"AsRFC1123":"Sun, 17 May 2009 01:00:00 GMT",
"AsSortable":"2009-05-17T01:06:21"}"

Redundant serialized with conversion support:
"{"ts":"2009-05-17T01:00:00"}"

Redundant deserialized with conversion support:
AsRFC1123 = "Sun, 17 May 2009 01:00:00 GMT",
AsSortable = "2009-05-17T01:00:00"

In closing, let me make one more key observation. Serialization requires that you can get the property values from an object. Deserialization requires that you can set the property values of an object. But notice that the Redundant class has no mutators (set handlers) on its properties. However, since we're doing custom conversion using the RedundantConverter class, that doesn't matter. The converter's Deserialize method simply uses a custom constructor to inject the DateTime value rather than using properties to do it. Yet another really cool thing that the JavaScriptSerializer can do that baffles the average competing serializer class.

That's all for now. Join me next time as we look at how to consume JSON-encoded content in a Silverlight control. Enjoy!

Welcome to Part 3 of a multi-part series on JavaScript Object Notation (JSON) support in the Microsoft .NET Framework. In this article, we'll focus on performing JSON serialization from a RESTful WCF web service. You can find the other parts of this series at these locations:

• Part 1 - An exploration of the DataContractJsonSerializer class
• Part 2 - An exploration of the JavaScriptSerializer class
• Part 3 - JSON serialization from WCF (including a REST primer)
• Part 4 - Using the JavaScriptConverter class to customize JSON serialization

Welcome back. In Part 1 of this series, we looked at using the DataContractJsonSerializer to dehydrate and rehydrate .NET classes to and from JavaScript Object Notation (JSON), respectively. In Part 2, we looked at the once rogue JavaScriptSerializer class which has the unique capability of being able to serialize Plain Old CLR Objects (POCO) that aren't marked as [Serializable] or [DataContract]. In this article, we'll focus on something a bit less academic: building a RESTful web service with Windows Communication Foundation (WCF) that delivers JSON encoded objects to its clients.

A Representational State Transfer (REST) Primer

Let's do a little rundown of what Representational State Transfer (REST) means. Like any architectural pattern, REST has some tenets or guidelines for describing what it hopes to accomplish and, in some sense, how it prefers for the work to be done. The REST principles¹ for application development are:

1. Application state and functionality are abstracted into resources
2. Every resource is uniquely addressable using a universal syntax for use in hypermedia links
3. All resources share a uniform interface for the transfer of state between client and resource, consisting of:
   • A constrained set of well-defined operations
   • A constrained set of content types, optionally supporting code on demand
4. A protocol which is client-server, stateless, cacheable and layered

We don't need to cover all of these but there are a couple of high points to take note of. Most people interpret the phrase universal syntax in the second principle to mean that URLs are to be used to address application resources. And that's usually true. Here's the resource identifier for my Twitter.com timeline, for example:

    http://twitter.com/statuses/user_timeline/KevinHazzard.xml

If you click that link, you'll get an XML document that contains the state of my status updates to Twitter.com. The term for this kind of output is Plain Old XML or POX for short. If you change the extension of the resource identifier from xml to json in the URL shown above, what do you think you'll get? That's right. You'll see my Twitter.com timeline in JSON format. Pretty cool, huh? You can learn more about the RESTful Twitter Application Programming Interface (API) if you're interested.

The key word in that second principle is uniquely. What it means is that the resource identifier for the application state of my user timeline on Twitter.com must be different from all the other resource identifiers. Perhaps it's stating the obvious but from an addressing standpoint, REST tends to refer to application state and functionality like a file system. Bits of application state and the functions that an application performs are like a set of files. If you open a file by a well-known name, you'll get the functionality or state that's inside the file. This, of course, helps to define what the first and third REST principles mean, too. In the RESTful URL above, the well-defined operation is not KevinHazzard.xml. That's an argument or parameter, if you will. The operation is user_timeline and, reading left to right, you might guess that it's inside a module called statuses. It doesn't mean that the Ruby code that Twitter.com uses to implement the service is organized that way. But the resources in that class have a constraint that they use to provide a set of uniquely addressable resources.

The sub-bullets of the third REST principle are really interesting to me. One of the complaints about the Simple Object Access Protocol (SOAP) that some web services use is that it's much too rigid to be useful for web application development. Web developers complain that they don't have a Software Development Life Cycle (SDLC) that allows them to digest Web Service Description Language (WSDL) contracts, generate proxy classes and re-compile their code to use the well-defined operations and content types that a web service might provide. But I see through the compaints and the dissatisfaction with SOAP and the SDLC it engenders. The heart of the complaints really have more to do with the fact that great tools for building SOAP clients are rare. Case in point: over the years, I've demonstrated various ways to call SOAP services from environments that don't have great web service support built in. In an article I wrote about invoking SOAP web service from a dynamic language like Windows PowerShell, I demonstrated that WSDL contracts can be consumed at runtime to create and call proxies to SOAP services on the fly. To do that, I had to write a class called WSDLHelper that performs dynamic code generation. Not simple stuff but it works beautifully, making access to SOAP-based services just as easy as calling RESTful services from PowerShell or Python, for example. So, it's not that SOAP is bad, per se. But the tools to make SOAP client development aren't all that great, even in some .NET contexts.

Those sub-bullets of the third REST principle speak to the desire to constrain and define the operations and the data types that flow to and from the service. Without some contraints, the experience of using a RESTful service would be chaotic and unpredictable. Services like Twitter.com have very detailed definitions for their RESTful resources outlined in the Twitter API. It defines the resources (operation contracts), the parameters, the authentication mechanism and the output schema (data contracts) for each resource. Well, that's no different than what SOAP and WSDL are attempting to describe. The fundamental difference, though, is that SOAP and WSDL have rigid and quite rich XML schema definitions for all of that, not just the output of each resource. So, when people say that they dislike SOAP because it's too complex or bulky, look through the veil and understand that they really only dislike parsing WSDL, generating proxies, recompiling code, etc. It's OK to dislike those things, but now that you've read my blog, you understand that if better SOAP client tools existed on more platforms, it would often be preferred due to the richness and type safety that it can lend to the design process.

RESTful WCF

OK, enough about REST principles. Let's look at what it takes to expose a RESTful service in WCF. There are a handful of simple steps you can follow to do this. Assuming that you're going to be using Internet Information Server (IIS) to expose your service, follow these steps:

1. Add a reference to System.ServiceModel.Web to your web application. This assembly includes the implementations of the WebServiceHostFactory and the WebGetAttribute that you'll be using in the next steps.
2. In your Web.config file, modify an existing endpoint or add a new endpoint that uses the webHttpBinding binding type. This binding tells WCF that we're going to use HTTP URLs to parse resource identifiers and parameters. It also makes it possible to use RESTful response formats like POX or JSON instead of generating SOAP envelopes to go back to the client.
3. In the SVC file for the WCF service, modify the <@ ServiceHost> element to include the attribute Factory="System.ServiceModel.Activation.WebServiceHostFactory". This factory class provides instances of the WebServiceHost class to handle the parsing of incoming RESTful URLs.
4. The last thing you need to do is mark each [OperationContract] that you want to act as a REST resource with the [WebGetAttribute] or the [WebInvokeAttribute]. This is where we'll provide a template that maps incoming URLs to specific operations. These attributes also allow us to map parameters from the URLs to each operation's parameters.

Here's some example code and configuration for my ThingySearchService. Let's start with the [DataContract] and the [ServiceContract]:

First of all, notice that the [DataContract] attribute is explicitly applied to the Thingy class and that each property is marked with a [DataMember] attribute. The serializers used by WCF to render POX and JSON require this. You could alternatively mark the Thingy class with the [Serializable] attribute but as we saw in Part 1 of this series, the DataContractJsonSerializer often does a better job when the [DataContract] attribute is used instead. The UriTemplate of the [WebGet] attribute tells the WebHostFactory that when URLs referencing the FindThingy resource identifier are encountered, they should be routed here to the FindThingy operation. In addition, the UriTemplate instructs the WebHostFactory how to map the parameters in the URL to the underlying method's parameters. For my ThingySearchService, the following URL:

    http://localhost:51976/ThingySearch.svc/FindThingy?name=Kevin&age=44

might render this XML:

Considering the title of this article, you may be wondering why I've gone to so much trouble to produce POX from my RESTful service instead of JSON. Of course, the answer is that there's a nice story to tell here. To switch from a POX response format to a JSON format, you only need to modify the [WebGet] attribute on your [OperationContract] methods, adding a ResponseFormat parameter.

Now, when I invoke my ThingySearchService, I get a JSON-encoded string containing the text '{"Age":44,"ID":1682,"Name":"Kevin"}' which can be cleanly and quickly deserialized into an object on a variety of browsers and platforms. That was a roundabout way of showing you how to build REST/JSON web services with WCF. Hopefully you found the background information helpful. In the next segment, I'll be looking into how you can exercise finer control over the JSON serialization process.

Footnotes:
¹ From http://en.wikipedia.org/wiki/REST

 

Welcome to Part 2 of a multi-part series on JavaScript Object Notation (JSON) support in the Microsoft .NET Framework. In this article, we'll focus on the very handy JavaScriptSerializer class. You can find the other parts of this series at these locations:

• Part 1 - An exploration of the DataContractJsonSerializer class
• Part 2 - An exploration of the JavaScriptSerializer class
• Part 3 - JSON serialization from WCF (including a REST primer)
• Part 4 - Using the JavaScriptConverter class to customize JSON serialization

Welcome back. In Part 1 of this series, we looked at using the DataContractJsonSerializer to dehydrate and rehydrate .NET classes into JavaScript Object Notation, also known as JSON format. In this second article in the series, we'll look at a competing class in the .NET Framework called the JavaScriptSerializer. This class, which is in the System.Web.Script.Serialization namespace and can be found in the System.Web.Extensions.dll assembly, has a somewhat checkered past. It first appeared as part of the AJAX Extensions v1.0 for ASP.NET 2.0. However, by the time the class had made its way into the .NET Framework distribution in version 3.5, it had been marked as obsolete. This meant that while the class was available for use in the current framework of the time, it was likely to disappear altogether in a future release. When we examine the class together below, you'll understand why many developers were upset by the decision to deprecate this class in favor of the WCF-oriented DataContractJsonSerializer.

The DataContractJsonSerializer has one major drawback from the vantage point of many AJAX developers.  Classes that are to be serialized or deserialized must be marked with the [Serializable] attribute or the [DataContract] attribute. In addition, when using the [DataContract] attribute, each member of the class that is to be serialized or deserialized must be marked with the [DataMember] attribute. Whew! That's a lot of markup for something that should be very simple. The JavaScriptSerializer has no such restriction. And what if you want to serialize a compiled class that wasn't marked with the correct attributes at compile time? The DataContractJsonSerializer, being somewhat formal in its approach to satisfying the tenets of Service-Oriented Archicture that Microsoft espouses, makes this scenario pretty much impossible. The JavaScriptSerializer was evidently written by folks on the AJAX Extensions team who felt as my friend Harry Pierson does when he says that it's time to Retire the Tenets. I agree with Harry one hundred percent. Those SOA tenets were developed in a vacuum, in a lab. The real world is messier and dirtier than anything that happens in a lab. Things move fast in the real world and sometimes we just need our specifications to be looser so that we don't get hurt by the impacts that occur naturally in design, development and in production.

These two classes, the JavaScriptSerializer and the DataContractJsonSerializer, put the debate over the tenets into stark relief. Thankfully, Microsoft heard the cry of the AJAXers and un-obsoleted the JavaScriptSerializer in the SP1 release of the .NET Framework 3.5. Thank you, Microsoft! It just goes to show you: if you make enough noise, our friends at Microsoft will respond positively and give us what we need. Of course, we should be polite and professional in our dissent. The pen is truly mightier than the sword. Let's take a look at the JavaScriptSerializer now. For now, this class supports two instance methods of interest:

• Serialize( object ) : string
• Deserialize<T>( string ) : T
  

Given any object, the Serialize method will convert it into a string of JSON text. Conversely, given a type paramter T and a string of JSON text, the Deserialize method will return a new instance of type T. You can think of the Deserialize method like a constructor or an object factory: the raw material of JSON text goes in, an object comes out. Let's start toying with the JavaScriptSerializer by using a provocatively-named class that looks like this:

Notice that the aptly-named Unserializable class has no attributes, base classes or interface implementations that would make it serializable using the half dozen or so patterns that .NET has evolved over the years for such purposes. However, even though the Unserializable class doesn't appear to be serialization-friendly, the JavaScriptSerializer has no problems handling it. Check out the Dehydrate and Rehydrate methods in this code:

In the Main method, serializing and deserializing the Unserializable class doesn't seem to be a problem. OK, so maybe the Unserializable class wasn't so aptly named after all. It appears that the very handy JavaScriptSerializer class does something that is extraordinary when compared to all of the other .NET Framework object serialization techniques. This little class appears to be able to serialize and deserialize Plain Old CLR Objects (POCO) without a fuss. So simple! And simple is good. I think I like this little class. I'll talk more about this class and the DataContractJsonSerializer in my next article. That's all for now. Enjoy.

Welcome to Part 1 of a multi-part series on JavaScript Object Notation (JSON) support in the Microsoft .NET Framework. In this article, we'll focus on WCF's DataContractJsonSerializer. You can find the other parts of this series at these locations:

• Part 1 - An exploration of the DataContractJsonSerializer class
• Part 2 - An exploration of the JavaScriptSerializer class
• Part 3 - JSON serialization from WCF (including a REST primer)
• Part 4 - Using the JavaScriptConverter class to customize JSON serialization

This is the beginning of a series in which I'll explore support for JavaScript Object Notation (JSON) in the Microsoft .NET Framework. In this part, I'll be using the DataContractJsonSerializer. You can use the DataContractJsonSerializer to stream any serializable POCO into its JSON representation. Of course, this is handy for working with AJAX controls but you aren't limited to using JSON in a web context at all. Look at the following simple class definition in C#:

Classes that will be serialized or deserialized with the DataContractJsonSerializer in .NET must be marked as [Serializable] or as a [DataContract]. Code that creates an instance of this class and serializes it to JSON using the WriteObject method of the DataContractJsonSerializer might look like this:

would yield the following text on the console window:

{
   "<Age>k__BackingField":44,
   "<ID>k__BackingField":1,
   "<Name>k__BackingField":"Kevin"
}

Simple enough. Notice that JSON doesn't qualify any of the data by type. You can see that the names of the automatic properties are odd-looking, too. This is because the C# compiler names the backing fields for automatic properties in a way that they are guaranteed not to clash with the names you can create but it also makes them ugly and hard to read. Let's mark the Thingy class as a DataContract instead to see what effect that has on the output of our little program.

If you run the small program above against the DataContract-oriented Thingy class, you now get this much cleaner-looking output:

{
   "Age":44,
   "ID":1,
   "Name":"Kevin"
}

That's much better, don't you think? It's also smaller which will make tranmission time faster, too. OK, now let's focus on deserialization. Of course, you could deserialize the MemoryStream containing the JSON text into a new Thingy instance using the ReadObject method of the DataContractJsonSerializer. But that's no fun at all. We should do something a bit more interesting with our time. Since JSON doesn't qualify the types of the properties, we should be able to deserialize the JSON text into a different class that has properties with compatible names. But does order matter? And does it matter if the JSON text contains data for properties that don't exist in the target type? Let's test those ideas. Look at the following class definition for an OtherThingy class:

The OtherThingy class has an Age property but it's a floating point type, unlike the integer Age property in the Thingy class. Also notice that the Age property of the OtherThingy class is specified before the Name property reading top to bottom. Finally, note that the OtherThingy class contains no ID property at all. Will the serialized Thingy JSON text deserialize correctly into a new OtherThingy instance? Here's a program that tests these ideas:

The output shows:

Serializing Thingy from:
(String) Name = Kevin
(Int32) ID = 1
(Int32) Age = 44

Deserialized OtherThingy as:
(Single) Age = 44
(String) Name = Kevin

Coolness! It behaved as I had hoped. We'll have more fun with JSON serialization in the next installment.

The more I study the Dynamic Language Runtime (DLR), the more I am convinced that it deserves to be the centerpiece of Microsoft's .NET 4.0 strategy. I am a statically typed languages guy. Or at least I have been. I love the robustness that the compiler applies to certain preconditions for using objects. Type safety is great but I don't think that statically typed languages go far enough to be honest. I am excited about Spec# because of its supports for invariants, pre and post conditions, static program verification, etc. Will Spec# ever go mainstream? It doesn't appear to have the momentum that F# or IronPython have, to be frank about it. But maybe we'll see some ideas creep into C# from the Spec# research that Microsoft is doing. I have my fingers crossed.

On the other end of the spectrum is IronPython 2.0, Microsoft's first DLR-based implementation of the Python programming language. I just love Python. It's so clean and simple that almost anyone can learn and use it effectively after studying it for an hour or two. If I could send a note to myself in the past to change one thing about my career (besides the winning lottery numbers for the past decade) it would be to encourage my younger self to learn Python back in the mid-1990s. I knew about the language back then but I was a C++ bigot. I wish I had been more open minded. So much of my approach to software development would have been better if I had learned Python back then. I was working in the Intel Architecture Labs and had the perfect opportunity to do so, too. We were helping a very young company named Yahoo.com to evaluate e-mail providers to purchase. We ended up recommending RocketMail technology which was based almost entirely on server-side Python scripting. I had the perfect opportunity to learn Python from real experts right there in front of me and I let it slip right by me.

Well, no sense living in the past, right? I know Python really well now and it has changed the way I think about software development. Learning Python made me fall in love with Ruby and LISP and F#, too. I'm a real polyglot now and being able to understand how these other languages solve problems helps me design better in the language I use every day: C#. By the way, the term polyglot from its Greek roots literally means many tongues. I'm excited about the DLR because it's enabling Python and Ruby on my favorite platform. But the DLR is so much more important than that. It's the Language of Languages. The DLR defines the primitives that must exist on the boundaries of our languages to make them interoperate. In the same way that COM unified the dispatch model in the 1990s through IUnknown and IDispatch, the DLR will define the call semantics and dispatch models for the .NET Framework for the 2010s through IDynamicObject and DynamicMetaObject.

Pretty soon, we'll see a flurry of languages appear on top of the DLR. A bunch of them are already in development. But we'll also see a wave of DLR object binders appear that have no significant language support on the platform, too. The effects will be profound. Imagine a Java RMI bridge written as a DLR object binder. Using the dynamic type in C# 4.0, you could make calls to remote Java objects from .NET as if they were just part of the framework. Or imagine a type binder that exposes RESTful services as a dynamic object model. There's really no end to the possibilities here. The key to all of these scenarios is the enablement of dynamic services and that sounds a lot like what Web 2.0 was supposed to be.