got net?

Kevin Hazzard's Brain Spigot

About the author

Welcome to Kevin Hazzard's blog.
E-mail me Send mail

Recent posts

Recent comments

Authors

Disclaimer

The opinions expressed herein are my own personal opinions and do not represent my employer's view in anyway.

© Copyright 2010

Exploring the F# Language Series Part 3 - Functional Programming

Throughout this series, I will be exploring the F# (pronounced F Sharp) language as a beginner. Perhaps you're just like me in that you've never worked with the F# language before but you are very curious about it. You may not understand the hype you've been hearing about so-called functional languages. But that's OK. If you want to learn along with me, that would be great.

Along the way, I welcome your comments and feedback, both to instruct me and other readers. You can get an overview of the complete series by visiting the series index. Enjoy.

 

Part 3 - Functional Programming

An Introduction to Functional Programming with F#

Many years ago, when a new programming language appeared, the few programmers who existed would, on average, spend a few days or a weeks learning the syntax and the concepts of the language. We invested that time because we were looking for ways to fill in the gaps left by our 9-to-5 languages. And there were many gaps to fill. I started out my programming career using C Language and a machine language assembler. We wrote to the metal and achieved truly amazing performance in our applications. But we had to build almost everything from scratch. As a result, we spent way too much time working on the fundamentals and way too little time adding tangible value to the businesses that we served.

But in recent times, languages like C# and Java have become so rich that most developers don't feel a need for spending a lot of their spare time understanding new things that are vastly different from that which they already know. While we may spend time learning new techniques, those new skills are often aimed at increasing the efficiency or reducing the complexity with which we program in our favorite language. Putting some rough numbers in place, I'd say that when I was a C Language developer, my language did 65 percent of what I wanted it to do. C++ gave me about 75 percent satisfaction and Java approached 85 percent. C# is so rich, however, that I believe I'm achieving 95 percent satisfaction in using it. But there are still some things that are hard to do in C#.

Over the past couple of years, I've been using the Python language to help in those cases where C# is just too statically typed. In .NET, I can inject Python code directly into C# using the Dynamic Language Runtime to get an interesting set of capabilities. Mixing Python into C# can give C# a kind of fluidity that cannot be achieved on it's own. But dynamically-typed, duck-typed languages like Python also have their problems. Sometimes they are just too weakly typed to be valuable in a large-scale commercial deployment. For larger applications, I rely on the safety of the static typing that C# provides. In searching for a language that would have the compile-time type safety of C# with the type inference capabilities of Python, I found F#.

My first stumbling block in learning F# was that in looking at the code, I couldn't tell what was going on. That's a huge barrier in learning any new language. Based on the richness of my favorite language as described earlier, if I can't figure out the syntax of a new language within ten minutes, I will probably stop trying at that point because with a little bit more effort, I can probably make C# do what I want. When I look at Visual Basic or Ruby code, I can generally tell what's going on despite the fact that I am not proficient in either of those languages. But F# really threw me for a loop. As I stared at F# code for the first time, I struggled to find some prototype in my brain deposited during my 25 year-long programming career that I could draw upon to make sense of this strange syntax. But I must admit, it just wasn't there. I did not possess the tools to discern the logic buried in the F# code I was looking at.

This was discouraging to me because my friends Amanda Laucher and Matt Podwysocki had told me so many great things about F# and I wanted to share their enthusiasm. But I asked myself, "If F# is that odd, should I step so far out of my comfort zone to learn it?" I almost gave up. Then my friend Justin Etheredge presented a talk on the Functional Programming Features in C# to the Richmond .NET User Group. In that talk, I recognized features that had crept into my favorite language and how they related to the fundamentals of F# which I had read about but didn't yet fully understand. I saw an opportunity to write better C# by learning F#. And that was enough to convince me that I needed to buckle down and learn F# once and for all.

Diving right into the F# syntax, let's take a look at some simple F# code. In this example, I am going to create an array of roman numerals and print them out to the console.

let romanize n =
    let numerals = ["nulla"; "i"; "ii"; "iii"; "iv"] in
    List.nth numerals n;;

let showArray =
    Array.iteri (fun ndx value -> printfn "%d : %s" ndx value);;

let i = Array.init 5 (fun value -> romanize value);;

showArray i;;

If you have the F# compiler and tools installed, you can simply paste this code into the F# Interactive tool (FSI). Since I am using version 1.9.4.19 of F#, I can change the directory to "C:\Program Files\FSharp-1.9.4.19\bin" in PowerShell or CMD.EXE and execute ".\FSI.exe". If you are using a different version of the F# distribution, your F# tools may be in a different location.

I am now going to walk through this sample step by step, pasting each part of the F# program into the FSI window. In this next graphic, I'll show what happens if you paste the first sample function named romanize into FSI.

Romanize1

Notice that when you paste the romanize function into FSI, it evaluates the code and outputs:

val romanize : int -> string

F# has essentially inferred the type of the parameter n as an integer and the output type of the romanize function as a string. If the F# compiler could talk through it's line of reasoning here, it might say something like, "I see how you are passing the parameter n to the List.nth function as its second parameter. So n must be an integer because that's the type expected by the nth function in the second parameter position. And by selecting the nth string from a list of strings as the last operation, the romanize function must return a string as its result type."

If you haven't already noticed it, the really interesting thing that's happening here is that although we didn't specify parameter type or return type for the romanize function, F# figured it out. And it did so at the time of compilation. So, we can already see that F# has some of the syntactical beauty of Python, Ruby or JavaScript combined with the type safety of C# or Java. Let's paste the next part of the F# program shown above into FSI.

Romanize2

After pasting the showArray function into FSI, we can see F#'s interpretation. However, in this case, we run into a type name that most of us will not recognize: unit. So, this function accepts a string parameter and returns a unit? What's a unit? None of the programming languages I use has a type by that name. And where's the parameter to the function that F# inferred to be a string? It's missing! Well, look at the implementation of the function as I walk you through its declaration step-by-step.

First of all, how did F# figure out that I was going to pass in a string array to the showArray function? Unlike the romanize function shown above, I didn't put the name of a parameter into the declaration of the function. As you may recall, the romanize function used a parameter named n in its declaration. But here, in the showArray function, there's no parameter shown at all. But, somehow F# figured out what was going on. How? Let me show you another way that you could have written the showArray function.

>let showArray a =
-    Array.iteri (fun ndx value -> printfn "%d : %s" ndx value) a;;

val showArray : string array -> unit

In this case, I've been a bit more explicit. But the output is (almost) the same. So, the original mystery still remains. How did F# figure out that the parameter, implicitly or explicitly defined, should be a string array? Well, as a test, check out this slightly modified method I'll call showArray2. Pay careful attention to the two differences you'll see (not counting the different function name).

> let showArray2 =
-    Array.iteri (fun ndx value -> printfn "%d : %d" ndx value);;

val showArray2 : (int array -> unit)

Going top to bottom, change one that you should have noticed is that the printfn function is using a different format string of "%d : %d" instead of "%d : %s" as it did in the original showArray function. Change two is that in this case, F# has determined that the input parameter to the function should be of type int array instead of string array. You can see this by the fact that it outputs (int array -> unit) to the console.

So, how did changing the format string for the printing function steer the F# interpreter to this conclusion? C Language programmers may recognize %s to mean string when formatting output and %d to mean integer. And that's what these symbols mean here when used with printfn in F#, too. OK, let's try an experiment. If F# is really parsing the format string to infer the type of the input to this function, let's feed it something invalid and see what happens. In C, C++ or C# which are all strongly-typed languages, an invalid format string wouldn't be caught until runtime. But if our hunch is correct, F# will see the problem much sooner. Let's define a new function called showArray3.

> let showArray3 =
-    Array.iteri (fun ndx value -> printfn "%d : %p" ndx value);;

Array.iteri (fun ndx value -> printfn "%d : %p" ndx value);;
------------------------------------------^^^^^^^^^^

stdin(20,42): error FS0191: bad format specifier: '%p'. stopped due to error

Wow! F# actually does validate the format string and it must also be using the information it finds there to infer the types of other values. It saw the invalid %p format specifier and failed to compile it. That's pretty amazing to me! We've figured out how F# determined that strings would be input, but we still don't know how it knew that an array of strings (not just one string) would be required as a parameter. To understand that, we need to break down the syntax of the showArray function a bit more.

Notice that the function being invoked inside showArray is called Array.iteri. The Array class, which is fully qualified by the name Microsoft.FSharp.Collections.Array, is a helper class that contains interesting functions like the iteri function being invoked here. The iteri function iterates over the elements in an array, executing a function on each one. That function takes two parameters: the zero-based index of the array element and the value of the array element at that index. That function that is invoked for each element has to return nothing. In languages like C++ or C#, we would call a function that returned nothing void. In Visual Basic, you would call it a Subroutined instead of a function. But F# uses the term unit for this concept instead. The unit type is sometimes shown as empty parentheses () in F# interpreter output.I like to think of the F# term unit as meaning the function performs some unit of work that may produce a side effect but has no real return value. If you were to paste a method into FSI matching the functional signature of the Array.iteri function, you would see F#'s interpretation of the method like this:

val iteri: (int -> 'a -> unit) -> 'a array -> unit

The 'a syntax is F#'s way of saying "a generic type". So reading this in English, I might say, "The iteri function takes two parameters. The first parameter is a function. See the parentheses I put around that parameter definition to help you understand that? That function parameter take two parameters of its own. The first is of type int and second is one of whatever generic type the array I'm iterating over contains. That function to be passed as a parameter returns nothing, otherwise called unit. The second parameter to the iteri function is the array itself. And, finally, the iteri function returns nothing."

Now go back and look at the first parameter passed to Array.iteri in the showArray function. It reads:

(fun ndx value -> printfn "%d : %s" ndx value)

This maps to the (int -> 'a -> unit) portion of the Array.iteri signature shown above. The F# keyword fun means this is an inline function. The two parameter are ndx which is an integer and value which is a something, a generic thing held in an element of the array. F# needs to figure out what type that something is. In this case, since we are invoking printfn to print a string to standard output, the only way to determine what type value is, is to look at the format string passed to printfn. Since the value parameter maps to the specifier %s in the format string, the type of value must be a string. And since we are iterating over an array using Array.iter, the entire array must contain strings as a result. This is how F# figured out that the showArray method must accept a string array parameter.

Wow! I don't know about you but that makes my head spin. Forget for a moment that in this language called F#, I can pass functions around as parameters. That's not too big of a stretch if you've used function pointers in C Language or delegates/anonymous methods/lambda expressions in C#. But the depth of probing being done by the F# compiler for type inference is way more than most of us are accustomed to, including me. I like it though. The fact that the compiler is doing that much work for me means that I can write far less code yet have all of the type safety to which I'm accustomed in languages like C#.

Let's take the next step in the program started above by pasting the next line of F# code into FSI.

Romanize3

With this line of F# code, we're using another of the helper functions in the Microsoft.FSharp.Collections.Array class called init. If you could paste the function for init into FSI, it would interpret it like this:

int -> (int -> 'a) -> 'a array

Let's see if we can use the F# skills we've developed so far to read this in English. Array.init must take two parameters: an integer and a function. And it must return an array of some generic, unnamed type. The second parameter, the function one, takes an integer and returns a single instance of the generic type that the array that will be returned contains. So, matching up the call to Array.init in our sample code, the number 5 must be the first integer parameter. That essentially says "Create a one-dimensional array of length 5." And as we saw before in the showArray function, the F# keyword fun is being used here again when invoking Array.init to pass a function as the second parameter to it. This is standard F# faire. Passing functions to other functions is the F# way of doing things.

So, using the signature for Array.iter above, let's analyze what's actually passed as the second function parameter.

(fun value -> romanize value)

If this function must accept an integer, then the value parameter must be an integer. And since it must return a single instance of the type contained in the array, then the call to romanize value must return one of those. Going back to our romanize function definition, we can see that it accepts an integer and returns a string. The value parameter given here is an integer. That matches which is apropos. And the return type of the romanize function is used by Array.init to infer that the array it should create consists entirely of strings. Then the F# interpreter has aptly discerned the type of the value i in our small program as a string array. Lastly, let's execute the showArray function passing the string array i as a parameter.

Romanize4

The showArray method, accepting the value i as an array of strings, iterates over them invoking the printfn function to print the index and the value at that index within the array to the console.

You've learned a lot today, I hope. In the remaining 4 parts in this series, we'll ease deeper into the functional aspects of F# as we examine other language features. That's enough for now though. In closing let me show you how our little program would look if we used the #light;; directive. This directive helps clean up the F# syntax a bit depending on white space indentation to determine functional boundaries. As a part-time Python developer, I just love that feature of F#. White space should be significant, despite what my Ruby-coding friends say. Here's the light version. As an exercise, compare it to the one at the beginning to see what's changed.

#light

let romanize n =
    let numerals = ["nulla"; "i"; "ii"; "iii"; "iv"]
    List.nth numerals n

let showArray =
    Array.iteri (fun ndx value -> printfn "%d : %s" ndx value)

let i = Array.init 5 (fun value -> romanize value)

showArray i;;

That's all for this installment of my F# tutorial. Feel free to take a look at the other parts of this series exploring the F# language by visiting the series index.


Categories: F# | Software Dev
Posted by kevin on Sunday, August 24, 2008 3:30 PM
Permalink | Comments (3) | Post RSSRSS comment feed

Follow Me Follow You Source Code

A few weeks ago, I blogged about a tool I had written called Follow Me Follow You to help me understand who was following me on Twitter.com and, from that group, who I was following. See the Venn diagram below to see what I was after. Since then, the idea of understanding who's who in your TwitterVerse has become somewhat fashionable. This is probably due to the low signal-to-noise ratio that you get when you follow a lot of people.

Dunbar's number says that we primates are limited to about 150 meaningful relationships. This seems to hold true in the social networking space, too. In the TwitterVerse, for example, when the number of friends you have approaches about 150, there's so much noise, you can't get much value from the tool without other tools like TweetDeck which allow you to create sub-groups within your tribe. So, understanding your friends and your mutual friends as shown in this diagram can be helpful. Also, being able to quickly scan those who follow you but you don't follow from time to time, can help you from becoming a Twitter snob, if you know what I mean.

I finally found a couple of hours to clean up the Follow Me Follow You source code. I've linked to a ZIP archive containing the source code below. It's C# 3 code and includes a stripped-down version of my Mingle.NET social networking API. The TwitterClient class demonstrates how powerful lambda expressions are in C# 3. Check out the code required to load up the NonFollowingFriends (green), MutualFriends (blue) and NonFriendedFollowers (yellow):

private void LoadFriendsAndFollowers( object sender, DoWorkEventArgs e )
{
 if (_creds == null)
  throw new ApplicationException( "The credentials are null. " +
   "The user must be known to complete this operation." );

 _friends = FetchTwitterUserScreenNames( String.Format(
  _friendsListUrlTemplate, ScreenName ), false );

 _followers = FetchTwitterUserScreenNames( String.Format(
  _followersListUrlTemplate, ScreenName ), true );

 // left side of Venn diagram - friends who don't follow me
 _nonFollowingFriends = _friends.FindAll(
  friend => _followers.BinarySearch( friend ) < 0 );

 // right side of Venn diagram - followers who aren't my friends
 _nonFriendedFollowers = _followers.FindAll(
  follower => _friends.BinarySearch( follower ) < 0 );

 // intersection of Venn diagram - my mutual friends
 _mutualFriends = _friends.FindAll(
  friend => _nonFollowingFriends.BinarySearch( friend ) < 0 );

 var onLoadComplete = e.Argument as
  Action<ICollection<string>, ICollection<string>>;

 if (onLoadComplete != null)
  onLoadComplete( _friends, _followers );
}

Follow Me Follow You C# Source Code (19KB)


Tags:
Categories: C# | Fun | Software Dev
Posted by kevin on Saturday, August 23, 2008 2:20 PM
Permalink | Comments (0) | Post RSSRSS comment feed

Why Learning F# Is So Difficult

[mp3:Why Learning FSharp Is So Difficult.mp3] 

My third installment in the Learning F# Series is coming out in the next day or two. While I was writing it, I had some time to reflect on why learning F# is so hard for an object-oriented programmer like me. So I decided to write some of those thoughts into the third installment. After reading it in context, though, I decided that my observations would be better stated as a separate article, outside of the tutorial altogether. If you are tracking my multi-part F# series, you don't have to read this to stay on track. But if you want to understand why F# and functional programming in general seems so hard to you, you may find this piece interesting.

Gestalt psychologists believe learning new things happens best when you already have a strong basis for the new information. That sounds obvious enough to be silly, doesn't it? Let's break it down a bit, though. The idea is that because most learning involves problem solving and because problem solving requires recognition, all learning must transitively be based first on recognizing the nature of the problems we face. In other words, the prototypes that we've developed throughout our lives become the glasses through which we see the world. More importantly, those prototypes must exist in the right combination and strength relative to the nature of the problems we face to become useful in solving them. When faced with a problem for which we have all the necessary prerequisites, an amazing thing happens which Gestalt psychologists call emergence. The rest of us call this phenomenon the proverbial a-ha moment that we've all experienced in learning new things.

Emergence has a peculiar quality that's worth understanding when you're about to study something new like the F# language. You see, emergence supposes that when the human mind discerns a thing, it happens as a single event in one discrete moment in time. Here's an example that may clarify my point. Suppose that someone handed you a picture of something you should recognize. You would never realize your recognition with this sort of pattern:

"Hmmm, what is this? I see an eye so this must be a living creature of some sort.
There's a long tube descending to the ground from the point between those eyes.
Could that be a snout? It's in the right place to be a nose but its odd looking, for sure.
And those big panels on the sides of what must be the face are absurdly large.
As strange as it may seem, they've got to be the creature's ears.
Those enormous curved white things protruding from the face couldn't be teeth, could they?
Its massive body seems to be supported by four thick pillars that must be the legs.
Oh, I see it now! That's a picture of an elephant!"

Think about it. In your adult life, if you've ever encountered the image of an elephant, you didn't reason through the recognition process in that way. You can't even admit that you followed those steps so quickly that it seemed instantaneous. No, in fact your recognition was instantaneous and no deductive process was required at all. The prototypes from your youth were already properly connected in your mind to help you recognize the image presented to you singularly and instantaneously as an elephant.

The pattern of your recognizing the elephant as a singularity is, in fact, where Gestalt psychology gets its name, by the way. The German word gestalt has the meaning "being more than the sum of a thing's parts." The elephant is certainly made up of parts but in the mind's eye, it's a single entity. In this same way, we go about our lives, instantly and singularly recognizing objects and ideas thousands of times each day. Some are visual, some are aural and some are purely cognitive. These points of recognition happen so often and so naturally in the course of an average day that we don't even realize that we're doing it. That instantaneous recognition process is emergence.

Now think for a moment about how a toddler, who may have no exposure to pachyderms would react to the presentation of a photo of an elephant. The thought pattern in the toddler's mind wouldn't be much different from the line of reasoning outlined above. Try to imagine that you're a child with cognitive prototypes for other mammals like cats, dogs, cows and humans in your mind. But no elephants yet. Now try to imagine your first encounter with an elephant. Go back to the paragraph above where I proposed a line of reasoning for the recognition process. In this case, there is no way for emergence to occur in the toddler's mind because he hasn't the prototypes necessary to finish the work. So he must follow a pattern of discovery that is slower and more deliberate than the emergence experience. And the last step of the discovery process cannot be completed for the child without some assistance.

When an adult (or another educated peer) teaches the child by confirming hints about the trunk and the tusks and the thick legs of the elephant, a new prototype will be created that will become a new singularity in the child's mind, ready for future chances at emergence. For the remainder of that child's life, the gestalt of elephant-ness, i.e. the entire elephant as a singular concept with parts bearing distinct features, will emerge instantaneously every time he encounters a new instance of an elephant.

So, what in the world does Gestalt Psychology and emergence have to do with learning F#? A lot, I think. If you've always used imperative programming languages like C++, Java, Visual Basic and C#, you might find functional programming as strange as our toddler finds his first encounter with an elephant. It may be that you don't have the prototypes necessary to learn F#, in which case, you may find yourself falling back to the discovery model that our toddler used to build his pachydermal prototype. Admittedly, the discovery mode of learning is much more arduous than learning that relies heavily on prototypes. Discovery learning often goes much slower, too.

Looking at F# code for the first time, you'll almost certainly ask, "What in the world is that thing?" And just as the toddler might use an eye in a photograph as a starting point, you may find yourself struggling to find a starting point in F# source code to discern the logic that you've been told flows from it. Finding that eye is possible, though. In the remainder of my Learning F# Series, I'm going to attempt to give you that starting point that will help you identify other parts that eventually will lead you to understanding the body of functional programming. My advice is to stick with it. Get outside of your comfort zone and do some real discovery. You'll be happy you did.


Categories: F#
Posted by kevin on Monday, August 18, 2008 6:00 AM
Permalink | Comments (9) | Post RSSRSS comment feed

On Quitting Your Job by James Avery

James Avery and I spent some time together at CodeStock this past weekend. He told me that he was tired of doing technology presentations. I entirely understand what he means. Education in the developer community is very important but at some point, you just want to switch gears and talk to your peers about other stuff that's important in our space. We're all geeks, that's for sure, but we need to be savvy business people, too. And that requires just as much peer training and influence.

James floated an idea past me for a presentation he would like to do and I liked it a lot. It's not your average Code Camp faire but I asked James if he would travel up from Raleigh on October 4th to present this session to our Richmond attendees. He agreed to come do it. The session is called "Tune In, Turn On and Quit Your Job" and it goes something like this:

Tune In, Turn On and Quit Your Job by James Avery

"We don't have a lot of time on this earth. We weren't meant to spend it this way. Human beings were not meant to sit in little cubicles staring at computer screens all day, filling out useless forms and listening to eight different bosses drone on about mission statements." - From the Movie Office Space (1999)

Stop working for the man and learn about writing a software product, building a successful web application or becoming an independent consultant.

I can hardly wait to attend this talk. I predict a packed house. I hope James keeps going on this vein and writes his next book on this topic.

James is one of many great regional and local speakers who will be presenting in Richmond at the upcoming Code Camp. Want to learn more about the Richmond Code Camp on October 4, 2008? As of August 11, 2008, we still need a few more speakers. Don't miss your chance to show the central Virginia developer community that you want to contribute your ideas, too. Submit your presentation abstract and bio here.


Posted by kevin on Monday, August 11, 2008 5:45 PM
Permalink | Comments (0) | Post RSSRSS comment feed

Josh Carlisle and His Best Friend Fred

Josh Carlisle spoke to the Richmond .NET User Group this evening on Sharepoint Development for ASP.NET Developers. He was carrying what looked like a flask of vodka with him which made me think, "This is a guy I've got to hang out with." It turned out to be Fred Bottled Water. Unfortunately, I didn't get a picture of Josh with Fred, but here are pics of both of them. Doesn't Josh look unusually happy? He swears it was water of "exceptional purity with a high degree of virginality." Yeah, right.

Josh Carlisle speaks to the Richmond .NET User Group on 7 August 2008

Josh's presentation was very good. He was a bit perplexed near the end because of SharePoint's pesky insistence on treating the term MasterPage differently from MasterPages (plural). Pfft! SharePoint is so picky like that. Anyway, we had a good time and Josh was just great. He really knows his stuff. He's welcome back to Richmond at any time. Maybe I could get him up here for Code Camp on October 4, 2008. We got to see Nas Ali, too, who travelled to Richmond with Josh, I think. Always good to see Nas. He confirmed with me that he will be speaking for us at the upcoming Code Camp. Nas is a good speaker and his talks are not to be missed.

Thanks again Josh for coming to Richmond!


Posted by kevin on Thursday, August 07, 2008 9:45 PM
Permalink | Comments (1) | Post RSSRSS comment feed

Making SQL and .NET SHA1 Hashes Match

A friend at SnagAJob.com came to me with an interesting problem today. He said that the HashBytes function in SQL Server was outputting different results from the HashAlgorithm.ComputeHash method in .NET. Here's a T-SQL script that hashes the URL to my blog.

DECLARE @data NVARCHAR(max)
SET @data = N'http://www.gotnet.biz/Blog'
SELECT HashBytes('SHA1', @data)

This script outputs 0x7FC8C5E43E9425C890AB96E660C86FC9CB077F4D as the hash value. The algorithm in C# attempting to do the same thing might look like this:

using System;
using System.Security.Cryptography;
using System.Text;

public class HashTest
{
    static void Main()
    {
        DoHash(new SHA1CryptoServiceProvider());
        Console.ReadLine();
    }

    private static void DoHash(HashAlgorithm algo)
    {
        var bytes = Encoding.UTF8.GetBytes(
            "http://www.gotnet.biz/Blog");
        var hash = algo.ComputeHash(bytes);
        Console.Write("{0} ", algo.GetType().Name);
        foreach (var b in hash)
            Console.Write("{0:X2}", b);
        Console.WriteLine();
    }
}

This code outputs 0x10397796345455fa6332db477972dc360b54ef2, a different hash value. Do you see the problem in the code? I didn't at first but it's simpler than you think.

The encoding that I used in the C# code is UTF8 which means the 8-bit Universal Character Set/UNICODE Transformation Format. That's a mouthful, isn't it? In .NET, the UTF8 encoding corresponds to Windows code page 65001 where each source character may map to between one and four characters in the encoded output. I used that encoding implicitly because in working with XML as often as I do, I'm accustomed to using the UTF8 encoding for nearly everything I do. My friend who posed the original question had done the same thing. However, in this case, it's a bad choice.

Looking at the T-SQL code above, notice that the data type for my string is NVARCHAR, that's UNICODE. And although all strings in .NET are stored in UNICODE and the UTF8 encoding is, as its name implies, just transforming the UNICODE to an 8-bit transportable format, the computed SHA1 hash on a UTF-8 encoded string in .NET is clearly not the same as SQL Server's result.

Playing around with some other transforms in the System.Text namespace, I discovered that by replacing the UTF8 encoding with the so-called Unicode encoding (or by switching the SQL data type to VARCHAR) makes the hash computations match between SQL and .NET in my example above. I capitalized Unicode as I did there quite deliberately because I am referring to the type in the System.Text namespace called UnicodeEncoding (which is available as the static Unicode property on the Encoding class) not the UNICODE standard.

In .NET, the Unicode encoding corresponds to Windows code page 1200 and goes by the familiar alias UTF-16. As that alias may imply, the.NET UnicodeEncoding uses a sequence of one or two 16-bit integers to represent each character in the original text. The results are easy to understand visually so I made the graphic shown here.

You can see that the contents of the byte stream from the two encodings is different. The UTF8 encoding strips the high order zero bytes for cultures where they are superfluous whereas the Unicode encoding preserves them. To sum up, when hashing NVARCHARs in SQL, the equivalent encoding to use in .NET code is the UnicodeEncoding. When hashing VARCHARs in SQL server, the matching .NET encoding is the UTF8Encoding.

 


Categories: C# | Security | Software Dev | SQL
Posted by kevin on Thursday, August 07, 2008 1:37 PM
Permalink | Comments (3) | Post RSSRSS comment feed

Enumerating the Silverlight AssemblyManifest XAML File

Go to the Silverlight home page.

From time to time, you may need to discover dynamically which .NET assemblies were packaged with the current Silverlight application. Here's a snippet that might help. It enumerates the AssemblyParts in the main XAP package and puts the names and source paths into a dictionary.

 

var assemblyParts = new Dictionary<string, string>();
try
{
   var settings = new XmlReaderSettings
      { XmlResolver = new XmlXapResolver() };
   var reader = XmlReader.Create("AppManifest.xaml", settings);
   try
   {
      reader.MoveToContent();
      while (reader.ReadToFollowing("AssemblyPart"))
      {
         var name = reader.GetAttribute("x:Name");
         var source = reader.GetAttribute("Source");
         assemblyParts.Add(name, source);
      }
   }
   finally
   {
      reader.Close();
   }
}
catch (Exception ex)
{
   Debug.WriteLine( ex.Message );
   throw;
}

 


Categories: C# | Silverlight | Software Dev
Posted by kevin on Wednesday, August 06, 2008 6:00 AM
Permalink | Comments (0) | Post RSSRSS comment feed

ApplicationDefinition Error in Silverlight

Go to the Silverlight home page.

I work on a lot of different development machines. I have a laptop with half a dozen Windows and Linux virtual machines on it. I have several virtual and physical machines at home and at the office. Keeping them all in sync is not easy. Recently, I've been doing a lot of Silverlight development. I ran into the following error when opening a Silverlight 2 Beta 2 project on one of my development servers:

Library project file cannot specify ApplicationDefinition element

I had forgotten to update the Silverlight Tools for Visual Studio to the latest Beta release. Installing the latest Silverlight beta tools for Visual Studio fixed the problem. Searching the net with this error didn't turn up enough useful information so I thought I should blog about it. Hopefully the search engines will guide others here for this potential solution. 


Posted by kevin on Tuesday, August 05, 2008 10:32 AM
Permalink | Comments (4) | Post RSSRSS comment feed

Follow Me Follow You

I've been playing around with the Twitter API and some of the more popular Twitter clients. My favorite desktop client for Twitter is TweetDeck. But one feature I really would like to have is a view of the relationships between my friends and followers. I want to see something like this:

I thought if I could manufacture that sort of diagram where the people in each part were clickable, I would be able to see and manage the relationships in my "tribe" more clearly. Tonight, taking a break from work coding, I created a desktop application that isn't as pretty but does what I want. It's called Follow Me Follow You and it looks like this:

OK, I warned you it wasn't pretty. But it is pretty cool. You can see the three parts of the Venn Diagram shown above as listboxes on the left side. The left side and right side are independent except that if you click on the screen name of a user on the left, it loads that user's Twitter page in the browser on the right. That feature alone is nice. Having persistent access to your tribe in a directory format is handier than I thought it would be. I can see part of a conversation in the browser window and hop around to related parties without having to use Summize, aka the Twitter search engine.

I can also scan the lists to find out other interesting information. For example, in the "Followers not Friended" list, I can scan for new names that I may have missed in e-mails that Twitter.com sends to me when someone starts following me. After clicking on their name and logging in within the web browser on the right side, I can look at their timeline and decide whether or not to follow them from there.

I used my new Mingle.NET library, a social networking library for the .NET Framework, to build the new tool in about an hour. I haven't released Mingle.NET or the Follow Me Follow You tool. If you're interested in them, let me know and I'll get them ready for production. I can only work on them in my spare time and there isn't much of that these days. :)


Categories: Fun
Posted by kevin on Monday, August 04, 2008 11:06 PM
Permalink | Comments (4) | Post RSSRSS comment feed

Exploring the F# Language Series Part 2 - Installation and Configuration

Throughout this series, I will be exploring the F# (pronounced F Sharp) language as a beginner. Perhaps you're just like me in that you've never worked with the F# language before but you are very curious about it. You may not understand the hype you've been hearing about so-called functional languages. But that's OK. If you want to learn along with me, that would be great.

Along the way, I welcome your comments and feedback, both to instruct me and other readers. You can get an overview of the complete series by visiting the series index. Enjoy.

 

Part 2 - Installation and Configuration

To get started with F#, you need to get the compiler installed. Go to the Microsoft Research F# Downloads Page and download the latest release. If you are using Windows, download the MSI installer. If you're on Windows and want the Visual Studio experience but can't afford those tools, there is another interesting option. You can first install the Visual Studio 2008 Shell which is a free Integrated Development Environment (IDE) that Microsoft made available to help people integrate with specialized tools like F# or IronPython, for example. Whether you are using the full edition of Visual Studio 2003, 2005 or 2008 or the free Visual Studio 2008 Shell, the F# installer will affect all of the necessary changes to make working with F# in the IDE a smooth experience. If you don't have Visual Studio and don't care to use an IDE, you can always run the F# tools from the command line. And if you are using Mono runtime on Linux or Mac OS, download the ZIP file instead and follow the instructions in the README file to complete the installation.

At the time of this writing, I am working with version 1.9.4.19 released on May 1, 2008. Today, the F# installer requires one of Windows 2000, Windows XP, Windows Server 2003 or Vista and the .NET Framework 2.0. I've been running F# on Windows Server 2008 Standard Edition with no problems although Microsoft doesn't officially claim support for that operating system yet. If you are using Linux or Mac OS, you must install the Mono runtime.

If you aren't going to be using F# through the Visual Studio integration, you can skip all the way to the end of this article to the section entitled Using F# Interactive from the Command Line.

Installation on Windows

When you run the MSI installer you downloaded on Windows, you'll see something like this:

The installation usually takes about 5 minutes although your experiences may vary. After you're done with the installer, if you have Visual Studio, start it up. There's an add-in for F# Interactive that's installed but not enabled by default. You may want to turn it on. To do that within Visual Studio, select the Add-in Manager... choice from the Tools menu. You'll see a dialog that looks like this:

Enable the add-in entitled F# Interactive for Visual Studio by checking the box on the left. Press OK to close that dialog and save your changes. You'll see a tool window appear for the F# Interactive that looks like this:

This interactive F# interpreter is very handy. You can type or paste F# code into the interpreter window to try things out. But what's even better is that when you highlight code in the Visual Studio text editor and press the combination Alt+Enter keys, you can send the selected F# source over to the interpreter and run it. That's very handy, indeed.

Changing the Alt+Enter Key Assignment (ReSharper Users Only)

If you are using ReSharper, however, you may run into a problem here. If you are not using ReSharper, you can skip ahead to the next section entitled Using Alt+Enter and Alt+' to Evaluate F# Code in the Interpreter. If you use the standard IdeaJ/ReSharper keyboard settings, the Alt+Enter key combination is set by ReSharper to perform a different function within the scope of the text editor. So, when you press Alt+Enter in the text editor, the associated ReSharper function will run instead. To fix this, let's take a look at the keyboard settings. Here's the Text Editor scope key setting that shows how ReSharper's QuickFix function is assigned to the Alt+Enter key combination. This dialog is shown by clicking the Options... choice on the Tools menu in Visual Studio.

And here's Global scope key setting that shows how F#'s MLSendSelection function is assigned to the Alt+Enter key combination.

The easiest way to fix this clash of key assignments is to change one of them. Since I use Alt+Enter in ReSharper already, I decided to change the F# key mapping instead. I did this my clicking the Remove button in the dialog shown above to disconnect Global Alt+Enter key combination from F#'s MLSendSelection method. Next, I picked a new keystroke that makes sense. For me, Ctrl+Alt+Enter is close to the original and not used by any other add-in. Pressing those keys in combination while the text box under the "Press shortcut keys:" text has the focus makes the description of that keystroke appear in the text box. Next, I changed the scope of the assigned key by selecting "Text Editor" from the dropdown list entitled "Use new shortcut in:" and pressed the Assign button. There's really no sense in having that key assignment be Global in scope because, "What would the MLSendSelection method receive if there's no text editor open?" Does that makes sense? The dialog looked like this when I was done.

Using Alt+Enter and Alt+' to Evaluate Code in F# Interactive

Note: If you changed the key combination for invoking MLSendSelection as described in the previous section, anywhere you see Alt+Enter in the remainder of this tutorial, please substitute your selected keystroke instead.

To try out the F# Interactive tool window, you can type some F# code directly into the window or you can use the Alt+Enter and Alt+' (Alt+Single Quote) key combinations to send F# code from an open text editor. Let's try something simple. Open a new text window in Visual Studio and type a single line containing the following text

   printfn "Hello World.";;

This F# statement will invoke the printfn function for printing text to the console window. The literal text that we're going to print is "Hello World" minus the quotes of course. Lastly, the two semi-colons indicate the end of the "batch" if you will. More on that later. If you were to select all of the text you typed in by highlighting it in the text editor and press the Alt+Enter key combination, you would invoke F#'s MLSendSelection function in the IDE. The selected text would be run as a script and the output would show in the F# Interactive window. Here's what you might see:

Hey, you just wrote and executed the proverbial Hello World program using F#. Notice how the text output to the console that we expected to see shows up there. The Alt+Enter key combination you used is appropriate when you want to select only a portion of one line or multiple lines to send to the F# interpreter.

Another key mapping, Alt+' (Alt+Single Quote), exists after installing F# which is bound to a method called MLSendLine instead of MLSendSelection. As its name may imply, pressing Alt+' will cause the line on which the caret currently rests to be selected in its entirety and sent to the F# interpreter. So, the Alt+' key combination is useful for running one line of F# in the current text editor window at a time without having to select any text first.

Using F# Interactive from the Command Line

You don't have to use the F# Interactive add-in for Visual Studion. And if you're working on Mac OS or Linux using the Mono runtime, you need another option. In this case, the command line version of the F# Interactive tool can be used. On a Windows system, the FSI.EXE program is installed in the following folder:

   C:\Program Files\FSharp-<version>\bin

where <version> is the version number of the F# product you installed. For me, since I am using version 1.9.4.19, my binaries folder is:

   C:\Program Files\FSharp-1.9.4.19\bin

On Windows, you can invoke the help for the FSI.EXE by adding the --help flag to the command line.

Running FSI.EXE allows for interactive interpretation of F# code. After running FSI.EXE, enter the following text:

   printfn "Hello World.";;

and press Enter. You've just written the proverbial Hello World program using F#. Here's what it looks like running F# Interactive in PowerShell: 

When you're ready to quit F# Interactive, type #quit;; and press Enter. 

That's all for now. Feel free to take a look at the other parts of this series exploring the F# language by visiting the series index.

 


Categories: F# | Software Dev
Posted by kevin on Sunday, August 03, 2008 5:30 PM
Permalink | Comments (5) | Post RSSRSS comment feed