Frank La Vigne called me out on the Software Development Meme. Michael Eaton started this one. The path to me is:

Michael Eaton >> Dan Rigsby >> Chad Campbell >> Pete Brown >> Frank La Vigne >> Kevin Hazzard 

Here's my response:

How old were you when you first started programming?

I got a hand-me-down TRS-80 Model II in 1981 from a friend who had recently upgraded to a new machine. I was 16 years old then. It was like crack for me. I was instantly addicted to software development or at least the idea of it.

How did you get started in programming?

In the early days, commercial opportunities outside of the mainframe computing space were rare. At age 16, I didn't have access to a mainframe yet. Small businesses likely to use things I could invent used paper to track their accounting and they weren't likely to switch to using a computing system that didn't fully replace their accounts payable, accounts receivable, general ledger and payroll systems.

Although spreadsheets like VisiCalc and Lotus 1-2-3 arrived on the scene early, they were seen initially only as helper applications for doing what-if analysis and such. And they weren't highly programmable anyway. So real programming jobs related to the first "killer application" were hard to create. As a result, those of us working on smaller, home-based computers had no real market for our ideas. So we built games and so-called bulletin boards until our age arrived.

It wasn't until I bought an AT&T PC 6300 Plus with an 80286 CPU in 1986 that I found my first real commercial opportunity writing a turn-key accounting and inventory tracking system for a chain of video stores.

What was your first language?

Zilog-80 Assembler. I was a math geek so assembly language spoke to my soul.

What was the first real program you wrote?

I had just turned 17. My Mom was working for a local hospital organizing paper-based records. The hospital had a new numbering system for patient records for which they had to apply brightly-colored sticky labels to manila folders. I wrote a program for her that determined how many of each numeric digit was required to label a range of numbers. I started with a brute force approach which was very slow and then created an algorithm with a fixed number of computations to yield the same result. The hospital bought the program from me for $50 to help them manage the disbursement of labels to all of the contractors working on the project. It was written in Color BASIC for the Tandy Radio Shack Color Computer. I ported it to GW-BASIC to run on the hospital's IBM PCs.

What languages have you used since you started programming?

Various assembly language dialects: Z80A, MC6809, Intel 80x86, DEC Alpha and Motorola PowerPC. I wrote assembler on a Perkin-Elmer mainframe but I don't remember what the CPU architecture was, to be honest. I remember it was pretty weird, more like a macro compiler than an assembler. I got my first C language compiler for the Tandy Radio Shack Color Computer although I preferred using the assembler on that machine.

I was introduced to C With Classes (which later became C++) in 1984. There was no C++ compiler for the Perkin-Elmer mainframe I was working on. So we used something called Cfront to translate the C++ text into C language which was then further compiled into a set of linkable objects. I worked in C and C++ almost exclusively from 1988 until 2002 with brief stints into Java-land.

In 2002, the bright light of C# shone upon me and I've been pretty happy since then. C# is more expressive and makes me more productive than any other language I've ever worked with. I love Python and I'm itching to use it commercially. However, finding real applications that I can't implement with C# is pretty tough. I'm not sure PowerShell is a real programming language but, if it is, it's definitely one of my favorites.

Every time I've been forced to work with JavaScript (directly or indirectly), I've felt like I needed to take a shower. If JavaScript or HTML ever becomes the right way to implement anything, I'm switching professions. Silverlight has me very excited, as you can imagine.

What was your first professional programming gig?

In my senior year as an undergraduate student, I was contracted to write an application for tracking the inventory and sales for a small chain of video stores. I built the whole thing using Symantec's Q&A product. It was a way-cool, weird product that was part word processor, part database, part reporting engine. You could do amazing things in a short time with Q&A that would take days or weeks to implement using other tools. I think it was the first RAD tool there ever was in the PC space.

If you knew then what you know now, would you have started programming?

Yes. I enjoy software development now more than ever.

If there is one thing you learned along the way that you would tell new developers, what would it be?

Teach part time. Whether you do it at work, in the user group community or as an adjunct faculty member at a local college (I do all 3), you'll find that you learn more, faster by preparing to teach than you could ever learn as a student. Teaching is a fantastic form of mental catharsis. Teaching also helps build your public speaking skills, which translates into greater responsibility and a higher salary as you move forward in your career. Developers who can build cogent arguments and present them to executives often go further and faster in their careers.

What’s the most fun you’ve ever had … programming?

When I worked at Intel Corporation in the Architecture Labs, the supercomputing group parallelized one of my fractal algorithms and made it available as a 3D "game". I had written the Pascal language implementation of the Julia Set solver in college while I was working on a paper concerning the Riemann-Stieltjes Integral. The Julia Set is a particularly beautiful fractal based on the concept of a Riemann map. Working with the supercomputing group to parallelize my algorithm and see it work in real-time 3D was a heart-poundingly cool experience. You could visually "drive through" the set to infinite levels of detail, zooming, panning and scanning in real time. What took my PC days to compute, the supercomputer was performing in milliseconds, allowing you to move through the Julia Set fluidly as if flying in a flight simulator. It was a breathtaking experience for me.

Who are you calling out?

• Justin Etheredge
• Alan Stevens
• Chris Love
• James Avery

I just read Andy Leonard's Five Bugs blog post concerning the Entity Famework Vote of No Confidence. You should read it, too.

That's the right approach Andy. As I said in my earlier blog post, this is not the Continental Congress facing off against King George. There are already plenty of alternatives to any technology that Microsoft makes available today. And yet, Microsoft's strategy of continual improvement works and they almost always gain market share.
 
Most good developers have the patience and foresight to use this strategy to their advantage. The weaknesses in a Microsoft technology early in the game create all sorts of opportunities for developers and architects to show off their prowess in closing those gaps. We build tools and techniques. We write about our experiences. We speak about issues in public, constructively. We build trust and support in our spheres of influence.
 
These actions are like a rising tide. Broad support for the right ways of serving our clients has the effect of lifting everyone's boat, including Microsoft's. Sharp attacks like the ones we've seen in recent days are more akin to using torpedoes instead. It just doesn't work. Look at Scott Bellware's tweets over the last day or two. There's a sample of them below. There's no professionalism in that vitriol. I agree with Andy. There's something else going on here. This can't be about making Microsoft better.
 
Here are some of Scott Bellware's tweets from yesterday and today with a few <snip>s to protect the innocent:

• @<snip> petty sabotage in this community is done because it's implicitly permissible by the towering gods of tech ed
• Lean Microsoft... for real... i would work myself to the bone for a software company that behaves like Toyota rather than Boris and Natasha
• imagine a microsoft that believed getting it right the first time was an imperative
• so much potential, so little will to transform it into betterment
• people, it's ok to say in public those things you say in email and in person. the more courages voices, the better things get for all
• we somehow got convinced that it's not ok to dissent to microsoft. when it's impolitic to hold a corporation to account, it has impunity.
• the tech ed crowd and rd crowd has the power to change .net mainstream culture. it appears too busy protecting its own interests to care
• dreaming of a .net community where integrity, courage, and diversity were unassailable values that everyone defends at all cost
• if only a high-profile blogger or podcaster in mainstream .net community had the integrity to talk about this problem. yeah. dreaming.
• THE hallmark of microsoft community is the drive to limit free speach
• mainstream .net's answer to the ef letter is to continue to spam the list. this is the clearest indicator of how shallow this segment is
• after years of dumbing down customer community, and taking input from that community, microsoft is itself a victim of its own craven ways
• the movie idiocracy is an allegory for the results of microsoft holding its customer community back from the advances it can't afford
• the long tail of fear, uncertainty, and doubt entrenched into ms culture by microsoft has limited its own community's intellectual potential
• alt.net: i think you could benefit from some OO design fundamentals / mainstream: nyah, nyah, nyah, i know you are but what am i
• even arguments that call people out by name are valuable when accurate and substantial, but name calling is what we get from ms community
• the mainstream .net community is always ready willing, willing, and able to be petty in its opposition to alt.net
• @<snip> other dev communities think the .net community is so primitive that they barely think it's worth paying attention to
• <snip> signed the ef letter. i think it's a hoax. can someone at ms confirm? if it's a fake, i'll delete it
• is there a magazine in the microsoft space that isn't on microsoft's payrol?
• the continued quoting of roger jennings' stilted characterization of the ef letter in microsoft media shows how corrupt microsoft media is

Read anything in there that makes you feel like a professional? Not me.

I use NHibernate and LINQ to SQL on a site that supports millions of end users. Murray Gordon, an Architect Evangelist at Microsoft, has written a nice synopsis of the Entity Framework Vote of No Confidence debate so far. More than anything else I've read on the subject, this article brings together facts and opinions that seem to ring true from my deep experience.

I'd like to go on record and say that the way this is being handle by the MVPs who signed the petition is unprofessional, in my opinion. Microsoft MVPs are not required to tow the line and agree with everything that Microsoft says and does. But there are ways to communicate with Microsoft that are constructive and there are ways that would make any corporation, including Microsoft, digs its heels into the dirt. This petition used the latter strategy unfortunately. In particular, Microsoft MVPs have channels that the rest of us don't have. They should use them and not the blogosphere to make plain there grievances.

In some sense, I feel as though the signatories of the petition feel like they are playing Continental Congress against King George. But there's no Boston Tea Party here. Microsoft didn't raise any undue taxes from any of us. They simply put out a framework that's clearly a v1.0 product. Microsoft doesn't win with v1.0 products. It wins with version 4 products because as a corporation, it knows how to get the first down (an American football term meaning the team gets to stay on the offensive).

I say let Microsoft run the ball for a few plays and let's see how they do.

Databases are just awful. I don't mean the products themselves but the concept of databases. Stop and think about how absurdly we behave when we write modern software. We generate scads of information in the course of our daily lives. (A scad for me is about 2Gb the last time I checked. May be more or less for you.) Much of this information approximates the lives we lead and the obligations we must honor. But rather than putting that information into a system that has the tools necessary to model the real world from which the data originally emanated, we usually choose to keep it in a place that does an efficient job of storage. When we need to put it back into the real life approximation engine, we shuttle the information in and out of our application servers as necessary. It's been estimated that as much as 50% of the time we spend in development is in bridging the gap between data storage and the business logic of our applications. That number may be an extreme, low or high. But even if this kind of work accounts for only 25% of our time, why would we choose to spend our development budget this way? Data is so simple. It should just be there, fully accessible to me all the time.

Some operating systems do a better job of closing the gap between code and data than others do. For example, the Pick System, originally developed by Dick Pick in the late 1960s uses a hash-based file system to create associative arrays that are super-efficient for many query operations. The only data type in the Pick System is the string. And most importantly, the Pick database engine is not relational. It is a multi-valued instead, meaning that any attribute that needs to have multiple values can just declare them. In the Pick mind, there's never a need to create related tables and join them for query or reporting. A platform that implements this type of database also typically ships with a Pick BASIC compiler which allows for direct manipulation of the query engine and the associative arrays it produces. The BASIC code runs right there in the database, not on a foreign system. Embedded Pick BASIC is not like the SQL CLR. The SQL CLR, for lack of a better term, is bolted onto the side of SQL Server. You can't do any real data manipulation in the SQL CLR. However, in Pick BASIC, you can freely manipulate schema and data directly. Forget for a moment that it's BASIC and you've got something great there. Compiled code running in the database that can manipulate database objects natively. Way cool and circa 1965.

IBM and InterSystems, among other vendors, still sell these databases like hotcakes today because they solve very real business problems for which relational databases are not ideally suited. First of all, they're fast. And I mean smokin' fast for many types of operations, especially high-volume transaction processing applications. This is partially due to the fact that because there are no join operations (in the classical sense), there's usually less work to do to obtain the data you're seeking. But even when there is a sub-select operation that is required to get what you're looking for, the efficiency of the underlying hash-based file system pays off handily. In database terms everything in the database is indexed, always.

My students and colleagues often hear me say that, "Databases are an unfortunate consequence of history." I say this (and believe it) because if you could travel back in time to 1948 and give the ENIAC developers at the University of Pittsburgh a handful of 4Gb DIMM chips and the necessary plans to connect them to their invention, relational databases like Oracle and Microsoft SQL Server would simply never have evolved. I think that the development path would have been more like what Dick Pick envisioned and built instead. Given enough memory early in computing history, associative arrays, set operations and in-memory manipulation of large data sets would have been the norm. However, as we know, memory was severely constrained in the early days of computing. In fact, it's only been in the last few years as new technology has allowed for memory prices to drop dramatically that it has been feasible to conceive of a solid-state database at all. Oracle's TimeTen and Microsoft's Project Code Name Velocity are leading-edge concepts in a new market-segment that will, one day, fully realize Dick Pick's vision, in my opinion. I predict that accessing data from distributed, in-memory databases will become the norm within my lifetime.

Many of the current Object/Relational Mapping (O/RM) debates are centered around my database evolution postulate because O/RM tools attempt the inverse of what the Pick OS does to achieve the same effect. O/RM tools essentially pull as many database semantics (sans execution) into the application tier as possible where the logic of the program is codified. Whether we run Pick BASIC in the database or use an O/RM to marshal data close to our C# code, the desired outcome is the same. But pulling data into an external execution engine as O/RM tools do is pretty close to nightmarish, to be frank. In fact, Ted Neward, whom I greatly respect, calls O/RM the Vietnam of Computer Science today, meaning a quagmire from which one cannot possibly be extricated and for which there is no good outcome. Ouch! What a stinging rebuke from a guy who's singularly qualified to make an assessment in this space. Even Ayende Rahien's blog post from earlier today reveals a sense of desparation about the state of O/RM technology. What a mess we've gotten ourselves into! No O/RM suite that I know of addresses the real problem at hand, i.e. making data access so transparent that you don't even know you're doing it.

We use both NHibernate and Language Integrated Query (LINQ) to SQL at SnagAJob.com for O/RM. They make life easier in some ways but so much more difficult in others. I cannot begin to count up the hours we've spent tuning the session management code in NHibernate to deal with authentication and transaction management issues. And you don't burn up welterweight programmer resources on that kind of work. Your heavy hitters need to be deeply involved because there are architectural design issues at every turn. Every minute that your senior developers and architects are distracted with this kind of stuff, they aren't focusing on what you thought you hired them for. LINQ is better than HNibernate in a couple of ways, chiefly because of the expressiveness afforeded by the IEnumerable<T> extension methods and the query comprehension syntax. But deploying LINQ to SQL or LINQ to Entities in a real-world environment is still not as simple as it should be. And the real goal of transparent data access is still far, far way using NHibernate or LINQ.

If you know of an O/RM suite that makes accessing SQL data more Pick-like as I've described, i.e. more transparent, I'd like to hear about it.

<Interesting Related Story> In 1993 while working for Datastorm Technologies, Inc., I attended Comdex in Las Vegas. At lunch one day, two fellows joined me at the table. The older fellow to my right introduced himself as Dick Pick. I asked him what he did for a living and he graciously and eagerly explained the Pick OS, it's simple power and beauty and a smallish version of his life story. I was impressed but didn't really get it at the time partly because the fellow seated across the table introduced himself as Phil Katz, the inventor of the PKZip file compression utility. For me, Phil Katz's fame overshadowed Dick Pick's because I didn't know any better. So, I didn't engage with Dick in conversation to the degree that I really should have. History, it seems, hasn't been all that respectful to Dick Pick either. Phil Katz has a detailed Wikipedia article about him yet Dick Pick doesn't, for example. Googling for Dick Pick yields scads (there's that word again) of Dick's Pick's Grateful Dead references and nearly nil related to the computer science genius of our time. In retrospect, even being seated with a legend like Dick Pick was a real honor. I wish I had known to take advantage of the opportunity that was given to me. Live and learn. </Interesting Related Story>

OK, I broke down and registered for CodeStock. That'll be a long drive. But the schedule looks awesome. This is a Code Camp on steroids in Knoxville, Tennessee. Lots of heavy hitters coming in to speak this time: Jeff Prosise, Alan Stevens, Steve Andrews, Todd Anglin, Rachel Appel, James Avery and many more.

Looks like I'll be riding there and back with Justin Etheredge. Spending some time with Justin will make the time go by faster.

The date is 9 August 2008 and the cost is $25. Hope to see you there.

Dave Balter has released an e-book entitled The Word of Mouth Manual: Volume II. I am not a marketer but I am familiar with concepts like Dunbar's Number and how the spheres of influence in a community dictate the rate of spread for a message. The popularity of social networking has everyone talking about viral marketing these days. "Let's attach our product to this social network or that one and drive new traffic by word of mouth," seems to be the buzz.

I don't think most people realize how improbable a successful Word of Mouth (WoM) campaign is. Dave Balter's premise in this book is that while you can and should do things to get into a WoM stream, getting into a WoM river is much harder. In fact, he says that when the timing's right, the opportunity picks you, not the other way around. That's some wisdom I think.

In the sphere of influence in which I participate, the real trust zone is very small. This is happening on a wide scale I think which results in the fact that, "Marketers are spending more and more money delivering messages that consumers are trying harder and harder to avoid." WoM marketing is all about raising the influence of the dollars spent. But with such a small zone of trust, can it deliver?

What does this have to do with software architecture, Kevin? Well, as the liaison between the business and technical groups at my company, it's difficult to count how many meetings I've endured where someone touts the brilliance of the latest viral marketing idea they've come up with. And there are real business resources at stake here. Every moment we spend chasing down a bad idea that seems good keeps us from spending time on those that are truly worthwhile. I need the vernacular that lets me challenge those ideas in a constructive way and I found Dave Balter's book instructive to that end.

Download the free e-book from the link above (or click on the book cover to check out Dave's blog) and decide for yourself. It's a quick read. Hey, look at that! The distribution of the book is viral. Pretty cool, huh?

It says it right there in the documentation. But somehow I had never picked up on it.

If you enable Web-garden mode by setting the webGarden attribute to true in the processModel element of the application's Web.config file, do not use InProc session state mode. If you do, data loss can occur if different requests for the same session are served by different worker processes.

We've been using ScaleOut StateServer for some time now. Aside from annoyances like not being able to see performance statistics for the in-memory cache, ScaleOut StateServer is a good product for managing state data in a large web server farm. (Read about my solution to the missing performance counters problem.) We followed most of the best practices for setting up StateServer. But as we approach the 11 million user mark at SnagAJob.com, we have been seeing more and more issues that point to ScaleOut being the root cause of some serious performance issues on our site during peak load.

We were using ten virtual machines running IIS6 with five IIS worker process instances per server. That's fifty worker processes. Not a huge web server farm but not puny, either. In any case, after speaking with ScaleOut support and learning that many of their customers report networking problems while using vmware ESX to host Windows Server and IIS, we decided to go back to InProc session state management for a while. To do this, we had to enable one of the session persistence modes on our load balancers which is far from ideal. Doing that makes things quite a bit less fault tolerant and slows down the deployment process considerably. But we had to do something and this seemed fairly low risk.

When we switched back to InProc session state management mode, we started noticing strange behavior for logged in users. Tests confirmed that the multi-instance web gardens were the culprit. We downgraded to one IIS worker process instance per server and everything is beautiful.

It's completely counter-intuitive to me (and to everyone else I've spoken) that multiple IIS worker process instances operating as a single web garden don't share InProc session state. Logically, it's one server, right? And Microsoft has had support for memory-mapped files in Win32 for more than a decade. Well, as I said, it's right there in the documentation. But an event log warning on application startup would have been nice in this case.

By the way, even though I like ScaleOut StateServer, our fifty IIS worker process instances have been reduced to three (yes, 3) and we are handling our normal volume nicely. We know that our networking environment isn't ideal. Any company that has to be meet the PCI Security Standards is going to jump through some intricate networking hoops. But there's little reason to deploy fifty server instances to solve a scaling problem when three will do.

If you are in central Virginia today (12 June 2008), be sure to join Andy Leonard, Ron Deskins, Mark Hudson and me for a roundtable discussion about SQL server. Andy, Ron and Mark are all great community leaders and they happen to be excellent SQL database architects. This will be a fun and informative meeting. Be sure to register using the link below. Here are the details:

When: Thursday, June 12th, 2008, 6:30 PM - 8:00 PM
Social Time starts at 6:00 PM!

Who: Everyone! In particular Andy, Ron, and Mark.

What: This is going to be round table discussion facilitated by Andy Leonard, Ron Deskins, and Mark Hudson.  The topic is SQL Server, but other than that it is wide open.  Please post your comments to the board so we can get a lively discussion going at the meeting!  

For twitter.com users, the near perfect depiction of the series of poor architecture choices leading to the current state of affairs shows twitters (birds) airlifting a seemingly blissful whale by threads over the sea. Aside from the fallacy of bliss in losing access to one's tribe, no image could be more apropos for indicating that twitter is down. Twitter is over capacity in so many ways.

I've been thinking and writing code in the personal connectivity space for a decade or more. I even have a patent in the peer presence space. I joined a freaky little start-up that Intel Corporation invested in called The Palace. Time Warner and SoftBank were in that mix, too. In the late 1990s, we had high hopes of turning our vision of inter-personal connectedness into a cash cow. There were distance learning angles, entertainment angles, etc. All painfully 1990s thinking in retrospect.

We made so many mistakes. However, the technology wasn't the real problem. It was our thinking that was specious. Even as our pens stroked out the patents on paper and our keyboards tapped those ideas into great code, we couldn't imagine the power of the tribe that services like twitter.com evoke in the average user. Our Palace code was built very much like twitter. Centralized message dispatch. It did OK at intranet scale but when it came to thousands or millions of users, it was just the wrong way to build the thing.

Success can be a powerful enemy. Hubris, it seems, is a much greater threat to any technology-oriented business. The scale of the Internet guarantees that. As Andy Grove taught me, Only the Paranoid Survive. Paranoia, like hubris, is an equally self-interested set of emotions. But paranoia is sigularly devoid of vanity. And vanity distracts us from truthful ideas like, "It's probably not smart to build a hub-and-spoke protocol for something that has to scale to millions of users."

I agree with Scott Hanselman that microblogging should not be centralized. Except for SMS and directory access, there's just no reason to make any other parts of a system like that centralized.

I snapped this pic walking through the Books-A-Million store in Richmond, Virginia on 5 June, 2008. Sign of the times or a simple shelf stocking error?