Stackoverflow.com

Joel (on Software) and Jeff Atwood are starting up a site called stackoverflow.com meant to be a free Q&A forum for helping to improve the collective knowledge on writing good software. It’s an exciting idea and I hope it’s successful!

Discussing Joel’s post at work, the head of our technology group asked me what I think of the site’s odds for success. I have a mixed response.

I think the traffic and volume of helpful solutions will be high if they can score well on google searches for things like “ASP.NET renders invalid xhtml”. In my experience that’s the best way to find solutions to daily problems, despite the questionable help from sites with which Joel would like to compete.

Joel says there is a problem with too much poor software, and the reason is that people aren’t buying software books.

People I talk to aren’t buying computer books anymore because the technical books are outdated by the time they reach the shelves and they’re too expensive to be continually refreshing your library. Nowadays, all of my technical learning comes from blogs, online documentation like MSDN, and by exploring the source code of good open source projects (like StructureMap, Hibernate and Castle).

I guess don’t see disappointing book sales as the problem itself. I see it as a symptom.

The real problem is the drastic onslaught of .NET development shops. In the past four years or so, .NET has stormed the market for mid-size companies who are growing into the need for robust software. And just like a puppy growing into its big feet, there’s lessons need learnin’.

The problem with .NET owning this HUGE market is that more than half of the developers come to it from VB and Classic ASP. I’m not saying VB and ASP are bad, but the two platforms are targeted for projects with a “let’s just get it out there ASAP, who cares how it runs next year” type of rapid application development.

.NET clearly began as a “rapid” development solution, and Microsoft has softly continued that tenor with things like drag-n-drop designers and providing “enterprise” tutorials with hard-coded sql statements embedded in the xml content of .aspx pages.

The thousands of new programmers are .NET’ers, and they’re being indoctrinated into the sloppiness of rapid application development under Microsoft’s “Enterprise” Practices. The demographic of apprentice programmers we expect to buy computer books like Joel’s has become non-existent simply because it’s not a common practice for .NET developers to wholeheartedly pursue the craft of quality software.

The kind of crafted software that Joel is trying to cultivate is more like the “we’re going to have to mature this into an invulnerable system and live with it for the next 10 years” type of enterprise development. Joel’s kind of software requires a tremendous amount of thought, discussion, discipline and practice.

There are some excellent conceptual books out there that instruct people on how to develop flexible, robust software. These books are not technical in nature and will never become obsolete.

I feel pretty strongly that these books are required reading for any software engineer who wants to be taken seriously. It is just too difficult to believe someone knows what they’re doing as a programmer when they don’t have battle-weary copies of The Pragmatic Programmer or Patterns of Enterprise Application Architecture on their desk. It’s like trusting a corporate accountant who’s never read any of the standards written by the Federal Accounting Standards Board.

For me the good news is that .NET also has a growing contingent of people who come from the worlds of C++, Ruby, Smalltalk and Java. These platforms tend to be more conceptual in their practices and consequently their young developers have been indoctrinated into the far-sighted planning of Martin Fowler, Kent Beck, Joel Spolsky and Dave Thomas. These souls are the remaining survivors of Joel’s lost demographic.

I’m excited by the incredible growth in volume of the ALT.NET mailing list, a forum for .NET developers who want to study the craft of software and are willing to look outside the bounds of the status-quo for more elegant solutions.

Microsoft is also helping the cause with its recent projects such as an Object/Relational Mapper, an xUnit-like unit testing framework, and a highly testable Model-View-Controller web application framework.

It seems the tide is shifting and an increasing number of the “quick and dirty” .NET’ers are learning how to become students of software design. As this happens, I think we’ll see a proportional growth in the sales of books like the ones Ayende recommends.

Shortly thereafter, we’ll see an overall improvement in the quality of software, and saying things like “WinForms databinding doesn’t belong in our domain model” in a .NET shop won’t get you blank stares or downright excluded from architecture discussions. Heck, there might even be architectural discussions!

The Robustness Requirement

There comes a point when writing professional software that a system simply must have the following characteristics, which I collectively refer to as The Robustness Requirement:

• Data Auditability: Every non-system configuration object record must contain fields to capture when and by which user the record was created, as well as when and by whom the record was deleted. “Deleted” rows should either remain and be marked as “deleted” or be moved to a _history table.
• Scalability: Services should be easily moved to additional servers and the web front-end should be clusterable.
• Cacheable: Data should cached in a distributed manner.
• Sophisticated UI: User interfaces should have a level of sophistication that intuitively displays invalid fields and error messages.
• Logging: The system should have a deeply-rooted logging system where all messages are aggregated in a searchable manner. Logging levels and filters should be able to change on the fly without interrupting any services.
• Fault Tolerance: The system should be relatively fault-tolerant so that services can automatically restart, and helpful messages are displayed to the user while services are unavailable.
• Permission-based Security: Almost any action should be permissioned on both a user and role basis.
• Data Storage Scalability: Data persistence read/write operations should be specifically routed through read-only and writeable connections so that read operations can be separated and distributed across multiple servers (and consequently be scalable beyond just one data server).
• Vendor Agnostic: Care should be taken to hide any vendor-specific implementations as much as possible so that service vendors can be changed with minimal effort. This includes the database platform. Too many software shops become tied to a specific platform or vendor.

Admittedly, this is a little bit of a Xanadu type of place to be. Hardly any system actually meets all these requirements, but it should be a conscious decision by the team to abandon any of the above requirements.

“Enterprisey”

I began using the term “enterprisey” along with Tyler, my friend and longtime co-worker, around the time we both read Martin Fowler’s post on Ruby and its relationship with the enterprise world.

Many developers consider enterprisey to be a derogatory term that means “complex; over engineered; too rigid and downright ridiculous.” The term has spawned a lot of discussion, which, I think cuts to the core of professional software: We want to build software that is strong, robust, complete, and a joy to use. But we also want to build software that can change at the same pace our business changes.

A growing business is too fragile and opportunities are too fleeting to allow its software and technology to hold it back. If software can’t change quickly enough or it is constantly derided by the business people when the geeks aren’t around, then the IT group either needs additional people to help meet the need or they aren’t doing a good enough job architecting their system.

Rule #4: There should be a 1:1 ratio between the ability for an organization to leverage a business opportunity and the speed its software can change in order to attend to the new process. A company’s software should never hold back its desire or ability to change the direction of the Bus.

Everybody knows that enterprise software becomes complex and rigid because there are so many requirements around robustness that simply don’t exist when I set to write, for instance, a quick web application for keeping stats in the never ending office foosball tournaments.

But it seems funny to me that in software “enterprisey” means a system that is too leaden to meet changing requirements while in business the term “enterprising” means something that is agile, creative, and capable of succeeding no matter the obstacles.

Because of this drastic dichotomy in meaning of two related words, I tend to think of enterprisey as describing not the over-engineered kind of software, but rather something that has simply been promoted from the quick-and-dirty “rapid application development” phase to something that is robust, auditable and fault-tolerant.

I firmly believe that enterprise-quality software doesn’t have to be enterprisey in the derogatory sense. There is an ever-adjusting balance between robustness and flexibility and good developers must learn the practices of Lightweight Enterprise Software in order to meet the requirements of an enterprising organization.

I also firmly believe that an organization should not have patience for anything less.