Is quality going out of fashion?

A lot of times it's very useful to have your users drive whenever you are looking for bugs or problems at their computers. Even though I'm a very inpatient person when I look at somebody else searching for the key 'A' on their keyboard or moving the mouse slowly to cut and paste through the menu, you have to preserve and look at what they are doing since often what they are saying that they are doing and what they are actually doing can be very different. So every now and then I'm standing behind someone watching what they are doing and my jaw drops. Once someone were in the middle of doing things and the application just crashed. Very quickly the user clicked on the dialog box that said it crashed and relaunched the application, ready to continue with whatever we were in the middle of. Without batting an eye! This happens on occasion and I see it more and more. I call it the windows syndrome.

Windows has this stigma around it that if you have any trouble at all, the first thing you should try is to reboot your computer. If the problem magically goes away, then it's considered to be fixed [1]. Second, if the problem is just that the program crashes every now and then, the solution is to save often and then just get very good at relaunching the app again (perhaps create a shortcut on the desktop, or the quick launch menu?). If you're a windows user, this pretty much describes your daily workflow I presume.

However, if you tell a UNIX person this same workflow, he/she would be horrified. Rebooting?!? That's that thing you do perhaps once a year! And it's scary! Crashing programs? That's serious business, have you hollered up their support for an emergency patch?

The important part here to note is that windows users expect a couple of reboots and crashes as part of their daily workflow, where on any other platform it would be unacceptable. Or at least it used to be, this mentality is spreading.

Console games

One of the things that I used to like about making console games is the fact that you need to get it right the first time, there are no second chances. You need to find all the bugs, all the crashes and get it right.To help you in this endeavor you have a uniform machine park, all the machines that you will run on will be exactly the same. It forced you to pay extra attention to quality, any crashes that were in the game once you went gold master were catastrophic. Today it's a very different matter it seems. While the above is supposed to be true, I guess people are playing it fast and loose now that they can provide a zero day patch through the internet. A lot of games have patches applied to them when you run them, even if it's on day 1 (launch day). This is kind of bad (and I'm sure that since I've written this I'll be eating crow), but I would very much like to get back to the old days where a console game simply didn't crash when it came out in the wild. I understand that you might want to patch for something else, like online games might want to patch balancing parameters (e.g. make weapons weaker, or more powerful, or more expensive to buy etc) still after launch, or make a patch that adds extra functionality (for example, adding trophy support to older games). But it should still be possible to run a game without any internet connection and have it not crash!

Ship early, ship often.

The mantra "ship early, ship often" is repeated as something that is a good thing, often for startups or independent game developers. But there is a big pink elephant in the room: "make it work first". It's very similar to the optimization problem, where if you don't actually have to make the algorithm work, then you can just "optimize" the problem away to 0 instructions. Shipping anything used to have an implicit assumption that the product was working! And by working I include all the crashes not happening. If you have any crash in your app then it's not working. Bulletproofing an application/game is part of those last 90%[2] of effort you need to put in.

Shipping early doesn't really make it better, if you ship beta level quality of a product that keeps crashing and gives the user a bad impression it's very hard to reverse this impression and make the users come back. Now for the masses, this might actually work since they are all blind to the crashes. "It's just the price of doing business on computers today, programs crash". Noooooo! A crashing program is a broken program. Claim a refund! If you bought a car and it had tires that would blow every 30 miles, that would be unacceptable. It's broken and treat it as such.

I see this quite a lot on the iPhone side actually, people release half baked applications and people then pay for the thing, and act like beta testers. That's just weird, we usually pay our testers, but people are drinking the Kool-Aid and asking for seconds.

In closing

Yes it's hard to make a proper program that doesn't crash. The more code you pile onto your shipping executable and the more features you add, the more time you will spend making it bulletproof. Because once it is out in your user's hands it's not allowed to crash. Ever. It sounds simple, several people even think they can make it, even though they probably failed. But let's all at least aim for it, and not simply give up and ship shitty software that crashes (ever installed a video card driver and not have it crash on you?) it simply doesn't make any sense for a user to keep buying software that crashes. Well, at least not when they've stopped drinking the Kool-Aid. Oh, yeah, users: stop drinking the Kool-Aid, crashing applications are not acceptable!

Update 2009-12-09

Almost a week later of course Atwood posts

Rumors also seem to surface that the now defunct Pandemic game Saboteur managed to ship with serious crashing issues.

Footnotes

[1] Actually, rebooting your computer in the case of a problem is probably the worst thing you can do. Sure it most probably will make the problem go away. Temporarily. Then it will return again, and you still don't know exactly what the hell is wrong! No better to diagnose the problem right then and there when you have it the first time, understand it and the fix it once and for all. No rebooting necessary.

[2] Did I mention that making anything is part 90% easy and quick development, and then 90% hard work? Yes, that is 180% of total.

Comments