Even the small children are used to questions like *Are you hungry?* It's very unlikely that they answer *Zero!* Yet, they have no formal knowledge of booleans. ;)
-- Petr Prikryl on comp.lang.python
Hotloading was for a while, well a hot, topic in the game development industry. As games progressed from being very simple programs to more of world simulators today the issue of tweaking and how assets are handled became more of an issue. When your data fits on a standard floppy disk it's not that bad, but when you have your system administrator breathing down your neck because your using up a Gigabyte of disk each night things become complicated.
The sheer amount of data often manifest itself in long loading times during development. This can become a real problem for the artists or programmers who need real time previewing at certain points in the game. The patch for this much larger issue seems to be hotloading, which indeed can be very useful. Hotloading basically means that for example if we press save inside our image editor we want the texture we worked on to automatically be loaded into the running game and show up.
Detect, build and load
It's pretty easy to detect inside a plain old win32 application that files have changed on disk, windows provide native support for this in the function ReadDirectoryChangesW. Once we've detected the changed file we can simply call load on this one and replace the asset in the game.
Chances are though that you are not editing the same file that you are loading up into the game, typically one have preprocessing tools that packages these files into neat little chunks that doesn't need very smart routines to load them, e.g. for textures we might provide a header that tells the loader the number of mipmaps, the format and the top level size. So we need to invoke the build system to drag the new asset through the pipeline (or a fastpath), but most likely we're not going to support loading of raw assets inside the game (it's certainly feasible for plain old textures, but consider the case of models, I'm not going to load Maya source files into my game, or even something more complex like your level data). Hm, this could be solved by a little watchdog monitoring changes and triggering a build script.
Again, our little friend python comes to the rescue. A small script can easily detect the changes in the filesystem and in response spawn a build process. This is a longer listing than what I usually show, but it's mainly to show a real live example of code that I run right now and that works.
import os,re import win32file import win32con VERBOSE = False UPDATE_ACTION = 3 FILE_LIST_DIRECTORY = 0x0001 BUFFER_SIZE = 2048 DEFAULT_COMMAND = 'nant 2>&1' EXTENSIONS = [ ".cpp", ".h", ".tga", ".jpg", ".fx", ".tmx", ".wav" ] def findErrorMessage( lines ): pattern = re.compile( "BUILD FAILED" ) for line in lines: if pattern.search(line): return True return False def executeTask( command ): if VERBOSE: print 'Now running the command: "%s"' % command pipe = os.popen( command ) lines = pipe.readlines() returnCode = pipe.close() if findErrorMessage(lines): returnCode = 1 if not returnCode: if VERBOSE: print "".join(lines) return True print "".join(lines) return False def watchDirectory( path, commandLine ): hDir = win32file.CreateFile ( path, FILE_LIST_DIRECTORY, win32con.FILE_SHARE_READ | win32con.FILE_SHARE_WRITE, None, win32con.OPEN_EXISTING, win32con.FILE_FLAG_BACKUP_SEMANTICS, None ) changeCount = 0 while 1: results = win32file.ReadDirectoryChangesW ( hDir, BUFFER_SIZE, True, win32con.FILE_NOTIFY_CHANGE_LAST_WRITE, None, None ) for action, file in results: fullFilename = os.path.join (path, file) name, ext = os.path.splitext(fullFilename) actionName = "Unknown" if action == UPDATE_ACTION: actionName = "Changed" if ext.lower() not in EXTENSIONS: if VERBOSE: print "%s %s ignored." % (fullFilename, actionName) continue if VERBOSE: print "Change %i: %s %s " % (changeCount, fullFilename, actionName) if executeTask( commandLine ): print "Ran build %i successfully." % changeCount else: print "Build failed!" changeCount = changeCount + 1 if __name__ == "__main__": watchDirectory( ".", DEFAULT_COMMAND )
The script starts to listen to changes in the current directory. Whenever it catches one, it checks against a known set of file extensions. If a match is generated, it calls the build task and let that one figure out how to rebuild whatever needs to be rebuilt.
All the voodoo magic happens in the two calls win32file.CreateFile and win32file.ReadDirectoryChangesW. These are the standard win32 functions for detecting changes in the filesystem. The parameters to the functions are of course standard win32, that means a lot of them and half of them redundant. MSDN have the full documentation on the functions for the curious.
Yes, I use nant to build the data. Nant got its issues, but for my little hobby project it's certainly sufficient (even a little overkill). How to mold nant into something that you can actually use is enough material for another article however. Sufficient to say, you could plug in any build system here, simply change the variable DEFAULT_COMMAND.
So, what are the benefits of this little script? The first thing is that it offloads logic from the game itself. We now have a chance to do our preprocessing in smaller tools before we load assets into the game. At which point we can assume that they don't contain any errors, since the small buildscript have caught them and rejected the new resources. This let us keep to the philosophy that the runtime should not need to handle bad assets, because it's impossible for them to make it that far.
In my little project at home, hotloading is certainly overkill. There are certainly some benefits to hotloading, in server/client architectures it's almost a requirement for any development speed. Since the data path I have for most of the assets at home is really short, there is only one path. One might consider to have a quick and dirty path alongside the regular path for previewing and hotloading for games with larger data sets, or more pre-calculating going on.