Thoughts on development tools and the future

I had a conversation with a friend recently about why Sony had really failed to match the quality of Microsofts devtools.

The answer is pretty simple, for the most part they are hacking their devtools into Visual Studio and copying what MS already has. At some level it's all they can do, as a developer I don't want two different dev environments for two different consoles, I have to switch back and forth too often for that. The reason I finally gave up on my seperate text editor and started editing files inside Visual Studio was because the VS brief emulation was bad enough that editing files in both places was inconsistent and irritating.

But the conversation got me thinking about how tools like Visual Studio haven't scaled well with the size and complexity of modern games, andin fact how much more painful development is now (despite the infinitely better tools) than it ever has been.

The issue is the cycle time, I make a change/add a feature, compile, link run and test. For me currently excluding the editting that's at least 3 minutes, and as much as 20 depending on what files I touch in the codebase. That makes any sort of itterative development hideously painful. We work around by having hot loaded tuning files, and exposing variables that we want to itterate on in those files, but that doesn't cover a lot of code specific cases.

Some of the pain is self inflicted, our startup time to get into the game on dev builds is pretty bad, because we generally develop with thousands of "loose files", loaded off the disc to make changing them easier, where as we ship with them packed into larger packages.

But a big chunk is in compilation and linking.

We use incredibuild, though it's only really a win in a rebuild all case. We could probably improve link times some by cleaning up some of the dependencies between source files and we do sweep files for un-needed header dependencies on occasion. Making some of our libraries DLL's would probably help the link time. But why is it even necessary?

In the Genesis/SNES days, for the most part I could edit, assemble and run in less than 10 seconds. OK I'm comparing less than 100K lines of assembler to over 1M lines of C++, but we need to find a solution.

And it's not just the compile/link time, managing all the files in the project becomes a challenge, I use Visual Assist and would recommend to to others, it has tools that help navigating code bases, but the some of the more useful ones like goto symbol really only work about 1/2 the time on large codebases.

I've championed the idea of providing artists and designers with powerful tools that let them itterate quickly, even at the expense of runtime performance (although that's a fine line to walk) for probably 10 years now. They produce better content and that makes the technology look good. We need to apply the same sort of thought process to engineering.

MS has made some efforts in this direction, Compile and Continue, is an incredible tool, unfortunately it seems to just stop working on large codebases. They have mechanisms to lookup symbols and go to them, but using it on large codebases leads to huge random pauses when editing.
What's worse is that because MS has concentrated on the feature set of Visual Studio, dealing with large codebases in visual studio has actually gotten worse as it's moved forwards. VS6 is better in this regard than 2003 and 2003 is better than 2005. My understanding is that 2008 is starting to move towards improvements in this area, but I haven't had a chance to test that.

All a user can really do to alleviate the situation is work with less source code. There are ways to do this if you make the right set of design decisions, but when you start making technical decisions based on the inadequacies of the development environment there is something very wrong.

The majority of programmers I know bitch about Visual Studio on a daily basis, most of what it does, it does just about adequately. If someone like a Sony or Whoever really wanted to compete in the devtool arena with MS I think the window is there to do so. For Sony they'd have to stop thinking in terms of PS3 tools and start thinking in terms of core development tools for multiple platforms, cross platform development is not going away and the cost of having to use two dev environments is just too high. You don't need the feature set of Visual Studio to beat it, you just need to do the useful things well, and let people do all there work inside it. If it were me I'd concentrate on any mechanism to let programmers itterate quickly on large codebases, and make it your differentiating feature.