How to debug your NDS rom using the Visual Studio Debugger

I have been working on Zelda DS for over a year now, and have loved the experience of using the homebrew tools provided by devkitPro. However, as the project grew, I quickly started to miss using a source-level debugger to help me debug and fix problems in the code. There is a debugger that you can download as part of devkitPro named Insight; but it's a fairly finicky application, and is a far cry from the powerful debugger I'm used to using on a daily basis: the Microsoft Visual Studio debugger.

Well, the good news is, after much research and playing around, I found a way to use the Visual Studio debugger to debug homebrew NDS roms! I've decided to share these steps with you in this tutorial:


Requirements

You must have the following software installed:

• Visual Studio: A full copy of Visual Studio, not an Express edition, I use version 2008, but 2005 and 2010 should work fine. The reason it can't be the Express version of Visual Studio is that it does not support plugins, and the next requirement is a plugin...

• WinGDB: This excellent plugin allows you to debug gcc compiled programs with gdb within Visual Studio. It's not free, however before you decide whether to purchase it, you can download a fully functional 30-day trial.

• DeSmuME: The best DS emulator available, and the only one that implements a gdb stub that allows you to use gdb to debug an NDS rom. Thankfully, this piece of software is not only free, but open source as well!

Setup

Visual Studio Makefile Project

The first step is creating a Visual Studio Makefile project that simply invokes the makefile for your homebrew NDS application, and often includes the source files if you wish to use Visual Studio as an editor as well. There is an excellent tutorial on how to do exactly this right here:

http://www.console-dev.de/2009/09/03/create-nintendo-ds-applications-with-visual-studio-2008/

The only difference in my setup is that I don't use no$gba as my emulator, but DeSmuME instead, which I discuss later. In any case, if you can at least build your rom from a Visual Studio project, you're good to go for the next step.

Compiling with Optimizations Disabled

In order to easily debug your NDS rom, you must disable compiler optimizations, otherwise you'll find that stepping through code will jump all over the place, and setting breakpoints won't always stay on the lines you set them. This is normal since the order of generated optimized code will often not match the order you specified in the source code.

The devkitPro templates for NDS applications enable optimizations by default. To disable them, open the Makefile, and find the line similar to:

CFLAGS := -g -Wall -O2\
  -fomit-frame-pointer\
-ffast-math \
$(ARCH)

To disable optimizations, you must minimally remove the -O2. Personally, I also remove the -fomit-frame-pointer and the -ffast-math to make sure I don't get wonky behaviour when stepping through code, but that may be overkill.

Once you've modified the Makefile, make sure to rebuild your rom. An incremental build will not work since no dependencies have changed.

As a side note, you'll probably want to reenable optimizations for your final NDS rom, especially if you notice a performance hit from having disabled them. Having to do this by manually editing the Makefile every time can be tedious. Personally, I made many modifications to the Makefiles that allow me to pass in a configuration (DEBUG or RELEASE), which modifies not only the compiler flags, but also the output folder for object files, and the name of the rom itself. I may post a tutorial on how to do this someday.

WinGDB Setup

Assuming you've installed Visual Studio and WinGDB correctly, you should see an menu entry for WinGDB in Visual Studio:

Select WinGDB → Preferences, General tab, the only option you'll need to set is the Default debugger path to the arm-eabi-gdb.exe in your devkitPro installation, which in my case is "C:\devkitPro\devkitARM\bin\arm-eabi-gdb.exe":

Click OK to close the dialog. Now we need to set the project-specific properties for WinGDB. To do this, in the Solution Explorer, right-click on your project and select WinGDB → Properties:

Now set the following fields in the dialog that opens:

General tab, Target type: Embedded Linux system (gdbserver)

Debug tab:

Executable path: here you must set the path to the arm9 elf file that gets built for your application. The arm9 elf file contains the code, data, and debug information for the arm9 processor, and is usually what you're interested in debugging. Note that if you used the arm9 template from devkitPro (i.e. c:\devkitPro\examples\nds\templates\arm9), there should be only one elf file that gets built next to your nds file. However, if you used the combined template (i.e. c:\devkitPro\examples\nds\templates\combined), you'll find the elf file within the arm9 subfolder with extension .arm9.elf.

Stop in main(): yes (I find it useful, but it's really up to you)

Byte order: little endian

And finally, on the Target tab, set Target specification to: remote localhost:20000
What this value means is that we want gdb to connect to a remote gdb stub that is listening on the local machine's port 20000, which is what we'll set up DeSmuME to do.

Now click OK to save your settings. For the curious, these settings are saved in your project's .suo (user options) file.


Debugging

Launch DeSmuME

Now it's time to debug your rom! The first step is to launch the development version of DeSmuME with a specific argument telling it to enable the gdb stub code and to listen on localhost:20000. For example, I run the following command:

c:\emus\desmume\DeSmuME_dev.exe --arm9gdb=20000 C:\coding\ZeldaDS\ZeldaDS_d.nds

A few things to note:

1. You must use the "_dev" executable
2. We specify that we want to debug the arm9 processor. You can instead specify arm7gdb as well if you wish (and set WinGDB to debug the arm7 elf file instead).
3. The port number, 20000, must match the one set in WinGDB. You can change this value as long as the two match.

This will launch DeSmuME with a console window, and if all went well, the console should report that your nds was loaded succesfully, Emulation unpaused, and the main DeSmuME window should not yet be running your game. DeSmuME is effectively waiting for gdb to connect to it.

Note: since you'll be doing this often, it's a good idea to create a batch file that runs the above command line. What I did was add a an External Tool to the Tools menu in Visual Studio so that I can easily launch DeSmuME within the IDE.

Launch Visual Studio Debugger via WinGDB

With your project loaded in Visual Studio, from the menu select WinGDB → Start Debugging and if all goes well, Visual Studio should switch into debugging mode, and you should eventually break on the first line of main()!

From here, you can do pretty much anything you can normally do in the Visual Studio Debugger, which includes, but is not limited to:

• Setting breakpoints (conditional, tracepoints, etc.)
• Hovering over variables to see their values
• Using the autos, locals, watch, memory and threads windows
• Using the disassembly view to see your source code mixed with the assembly code that was generated
• Setting a breakpoint at arbitrary locations while your application is running (this has never worked for me with other debuggers like Insight or Eclipse).

Conclusion

Phew, that was a little longer than I thought it would be, but hopefully this tutorial will help some of you debug your NDS applications more easily! If you have any problems, or if you feel certain parts of this tutorial could be more clear, don't hesitate to let me know.