Video Game Coding

Tuesday, January 18, 2011

How to debug your NDS rom using the Visual Studio Debugger

I have been working on Zelda DS for over a year now, and have loved the experience of using the homebrew tools provided by devkitPro. However, as the project grew, I quickly started to miss using a source-level debugger to help me debug and fix problems in the code. There is a debugger that you can download as part of devkitPro named Insight; but it's a fairly finicky application, and is a far cry from the powerful debugger I'm used to using on a daily basis: the Microsoft Visual Studio debugger.

Well, the good news is, after much research and playing around, I found a way to use the Visual Studio debugger to debug homebrew NDS roms! I've decided to share these steps with you in this tutorial:


Requirements

You must have the following software installed:

  • Visual Studio: A full copy of Visual Studio, not an Express edition, I use version 2008, but 2005 and 2010 should work fine. The reason it can't be the Express version of Visual Studio is that it does not support plugins, and the next requirement is a plugin...
  • WinGDB: This excellent plugin allows you to debug gcc compiled programs with gdb within Visual Studio. It's not free, however before you decide whether to purchase it, you can download a fully functional 30-day trial.
  • DeSmuME: The best DS emulator available, and the only one that implements a gdb stub that allows you to use gdb to debug an NDS rom. Thankfully, this piece of software is not only free, but open source as well!

Setup

Visual Studio Makefile Project

The first step is creating a Visual Studio Makefile project that simply invokes the makefile for your homebrew NDS application, and often includes the source files if you wish to use Visual Studio as an editor as well. There is an excellent tutorial on how to do exactly this right here:

The only difference in my setup is that I don't use no$gba as my emulator, but DeSmuME instead, which I discuss later. In any case, if you can at least build your rom from a Visual Studio project, you're good to go for the next step.


Compiling with Optimizations Disabled

In order to easily debug your NDS rom, you must disable compiler optimizations, otherwise you'll find that stepping through code will jump all over the place, and setting breakpoints won't always stay on the lines you set them. This is normal since the order of generated optimized code will often not match the order you specified in the source code.

The devkitPro templates for NDS applications enable optimizations by default. To disable them, open the Makefile, and find the line similar to:

CFLAGS := -g -Wall -O2\
  -fomit-frame-pointer\
-ffast-math \
$(ARCH)

To disable optimizations, you must minimally remove the -O2. Personally, I also remove the -fomit-frame-pointer and the -ffast-math to make sure I don't get wonky behaviour when stepping through code, but that may be overkill.

Once you've modified the Makefile, make sure to rebuild your rom. An incremental build will not work since no dependencies have changed.

As a side note, you'll probably want to reenable optimizations for your final NDS rom, especially if you notice a performance hit from having disabled them. Having to do this by manually editing the Makefile every time can be tedious. Personally, I made many modifications to the Makefiles that allow me to pass in a configuration (DEBUG or RELEASE), which modifies not only the compiler flags, but also the output folder for object files, and the name of the rom itself. I may post a tutorial on how to do this someday.



WinGDB Setup

Assuming you've installed Visual Studio and WinGDB correctly, you should see an menu entry for WinGDB in Visual Studio:



Select WinGDBPreferences, General tab, the only option you'll need to set is the Default debugger path to the arm-eabi-gdb.exe in your devkitPro installation, which in my case is "C:\devkitPro\devkitARM\bin\arm-eabi-gdb.exe":



Click OK to close the dialog. Now we need to set the project-specific properties for WinGDB. To do this, in the Solution Explorer, right-click on your project and select WinGDB → Properties:



Now set the following fields in the dialog that opens:

General tab, Target type: Embedded Linux system (gdbserver)



Debug tab:

Executable path: here you must set the path to the arm9 elf file that gets built for your application. The arm9 elf file contains the code, data, and debug information for the arm9 processor, and is usually what you're interested in debugging. Note that if you used the arm9 template from devkitPro (i.e. c:\devkitPro\examples\nds\templates\arm9), there should be only one elf file that gets built next to your nds file. However, if you used the combined template (i.e. c:\devkitPro\examples\nds\templates\combined), you'll find the elf file within the arm9 subfolder with extension .arm9.elf.

Stop in main(): yes (I find it useful, but it's really up to you)

Byte order: little endian



And finally, on the Target tab, set Target specification to: remote localhost:20000
What this value means is that we want gdb to connect to a remote gdb stub that is listening on the local machine's port 20000, which is what we'll set up DeSmuME to do.



Now click OK to save your settings. For the curious, these settings are saved in your project's .suo (user options) file.


Debugging

Launch DeSmuME

Now it's time to debug your rom! The first step is to launch the development version of DeSmuME with a specific argument telling it to enable the gdb stub code and to listen on localhost:20000. For example, I run the following command:

c:\emus\desmume\DeSmuME_dev.exe --arm9gdb=20000 C:\coding\ZeldaDS\ZeldaDS_d.nds

A few things to note:

  1. You must use the "_dev" executable
  2. We specify that we want to debug the arm9 processor. You can instead specify arm7gdb as well if you wish (and set WinGDB to debug the arm7 elf file instead).
  3. The port number, 20000, must match the one set in WinGDB. You can change this value as long as the two match.
This will launch DeSmuME with a console window, and if all went well, the console should report that your nds was loaded succesfully, Emulation unpaused, and the main DeSmuME window should not yet be running your game. DeSmuME is effectively waiting for gdb to connect to it.

Note: since you'll be doing this often, it's a good idea to create a batch file that runs the above command line. What I did was add a an External Tool to the Tools menu in Visual Studio so that I can easily launch DeSmuME within the IDE.


Launch Visual Studio Debugger via WinGDB

With your project loaded in Visual Studio, from the menu select WinGDB → Start Debugging and if all goes well, Visual Studio should switch into debugging mode, and you should eventually break on the first line of main()!

From here, you can do pretty much anything you can normally do in the Visual Studio Debugger, which includes, but is not limited to:
  • Setting breakpoints (conditional, tracepoints, etc.)
  • Hovering over variables to see their values
  • Using the autos, locals, watch, memory and threads windows
  • Using the disassembly view to see your source code mixed with the assembly code that was generated
  • Setting a breakpoint at arbitrary locations while your application is running (this has never worked for me with other debuggers like Insight or Eclipse).

Conclusion

Phew, that was a little longer than I thought it would be, but hopefully this tutorial will help some of you debug your NDS applications more easily! If you have any problems, or if you feel certain parts of this tutorial could be more clear, don't hesitate to let me know.

9 comments:

  1. Wow man, awesome article! Well done!

    ReplyDelete
  2. Yay, now lets see it work on the actual ds over wifi/serial comms! :)

    Good stuff though, i have played with it before.

    -Dovoto

    ReplyDelete
  3. Hi!
    I found this guide very interesting, but I'm unable to run the project with the debugger without problems:
    When I select "Start Debugging" from WinGDB, it tell me that it can't connect to port 20000 because it can't connect (this is because it won't run the emulator and after start the debugging). So I tried to run the emulator with the debugger and started the WinGDB debugger: ok, it works, the cursor is over the first line, but with every command from the IDE, the result is "Protocol error: Z0 (software-breakpoint) conflicting enabled responses.". Can you tell me some advice to run the debugger properly?
    Thank you so much for the guide!

    PS. But the development of ZeldaDS?

    ReplyDelete
  4. Xeeynamo: as I explained in my tutorial, you must launch DeSmuME manually first before you can launch the WinGDB debugger, as you figured out on your own. In fact, to make this a little smoother, I added the path "c:\emus\desmume\DeSmuME_dev.exe --arm9gdb=20000 C:\coding\ZeldaDS\ZeldaDS_d.nds" to my Tools menu in Visual Studio, so I can quickly launch it from the menu.

    As for your protocol error, I'm not quite sure what that could be. I've never seen that before. Are you sure you're following all the steps exactly? Perhaps go over the tutorial again and make sure everything is setup correctly. Good luck!

    Zelda DS is currently on hold due to things having gotten much busier in my personal life. I'm not sure when I'll get back to it, but when I do, I'll be posting about it right here!

    ReplyDelete
  5. Can i ask for a copy of WinGDB 30 day trial, the website isnt letting me through, its releasing a 403 Forbidden for all of its pages, oh and if I cant do that, can you link me to it.. OR tell me if this website has the same tools:
    http://www.windbg.org/

    ReplyDelete
    Replies
    1. Hi,

      The link you posted in your comment is not correct, it's wingdb, not windbg. Here's a direct link to the installer for the 30-day trial:
      http://www.wingdb.com/WinGDB-latest_trial.msi

      Of course, it has been a few years since I posted this tutorial, and I suspect the product has changed since then, so I'm not sure if the steps are accurate anymore. Let me know if you are able to make it work. Thanks!

      Delete
  6. Hey, i managed to find another download, and i also found that site and its down.. go to it and it just 403s everything. So i had to search all over the net.. Oh and also I use the Drunkencoders Project wizard to create the NDS projects, because nearly every tutorial there to create projects was either down or for VS 2005 and i have VS 2008 pro

    ReplyDelete
    Replies
    1. Odd about you getting 403s as I don't have that problem. Perhaps it's a regional thing. In any case, I'm glad you were able to find a copy. Have you been able to successfully debug an NDS rom this way yet?

      Delete
  7. It is perfect time to make some plans for the future and it is time to be happy. I've read this post and if I could I desire to suggest you some interesting things or suggestions. Perhaps you could write next articles referring to this article. I want to read more things about it! best ds emulator

    ReplyDelete