In this article I show what you can do with this information and why it is useful during development.

libnds exceptionTest example

If you installed the libnds examples, you should find the exeptionTest example in devkitPro\examples\nds\debugging\exceptionTest.

Copy&Paste from exceptionTest.c:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30

#include <nds.h>
 
/*---------------------------------------------------------------------------------
 
 The default exception handler displays the exception type - data abort or undefined
 instruction you can relate the exception to your code using
 
    arm-eabi-addr2line -e <elf file> <address>
 
 assuming you built with debug info this will display a source file and a line number
 The address of the instruction is shown as pc, beside the address which faulted
 the rest of the screen is a dump of the registers.
 
---------------------------------------------------------------------------------*/
 
//---------------------------------------------------------------------------------
int main(void)
{
  // install the default exception handler
  defaultExceptionHandler();
 
  // generate an exception
  *(unsigned int*)0xFA = 0x64;
 
  // never comes here
  while(1)
    swiWaitForVBlank();
 
  return 0;
}

When you build and run the example project, it displays:

This is actually a fake screenshot. I rebuilt the Guru Meditation Error with printf, because the example didn’t behave in any emulator as on actual hardware. I tested no$gba 2.6a, iDeaS 1.0.3.3 beta and DeSmeME 0.9.5, none of them displayed the error report as on the real Nintendo DS hardware.

guru meditation origin

Guru Meditation is the name of the error that occurred on early versions of the Commodore Amiga computer when they crashed. It is analogous to the “Blue Screen Of Death” in Microsoft Windows operating systems.

The Guru Meditation error message was a black screen with the following animation at the top:

decode nds guru meditation data abort message

One of the Nintendo DS variants of the Guru Meditation error message is the data abort, which includes the following information:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24

     Guru Meditation Error!    
          data abort!          
 
  pc: 02000386 addr: 000000FA  
 
  r0: 020005BD   r8 : 00000000 
  r1: 00000000   r9 : 00000000 
  r2: 00000064   r10: 00000000 
  r3: 000000FA   r11: 00000000 
  r4: 0B000000   r12: 02000000 
  r5: 00000000   sp : 0B003CF8 
  r6: 00000000   lr : 020005B5 
  r7: 00000000   pc : 0200038E 
 
  0B003CF8: 0B000000 02000180  
  0B003D00: 00000000 00000000  
  0B003D08: 00000000 00000000  
  0B003D10: 00000000 00000000  
  0B003D18: 00000000 00000000  
  0B003D20: 00000000 00000000  
  0B003D28: 00000000 00000000  
  0B003D30: 00000000 00000000  
  0B003D38: 00000000 00000000  
  0B003D40: 00000000 00000000

The first line is always “Guru Meditation Error!”, followed by the type of the exception:

1
2

     Guru Meditation Error!    
          data abort!

The libnds default exception handler displays “Data Abort” and “Undefined Instructions” exceptions.

Below the exception type is pc and addr displayed, which I guess is not very obvious for many novice programmers:

1

  pc: 02000386 addr: 000000FA

pc is short for Program Counter. The Program Counter register indicates where the processor is in its instruction sequence. In this case, it’s the address of the instruction that caused the exception.

addr represents the exception address. In this case, writing to the memory address 0xFA:

1
2

  // generate an data abort exception
  *(unsigned int*)0xFA = 0x64;

This again is followed by a register dump:

1
2
3
4
5
6
7
8

  r0: 020005BD   r8 : 00000000 
  r1: 00000000   r9 : 00000000 
  r2: 00000064   r10: 00000000 
  r3: 000000FA   r11: 00000000 
  r4: 0B000000   r12: 02000000 
  r5: 00000000   sp : 0B003CF8 
  r6: 00000000   lr : 020005B5 
  r7: 00000000   pc : 0200038E

• r0 – r12 are the so called General Purpose Registers.
• sp (r13) is the Stack Pointer Register, which indicates the current top of stack memory.
• lr (r14) represents the Link Register, which holds the return address when the current function completes (it’s the address of the caller).
• pc (r15) is short for Program Counter. The Program Counter register indicates where the processor is in its instruction sequence.

The last information in the error report is a memory dump of the top 80 bytes of stack memory:

1
2
3
4
5
6
7
8
9
10

  0B003CF8: 0B000000 02000180  
  0B003D00: 00000000 00000000  
  0B003D08: 00000000 00000000  
  0B003D10: 00000000 00000000  
  0B003D18: 00000000 00000000  
  0B003D20: 00000000 00000000  
  0B003D28: 00000000 00000000  
  0B003D30: 00000000 00000000  
  0B003D38: 00000000 00000000  
  0B003D40: 00000000 00000000

• First column specifies the stack address (0B003CF8)
• Second column shows the 32bit value from the stack address (0B000000)
• Third column shows the 32bit value from the stack address + 4 bytes (02000180)

how to make use of all this

This information is informative only when the output file was built with debug information. You can include debug information by adding the -g option (produce debugging information) to CFLAGS in the project makefile:

Copy&Paste from exceptionTest makefile:

1
2
3
4
5

CFLAGS	:= -g -Wall -O2\
		-march=armv5te -mtune=arm946e-s \
 		-fomit-frame-pointer\
		-ffast-math \
		$(ARCH)

devkitARM includes an application named “arm-eabi-addr2line.exe”, located in devkitPro\devkitARM\bin, that can be used to convert addresses into line number/file name pairs. It supports the following command line options:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15

Usage: arm-eabi-addr2line.exe [option(s)] [addr(s)]
 Convert addresses into line number/file name pairs.
 If no addresses are specified on the command line, they will be read from stdin
 
 The options are:
  @<file>                Read options from <file>
  -b --target=<bfdname>  Set the binary file format
  -e --exe=<executable>  Set the input file name (default is a.out)
  -i --inlines           Unwind inlined functions
  -j --section=<name>    Read section-relative offsets instead of addresses
  -s --basenames         Strip directory names
  -f --functions         Show function names
  -C --demangle[=style]  Demangle function names
  -h --help              Display this information
  -v --version           Display the program's version

We are interested in the address where the Data Abort exception occured, named pc here:

1
2
3
4

     Guru Meditation Error!    
          data abort!          
 
  pc: 02000386 addr: 000000FA

I used the following commandline for conversion:

1

arm-eabi-addr2line.exe -f -C -e  -s -i exceptionTest.elf 02000386

The output is:

1
2

main
exceptionTest.c:23

When you scroll at the beginning of this article, where I present the source code of the exceptionTest example, you see line 23 is inside main(), which is stored in exceptionTest.c and contains the code that writes to the address 0xFA.

conclusion

Make sure to install an exception handler in your projects. It’s very helpful to actually see that an error occured rather than a freeze. libnds makes it enormously easy, it’s just one line of code!

In this article I show how to set up Microsoft Visual Studio 2008 to use the devkitARM compiler to create applications for the Nintendo DS. After you have read this article, you are able to use Visual Studio 2008 to build, rebuild, clean and execute your Nintendo DS application.

Visual Studio 2008 Express can be downloaded for free at microsoft.com, you want to have the C++ edition: Download Visual C++ 2008 Express Edition.

prerequisites

I assume you already have devkitARM and Visual Studio 2008 installed and both function correctly. I further assume you have a project that you want to migrate from e.g. VisualHAM, Programmers Notepad, etc… to Visual Studio, so I use the hello world example from libnds as existing project reference, which is located in devkitPro\examples\nds\hello_world.

create new visual studio project

We need to create an empty makefile project. You can find this in Visual Studio 2008 under Main menu -> New -> Project:

Select the project template and target location:

Rather than selecting the hello_world directory, we specify its parent, because Visual Studio creates a sub directory in the specified location with the specified name. This configuration will save the project file directly in the existing hello_world directory.

Click OK to continue.

configure new project

A new dialog pops up, the New Project Wizard:

Just click Next here to move on to the next page.

The following page looks more like fun! Settings specified in this New Project Wizard can be changed at any time in the Project Settings as well:

Please note that none of these settings have influence on the build process: Output, Preprocessor definitions, Include search path, Forced include files, CMR assembly search path and Forced using .NET assemblies.

We set Preprocessor definitions so Visual Studio knows which ARM9/ARM7 #if’s to syntax highlight differently. This is not passed to devkitARM.

We set Include search path so Visual Studio knows where to look for the libnds headers to provide code completion and intellisense, it will not add it to the search path used by devkitARM.

Here is an example what code completion and intellisense look like, first code completion:

Intellisense:

The remaining commands in the New Project Wizard are what you typically use when working via the command prompt or shell. Now click Finish to complete the Wizard.

The New Project Wizard closes and Visual Studio shows an empty workspace now:

add files to project

It’s essential to know that Visual Studio will just invoke the commands specified in the New Project Wizard to build the project.

Compiling and linking your Nintendo DS application is entirely handled by devkitARM, NOT Visual Studio!

Visual Studio is only the graphical front-end that triggers these devkitARM build processes and serves as code editor.

The devkitPro makefile system has a feature to add directories that are supposed to contain source code to the so called SOURCES variable. All source files in those directories, added to this variables, are being compiled and considered during the link process.

Copied from devkitPro\examples\nds\hello_world\Makefile:

1
2
3
4
5
6
7
8
9
10

#---------------------------------------------------------------------------------
# TARGET is the name of the output
# BUILD is the directory where object files & intermediate files will be placed
# SOURCES is a list of directories containing source code
# INCLUDES is a list of directories containing extra header files
#---------------------------------------------------------------------------------
TARGET		:=	$(shell basename $(CURDIR))
BUILD		:=	build
SOURCES		:=	gfx source data  
INCLUDES	:=	include build

This means that source files don’t get compiled when you add them to the Visual Studio project, but when you store them in the source directory of your file system!

add files to project, really

Now that you know your files don’t get compiled when you only add them to the Visual Studio Project, let’s really add them. You can do this via drag&drop operations from the Windows Explorer or via the Visual Studio Project context menu:

A new “File Open” dialog appears, which you have to navigate to your source files, select them and click Add. I recommend you add the makefile too.

Once done, your Visual Studio Project should look similar to this:

compile project

The magic moment has arrived, to compile the project you can use the Build sub-menu from the mainbar or hit CTRL+SHIFT+B:

When everything is configured properly, the project should build just fine and the output looks like:

1
2
3
4
5
6
7
8
9
10
11
12

1>------ Build started: Project: hello_world, Configuration: Debug Win32 ------
1>Performing Makefile project actions
1>main.cpp
1>arm-eabi-g++ -MMD -MP -MF /c/devkitPro/examples/nds/hello_world/build/main.d -g -Wall -O2 -march=armv5te -mtune=arm946e-s -fomit-frame-pointer -ffast-math -mthumb -mthumb-interwork -I/c/devkitPro/examples/nds/hello_world/include -I/c/devkitPro/examples/nds/hello_world/build -I/c/devkitPro/libnds/include -I/c/devkitPro/libnds/include -I/c/devkitPro/examples/nds/hello_world/build -DARM9 -fno-rtti -fno-exceptions -c /c/devkitPro/examples/nds/hello_world/source/main.cpp -o main.o
1>linking hello_world.elf
1>built ... hello_world.arm9
1>Nintendo DS rom tool 1.41 - May  1 2009
1>by Rafael Vuijk, Dave Murphy, Alexei Karpenko
1>built ... hello_world.nds
1>Build log was saved at "file://c:\devkitPro\examples\nds\hello_world\Debug\BuildLog.htm"
1>hello_world - 0 error(s), 0 warning(s)
========== Build: 1 succeeded, 0 failed, 0 up-to-date, 0 skipped ==========

compile project, the Vista barrier

I use Microsoft Vista and have to run Visual Studio 2008 as administrator, otherwise it outputs this error:

1
2
3
4
5
6
7
8
9

1>------ Build started: Project: hello_world, Configuration: Debug Win32 ------
1>Performing Makefile project actions
1>      0 [main] us 0 init_cheap: VirtualAlloc pointer is null, Win32 error 487
1>AllocationBase 0x0, BaseAddress 0x71110000, RegionSize 0x2C0000, State 0x10000
1>c:\devkitPro\msys\bin\make.exe: *** Couldn't reserve space for cygwin's heap, Win32 error 0
1>Project : error PRJ0019: A tool returned an error code from "Performing Makefile project actions"
1>Build log was saved at "file://c:\devkitPro\examples\nds\hello_world\Debug\BuildLog.htm"
1>hello_world - 1 error(s), 0 warning(s)
========== Build: 0 succeeded, 1 failed, 0 up-to-date, 0 skipped ==========

If you encounter the same problem, see if it fixes the problem when you start Visual Studio 2008 with administrator privileges. You can do this when you right-click the Visual Studio 2008 startmenu entry and select “Run as administrator”.

add a rebuild target

We specified “make rebuild” in the New Project Wizard, but this target does not exist by default, so we have to create it first. Basically it should perform:

1
2

make clean
make

Open the makefile and add this below clean:

1

rebuild: clean $(BUILD)

Select Build -> Rebuild Solution and see the output if it’s performing correctly.

add run target

So far we were able to compile, clean and rebuild the Nintendo DS project out of Visual Studio, but how to run it? We use a software emulator! I use no$gba to run my Game Boy Advance and Nintendo DS applications, so I want Visual Studio to start no$gba as well!

We do this like we did with rebuild, we add a new target to makefile which is responsible to start and pass the filename to the Nintendo DS emulator, in my case no$gba.

Put this below our earlier added rebuild target:

1
2

run: $(BUILD)
	$(DEVKITPRO)/utilities/nogba/nogba.exe $(TARGET).nds

The tab at the beginning at the 2nd line is important and must not be removed, you also want to modify the path to where the emulator of your choice is located.

Now open the Visual Studio Project Properties:

Select Debugging in the following dialog and fill out the Command and Command Argument text boxes:

Confirm the dialog with OK and select Debug -> Start Debugging, the project should build and start in the specified emulator!

Visual Studio compatible build messages

All build messages are redirected to the Visual Studio output “Build” pane. Warnings and errors include the filename and line number, but those are in the GCC format and when you double click them, Visual Studio is smart enough to open this file, but not to go to the line.

1

1>c:/devkitPro/examples/nds/hello_world/source/main.cpp:17: error: expected constructor, destructor, or type conversion before 'void'

Double clicking this line opens main.cpp, but keeps the cursor at line 1.

Fortunalety, Jasper ‘cearn’ Vijn once had a regular expression on his website that transforms GCC messages in to Visual Studio format and I was clever enough to make a backup before the information went offline:

1

's/\(\w\+\):\([0-9]\+\):/\1(\2):/'

We can redirect the build output to sed.exe (stream editor), which uses this regular expresssion to transform and output the text so Visual Studio can handle it.

We do this by extending the commands earlier specified in the New Project Wizard.

In the following dialog select NMake and modify the settings as shown below:

Build Command line:

1

make 2>&1 | sed -e 's/\(\w\+\):\([0-9]\+\):/\1(\2):/'

Rebuild All Command line:

1

make rebuild 2>&1 | sed -e 's/\(\w\+\):\([0-9]\+\):/\1(\2):/'

If you have never seen this 2>&1 before, it’s called command redirection operator and a feature of Microsoft Windows. >& writes the output from one handle to the input of another handle. Next comes |, the pipe operator. It takes the output of one command and directs it into the input of another command.

Confirm the Project Properties Dialog with OK, add an invalid line of code to one source file and build. The error message should look like this now and is clickable:

1

1>c:/devkitPro/examples/nds/hello_world/source/main.cpp(17): error: expected constructor, destructor, or type conversion before 'void'

we are done

Happy compiling!

I was experimenting how I could retrieve a stack trace on the Nintendo DS, using the devkitARM compiler, recently. A callstack is a list of active subroutines of a computer program. This can be used to identify from which code-path a particular subroutine was called, which is enormously helpful when detecting logical errors at run-time.

I tend to flood my code with “ASSERT’s”. ASSERT is used to identify logic errors during program development by implementing the expression argument to evaluate to FALSE only when the program is operating incorrectly and then calling a function to report the error:

1
2
3
4
5
6
7
8
9

void Vector2Add(Vector2 *dest, const Vector2 *v0, const Vector2 *v1)
{
  ASSERT(dest != NULL);
  ASSERT(v0 != NULL);
  ASSERT(v1 != NULL);
 
  dest->x = v0->x + v1->x;
  dest->y = v0->y + v1->y;
}

When any of the incoming arguments point to NULL, ASSERT will call a subroutine to report the error. Typically the report includes the filename, line number and functionname. This information is helpful, but lacks of information from where the function was called.

I use Vector2Add all over the place, how should I know what part in my program code does not work, when I only know at some point a NULL argument is passed to Vector2Add?

Right, I need to know from where Vector2Add was called, I need a callstack trace!

my earlier instrumentation approach

Around 2005-2006 I implemented a callstack trace feature in HEL Library. HEL Library is a middleware solution to ease development of Game Boy Advance titles.

The callstack trace was displayed together with an assertion-report, that looked like:

Back then I used function instrumentation (-finstrument-functions) to build a list of subroutine calls at run-time. It was basically a callback that was called for every function that is about to be entered (__cyg_profile_func_enter) and left (__cyg_profile_func_exit).

This was a huge overhead and slowed down program-execution dramatically, but the callstack trace was so precious, that the performance penalty didn’t matter for the debug library.

fail does not mean to stop

When I started the Nintendo DS version, one goal was to use an approach which does not have any impact on run-time performance! I have tried several libraries, unfortunalety I didn’t find anything that worked for me.

This includes libunwind, backtrace, several GCC builtin-keywords to get return addresses of different calldepths, just to name a few.

I gave all up. It was either a never ending story to integrate the library in to the project or headers / functions were missing in libraries that come with devkitARM.

hacking to success, then fail

Rather than giving up, I decided to accept it as challenge and come up with my own callstack trace code. It was a long and rocky journey, but at the end I had something that worked suprisingly well.

What I do is basically pretty simple, I interpret instructions of interest to simulate the beaviour of the Program Counter (PC) and Stack Pointer (SP). The Program Counter indicates where the program is in its intruction sequence. Stack Pointer indicates the current top of stack memory. The stack this is where local variables are located.

Armed with my own Program Counter and Stack Pointer variables, I can traverse program code and react when Program Counter is assigned a new value, which basically means to jump to a different address and continue to operate there.

Example:

1
2
3
4
5
6
7
8
9
10

void FirstFunc()
{
  SecondFunc();
}
 
void SecondFunc()
{
  // Program Counter is located here  <--
  ThirdFunc();
}

Let’s presume the Program Counter is located in SecondFunc, which was previously called by FirstFunc.

The thumb assembler version of this looks like:

1
2
3
4
5
6
7
8
9
10
11
12
13

FirstFunc:
  push {lr}       ; Push Link Register (LR) on stack
  sub sp, sp, #4  ; Update Stack Pointer by 4bytes because LR was pushed
  bl  SecondFunc  ; Call to SecondFunc()
  add sp, sp, #4  ; Update Stack Pointer to point to pushed LR again
  pop {pc}        ; Pop LR from stack and store in Program Counter, this will return to caller
 
SecondFunc:
  push {lr}       ; Push Link Register (LR) on stack
  sub sp, sp, #4  ; Update Stack Pointer by 4bytes because LR was pushed
  bl  ThirdFunc   ; Call to ThirdFunc()
  add sp, sp, #4  ; Update Stack Pointer to point to pushed LR again
  pop {pc}        ; Pop LR from stack and store in Program Counter, this will return to caller

I need to get my hands on the Link Register (LR) to return to SecondFunc’s caller (*1). Since LR was being pushed on the stack, I know where to look! What is left is to traverse the program code in reversed order and interpret the “sub” and “push” instructions.

Moving the Program Counter in reversed order, which actually would be a backtrace imo, didn’t work out for some reasons and I wasn’t able to solve those problems. It did, however, worked in some cases, unfortunalety too unreliable.

when in doubt, try it out

I stayed away from this problem for week and then had the idea why not trying to simulate the Program Counter the same way it would walk along when it continues program execution normally. It means rather then traversing program code backwards, I tried what happens, when I forward advance PC. Now I have to interpret “add” and “pop” instructions and this works suprisingly well!

From here on it took a few hours to come up with a version that is robust enough to work in all places that I tested with my code base, which consists of both, C and C++ source code.

Obviously it does not mean it always works in every sitiuation with all source code in the world. For example, optimization level -O0 generates so many unconditional branches and places return code all over the place, that the Stacktrace function fails. It supports thumb code only and has problens with hand written/optimized assembler routines (when LR is not push’ed and PC not pop’ed).

I added a couple of more instruction Stacktrace looks out for:

1
2
3
4
5

pop {...pc} ; adjust PC to the pop'ed value
add sp, nn  ; adjust SP by nn
sub sp, nn  ; adjust SP by nn
b   ofs     ; set PC to the addr PC+ofs
add sp, rn  ;  adjust SP by the value rn points to

The unconditional branch instruction “b” is from hell, because it could lead to an infinite loop in Stacktrace, such as while(1) {}. For this reason I added two special cases:

• Limit how many instructions are allowed to touch
• Branch only to the target address when it’s different from PC

When any of the conditions evaluates to true, Stacktrace aborts and outputs the callstack only to this position. It will, however, report that an error occured.

Support for the unconditional branch was important, because the compiler generates code like this sometimes:

1
2
3
4
5
6
7
8
9
10
11
12

ExampleFunc:
  push   {lr}
  ...
  tst    r0, #2
  bne    .L92 ; if r0 is not 2 then jump to L92
  ...
.L90:
  pop    {pc} ; returns to caller
 
.L92:
  bl     AnotherFunction
  b      .L90 ; jump to L90 (above)

When we return from AnotherFunction we must jump to L90 to pop the return address from the stack and return to the caller. This is the reason why I added support for unconditional branches. I haven’t noticed other branches that are of importance for Stacktrace yet.

At this point of time I was able to return to each function-caller and have the return addresses, e.g:

resolving function names by hand

In order to map the memory address to an actual function, we have to look up the address in the so called Map file. Map files are typically plain text files that indicate the relative offsets of functions for a given version of a compiled binary. This file is generated by the linker when you pass -Map to it, here is a part of it:

1
2
3
4
5
6
7

.text   0x02000310    0xdf28
.text   0x02000380    0x1ac main.o
        0x02000394      PrintStacktrace
        0x0200046c      FirstFunc
        0x0200040c      ThirdFunc
        0x020004c8      main
        0x02000450      SecondFunc

Line 1 specifies where the .text section (program code) starts and its size. Line 2 specifies the start address of the object file main.o followed by all functions it contains. While writing this article, I have learned static functions are not listed in the map file.

In order to find the function name of the memory address 0×02000458 (from screenshot above), we have to check in which range it is located. But it would be too simple when the .map file has everything sorted by addresses, so we have to do this instead.

Here is a sorted list by addresses:

1
2
3
4
5
6

.text   0x02000380    0x1ac main.o
        0x02000394      PrintStacktrace
        0x0200040c      ThirdFunc
        0x02000450      SecondFunc
        0x0200046c      FirstFunc
        0x020004c8      main

The memory address 0×02000458 is between:

1
2

        0x02000450      SecondFunc
        0x0200046c      FirstFunc

So 0×02000458 belongs to SecondFunc. While this approach seems to work, it is complicated and really only works when you have the exact .map file available that was used to build the project. You can’t use a .map file of a different version of your code, this is very unlikely to work.

It would be so much better when the actual function name could be displayed instead!

what is your function name, dear memory address

Fortunalety, the compiler has an option to insert function names in plain-text infront of each function. This can be activated by adding -mpoke-function-name to the CFLAGS variable in your makefile.

But it not only inserts the name, it also inserts a bit-pattern to recognize that there is a function name as well as a relative offset to the name from this pattern.

What Stacktrace function does, it takes a memory address and then decreases this address until it detects the bit-pattern of the function name and we have it!

Well, there is of course a special case again. When -mpoke-function-name has not been specified, the function name bit-pattern is missing, thus Stacktrace would try to find it until it reaches the start of the text section. While this makes little sense, I added a limit how many 32bit words the “GetFunctionnameTag” function is allowed to touch before it returns “did not find”.

how to integrate in your own project

The stacktrace.zip archive contains the entire source code, you only need those two files:

• stacktrace.h
• stacktrace.c

Copy them to your source code directory. Then open the makefile file which should be located in your project directory also. You have to adjust the following variables:

1
2

ARCH   := -mthumb -mthumb-interwork
CFLAGS := -O2 -mpoke-function-name

Now do a rebuild, rather than a regular build. You can do this by performing a “make clean” first, then a “make”. Don’t forget to include stacktrace.h in that file where you want to use Stacktrace.

Optimization level -O2 worked best with Stacktrace so far. When you use more aggressive optimization levels the compiler also starts to inline code heavily, which can be quite confusing when the Stacktrace output does not reflect what you see in your high level code.

stacktrace interface and usage

The usage of Stacktrace function is fairly simple.

All it expects is an array of StacktraceEntry elements and the count of it. Here is the interface from stacktrace.h:

1
2
3
4
5
6
7

typedef struct
{
  unsigned int addr;  //!< Return address
  const char *name;   //!< Name of function
} StacktraceEntry;
 
unsigned int Stacktrace(StacktraceEntry* dest, unsigned int count);

Use it like this:

1
2
3
4
5
6
7
8
9
10
11
12

void OutputStacktrace()
{
  unsigned int n;
  unsigned int count;
  StacktraceEntry entries[32]; // allocate memory for 32 stacktrace entries
 
  // get only the 10 deepest stacktrace entries
  count = Stacktrace(entries, 10);
 
  for (n=0; n<count; ++n)
    printf("0x%08x: %s", entries[n].addr, entries[n].name);
}

improving stacktrace

If you want to improve Stacktrace, I recommend to read through the comments in stacktrace.c and enable STACKTRACE_VERBOSE, by setting it to 1.

This will, as far as OutputDebugString (also in stacktrace.c) uses a debug message type your Nintendo DS software emulator understands, output what instructions Stacktrace comes across and interprets.

I’ve added an opcode structure for all instructions Stacktrace interprets as well as print-routines to dump the contents.

I have used no$gba during development, which was of great help because of its “debugger“. Usually I yell about only see the assembler code in the shareware version of no$gba, but for this project it was exactly what I needed

conclusion

The Stacktrace function…

• works suprisingly well for being such a hack
• does not need debugging information/symbols
• it runs directly on the target device, not needed to be connected to GDB or a similar debugger
• does not come with any run-time performance penalty
• works best with optimization level -O2
• is most informative when adding -mpoke-function-name to CFLAGS in your makefile
• comes with full documented source code
• uses very little Nintendo DS specifics, except the text section start and length (portable to other ARM devices?)
• works with thumb code only
• does not detect when switching from thumb to arm mode
• depends on LR / PC being push’ed and pop’ed, hand-written assembler routine can cause trouble
• is tested in/with my code base only

frequently asked questions (faq)

Q: Why do I see so weird function names, like _ZN13DEMOPARTQUEUE9CutToPartEj and the like?

A: Because in C++, the function names go through name mangling. Name mangling is a technique used to solve various problems caused by the need to resolve unique names for programming entities.

download

Dear visiter, 100% OFF on console-dev.de, take your chance and download stacktrace.zip now!

what I didn’t tell you yet

*1) LR does not always contains the address from where it was being called, but it holds the address where to return when the function completes. When using code optimization levels above -O2 the compiler e.g. generates code that does not return to functions where it was the last instruction involved anyway.

Well, Jasper “cearn” Vijn suggested to use a higher resolution fixed point format for 1.0f/(2*pi) and this solves the problem indeed! But it does not end here!

He also created an awesome article about the ins and outs of fixed point sine approximation and presents a couple of highly precise and optimized routines.

Head over to Jasper Vijn’s document Another fast fixed-point sine approximation now!

I’ll post a corrected version of the f32sin routine from my previous post for completeness, but I highly encourage to vists Jasper’s site and get one of his!

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31

// Gets sin of specified radians.
// Input and output format is fixed point sign.19.12
int32 f32sin(int32 radians)
{
  const int32 PI2_PRECISION_BITS = 20;
  const int32 PI2_RECIPROCAL     = (1.0f / (2*M_PI)) * (1<<PI2_PRECISION_BITS);
 
  // Divide incoming radians by 2*PI, so it is a "normalized"
  // angle between -1 and +1, where 1 equals 2*PI (360 degree).
  int32 value = (radians * PI2_RECIPROCAL) >> PI2_PRECISION_BITS;
 
  // Now that we have our normalized angle,
  // we just discard all numbers which are not in the range -1..+1.
  // Since this is fixed point, a simple AND operation can be used.
  value &= floattof32(1)-1;
 
  // Always wrap normalized angle to -0.5(-PI)..+0.5(+PI)
  if(value > floattof32(0.5f))
    value -= floattof32(1); // subtract 2*PI in normalized form
  else
    if(value < floattof32(-0.5f))
      value += floattof32(1); // add 2*PI in normalized form
 
  // Convert normalized angle (-1..1) back to radians (-2*PI..2*PI)
  value = f32mul(value, floattof32(2*M_PI));
 
  // Approximate sin value
  const int32 B = floattof32(1.2732395447351f);
  const int32 C = floattof32(-0.405284734569f);
  return f32mul(B, value) + f32mul(f32mul(C, value), f32abs(value));
}

I was searching for an approximation method using taylor series and came across the following sites:

• polygonal labs – Fast and accurate sine/cosine approximation
• devmaster.net – Fast and accurate sine/cosine

The only task that remained was porting it from float to fixed point, which is rather trivial, yay!

Most amazing for me is how precise the approximation actually is. Here is a screenshot of the original sinf routine (green) and the approximated one (red) using the source code below:

Sine approximation

The routine from polygon labs also can’t really handle radians greater than 2*PI, which I tried to fix by normalizing it to an angle between -1..+1 and then just use a bit-AND to discard all numbers outside this range.

I wanted to keep the radians format for the incoming parameter, because it’s how the standard sinf routine works. Using another range, like 0..4096 to represent a full rotation would probably solve that precision problem with large radians because this would remove the need for the first two code lines, but I wanted 2*PI.

Here comes the source code, to be compiled with libnds and devkitARM:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44

// Gets sin of specified radians.
// Input and output format is fixed point sign.19.12
int32 f32sin(int32 radians)
{
  // Offset to slightly restore precision when radians is large.
  const int32 offset= f32mul(radians, floattof32(0.00147f));
 
  // Divide incoming radians by 2*PI, so it is a "normalized"
  // angle between -1 and +1, where 1 equals 2*PI (360 degree).
  int32 value = f32mul(radians + offset, floattof32(1.0f / (2*M_PI)));
 
  // Now that we have our normalized angle,
  // we just discard all numbers which are not in the range -1..+1.
  // Since this is fixed point, a simple AND operation can be used.
  value &= floattof32(1)-1;
 
  // Always wrap normalized angle to -0.5(-PI)..+0.5(+PI)
  if(value > floattof32(0.5f))
    value -= floattof32(1); // subtract 2*PI in normalized form
  else
  if(value < floattof32(-0.5f))
    value += floattof32(1); // add 2*PI in normalized form
 
  // Convert normalized angle (-1..1) back to radians (-2*PI..2*PI)
  value = f32mul(value, floattof32(2*M_PI));
 
  // Approximate sin value
  const int32 B = floattof32(1.2732395447351f);
  const int32 C = floattof32(-0.405284734569f);
  return f32mul(B, value) + f32mul(f32mul(C, value), f32abs(value));
}
 
inline int32 f32abs(int32 a)
{
  return a >= 0 ? a : -a;
}
 
// multiplies two fixed point numbers.
// since the result is shifted rather than the two
// operands, make sure a and b are small.
inline int32 f32mul(int32 a, int32 b)
{
  return (a*b)>>12;
}

Let me know when you fixed the precision problem with large radian values

During my daily forum visits, I came across a thread about how to create a game in more than one language.

This reminded me on the first multi-language game I was working on. We ran into several issues back then and I want to share my experience with you, so you don’t make the same mistakes.

When you create a multi-language game, several things change that would be static otherwise. Here is a list a few things to consider:

• You can’t hard-code every text, because you’re usually not the person who translates text.
• You can’t assume texts have same lengths in different languages.
• Prepare your GUI to word-/line-break and squeze text.
• Different languages might imply different character ranges (char, wchar, utf-8, choose which one to support before you start)
• Different languages might imply different font definitions
• Different languages might imply different graphics: title logo, textures with baked-in text, etc
• Different languages might imply different sounds: voice output
• Being able to change language in the options menu, in case it is required.
• Being able to change language at runtime during development.
• Testing time increases with more languages
• You might need to communicate with people who don’t speak your language (bug reports etc)
• Several persons involved in development have more work to do: programmer, translator, quality assurance, (artist, sound artist maybe)

Localization from a programmers perspective

If text is available in more than one language, it must not be hard-coded in the game code.

Functionality to retrieve texts, depending on the current language setting is essential. It can be something simple like a function that expects an identifier that represents the text in question and returns its text or a database query.

We decided to edit texts in Microsoft Excel, because it’s standard in many offices and we don’t wanted to force translators to use some unknown custom in-house tool that he/she might not understand and we would have to support.

The spreadsheet structure looked like:

1
2
3
4
5

ID               EN                  DE            DESC
---------------+-------------------+-------------+---------------
helloworld     | HELLO WORLD       | HALLO WELT  | Displayed in the welcome screen
goodday        | GOOD DAY          | GUTEN TAG   | Displayed when the player confirms the welcome screen
...

ID represents the text identifier that is used to query the translated text.

EN and DE are the english and german translations.

DESC is an optional field that describes the purpose of this text. It can be quite challenging to find a good translation when you don’t know the context of the text, that’s what DESC should solve.

Localization, 1st iteration

Our first language system iteration worked like this:

• Create texts in Microsoft Excel and save as XML
• Custom tool to convert XML to our text format
• Custom tool outputs .cpp file with texts as array
• Custom tool outputs .h file with identifier #define’s that index into the text arrays

In order to get the translated text, we had a function that expects one of the generated identifiers and returned the corresponding text. The text system looked like this:

language.h file:

#ifndef __language_h__
#define __language_h__
 
#define TID_helloworld  0
#define TID_goodday     1
#define TID__MAX        2
 
#define LANG_EN  0
#define LANG_DE  1
 
const char* GetText(unsigned int textId);
#endif // __language_h__

language.cpp file:

#include "language.h"
 
// initial language is english
int CurrentLanguage=LANG_EN;
 
// german texts
const char* const TEXT_DE[]=
{
  "HALLO WELT",
  "GUTEN TAG"
};
 
// english texts
const char* const TEXT_EN[]=
{
  "HELLO WORLD",
  "GOOD DAY"
};
 
// get text of the specified text identifer
const char* GetText(unsigned int textId)
{
  assert(textId < TID__MAX); // invalid text id
 
  switch(CurrentLanguage)
  {
    case LANG_DE:
      return TEXT_DE[textId];
 
    default:
      break;
  }
 
  return TEXT_EN[textId];
}

Once we worked with it for a while, we realised updating the language file is a time killer.

Everytime we modified text and exported, the header file changed, thus all source files that include language.h were recompiled due to header dependencies. This was a huge problem for us, nobody wanted to wait several minutes for a recompile, only because someone fixed a typo in the translation or added a new text.

Localization, 2nd iteration

What we learned from the 1st approach is no matter if we change, add or even remove text, it must not have a significant influence on compile times.

Rather than using generated identifiers, we used crc32’s of string literals to identify texts. This completely removed the header dependency / recompile problem! Our text system now worked like:

• Create texts in Microsoft Excel and save as XML
• Custom tool to convert XML to our text format, verify that all text identifiers generate unique checksums
• Custom tool outputs .cpp file with texts as array and checksum lookup table

In order to get a tranlated text, GetText() now expects a string literal as id, generates a crc32 of the incoming id, performs a binary search on the checksum table and then uses the lookup position to index into the language array.

This even allowed us to return the text id when the translated text was not found, so the tester could add the text id of the translated text that is missing to the bug report. But it also allowed us to switch text at runtime to display the id rather than text (”uh what text id is display here” belongs to the past).

The new text system was a bit slower than the 1st iteration, but it had no significant influence on the overall runtime performance.

We worried more about being able to have typos in text id’s, as those are strings and located in game code now, but this was never really a problem I think.

We added text id’s to the Excel file first, then always copy/paste text id’s from Excel to game code. However, we can’t be 100% certain that all possible missing texts were hunted down by the QA team. We still needed a verification system that does this automatically and 100% reliable, but more on this later.

Due to the additional checksum lookup table and the string literal id’s in the code itself, the game also requires more memory.

Most development systems feature some kind of debug memory (eg no$gba has an option to emulate 8MB rather than 4MB main memory), this is at least during development not a problem. More on this later, again.

language.h looked like this:

#pragma once
 
enum Languages
{
  LANG_EN,
  LANG_DE,
};
 
const char* GetText(const char* textId);

We changed from #defines to enum’s too, because those are much more debug friendly.

language.cpp looked like this:

#include "language.h"
 
// initial language is english
Languages CurrentLanguage=LANG_EN;
 
// crc32 checksums / hashes of text id's
// in sorted ascending order
const unsigned int TEXT_IDs[]=
{
  12345, // text id: helloworld
  23456, // text id: goodday
};
 
// german texts
const char* const TEXT_DE[]=
{
  "HALLO WELT",
  "GUTEN TAG"
};
 
// english texts
const char* const TEXT_EN[]=
{
  "HELLO WORLD",
  "GOOD DAY"
};
 
// performs a binary search on the the TEXT_IDs array
// and returns the index where the hash is located, or
// -1 when it could not be found.
int FindTextIndex(unsigned int hash)
{
  int left  = 0;
  int right = (sizeof(TEXT_IDs) / sizeof(TEXT_IDs[0])) - 1;
 
  while (left <= right)
  {
    int index = (left + right) / 2;
    if (hash == TEXT_IDd[index])
      return index; // hash found, leave!
 
    if (hash > TEXT_IDd[index])
      left = index + 1;
    else
      right = index - 1;
  }
 
  return -1; // hash not found
}
 
// get text of the specified text identifer,
// returns the textId if text could not be found
const char* GetText(const char* textId)
{
  // generate checksum / hash of incoming text id
  unsigned int hash = CalcCRC32(textId);
 
  // search for the checksum / hash in our TEXT_IDs array
  int index= FindTextIndex(hash);
  if(index == -1)
  {
    // text not found, return the id instead!
    return textId;
  }
 
  switch(CurrentLanguage)
  {
    case LANG_DE:
      return TEXT_DE[index];
 
    default:
      break;
  }
 
  return TEXT_EN[index];
}

Localization, 3rd iteration

The 2nd iteration is not bad, but as the project came along, new requirements did pop up.

We not only needed to display texts in different languages, different title logos should be displayed. We just hacked to load different resources, depending on the language setting, in game code.

But we should have known before, this ain’t fulfils the artists vision. So before we clump the whole game code, we decided it should be handled automatically without any action from a programmer and this was easier than we thought.

We already used some sort of file archive, where all game content is stored, to load files from. Think of it as a zip archive. In order to load language dependend resources, all we had to do is to support more than one file archive and priorize it. When the game requests a file, file archives are searched by priority.

We added an additional file archive with german content and high-priorized it. When the game requests “title.bmp”, the german archive was searched first. If the resource could not be found, the next archive was consulted. This allowed to add language dependend resources without any programmer work!

Localization, 4th iteration

Having all languages in main memory is quite a waste, at least on systems that don’t feature hundrets of mega-bytes. In my experience, non-text-heavy games contain about 700 texts, where edutainment games can contain thousands.

If every text would be 50 chars and 700 texts are available, it’s 50*700 = 35000 chars, which in ASCII is about 34kb for one language! 5 languages sum up to 34kb*5 = 170kb.

This is more than 4% of the Nintendo DS main memory, only for text! Not really an option to spend that much precious memory for text if you could use those wasted 135kb and spend them on a larger level, more sounds or more textures instead.

On memory limited systems it makes sense to have one language in memory only, namely the current language. However, this comes with several things to consider:

• Memory consumption is different for different languages.
• GetText() returns different pointers for different languages.

Different memory consumption is a huge problem. It’s irresponsible if some levels don’t load when german language is activated, because german texts consume 2kb more memory than the english ones. It makes it also impossible to replace entire language files on-the-fly, change language setting in options for example, without any delete/new mechanism involved.

Furthermore it’s also quirky when GetText() returns a different pointer for every language, because GUI widgets can no longer store pointers to texts, because they would point to whatever memory if the language setting changes.

The secret is always try to keep memory consumption as static as possible! Our custom text tool compared texts of every language and padded shorter texts of different languages with zero-bytes to consume as much space as the longest text, for example:

You won.000000000
Du hast gewonnen.
 
Hello World
Hallo Welt0

Where “0″ represents the padding 0×00 byte.

This makes sure that:

• every language file has the exact same size.
• offsets to texts inside the language file are always the same.

This approach makes it possible to allocate memory once for the language file and then being able to work with that buffer, because the size never changes for different language files of the same category. You can load other language files to this buffer and the text system still works.

When text is located in a language file, we also no longer have the const char* overhead from our text arrays, just make sure to null-terminate every text!

Localization, 5th iteration

The additional memory footprint introduced with the 2nd iteration bothered us and we wanted it better spending on textures than text and this is very simple again.

We supported a hybrid system of the 1st and 2nd iteration. The 2nd iteration was perfect for development purposes, as it does not require much recompile, but comes with higher memory footprint.

The 1st iteration on the other hand is horrible during development, because of the recompile times, but does not require any additional data (crc table) and is lighting-fast.

Instead of using string literals for the text identifier directly, we wrapped them in a TXT macro. The debug build stringified the incoming parameter, where as the release build concatenated it to create an identifier:

#if _DEBUG
  #define TXT(id) #id
#else
  #define TXT(id) TID_##id
#endif

It was used like this:

const char* text = GetText(TXT(helloworld));

The debug build replaced it with:

const char* text = GetText("helloworld");

and the release build with:

const char* text = GetText(TID_helloworld);

where TID_helloworld is the generated #define identifier of our custom text tool, as shown in 1st iteration.

We used the 2nd approach for debug builds and the 1st approach for release builds. When you switch between debug and release builds, you need to do a recompile anyway, so using the 1st does not hurt.

And at this point I can also resolve the “more on it later..” note from the 2nd iteration paragraph.

Because we used the 1st approach in release builds, we could catch all invalid text identifiers at compile time and had no memory overhead anymore, yay! Supporting both systems is also not really problematic in my opinion, since they’re pretty similar and not complicated anyway.

Conclusion

Creating a multi-language game comes with a couple of new tasks, don’t underestimate it!

In the palib homebrew community (Nintendo DS homebrew development) is #include “data.c” a daily occurence. Several people point out not to follow this approach, because it’s “bad practice”.

In this article, I’m not so much trying to convince you about the evilness of including data, but I’m telling you when this approach is appropriate, when it isn’t and why. The article assumes you use devkitARM to build your .nds files.

History

I guess the palib author adapted this approach from the early HAM days. HAM was around 2001-2005 a very popular free software development solution for the Nintendo Game Boy Advance, including almost everything you would need to create games for this lovely device.

The example projects in HAM used to integrate resources (graphics, sounds, etc…) by converting resources to C source code arrays. The generated files were #included in the example source file (main.c).

The intention behind this approach makes sense for the Game Boy Advance. GBA games are shipped on Game Pak’s that consist of Read-Only-Memory (ROM) containing the program code (.text) and data (.rodata). When you declare const data on the heap, the memory is allocated in ROM. In order to load graphics to video ram, you could tranfer image data directly from ROM to Vram, without the need to have it in main memory.

What’s the deal with that?

The Nintendo DS on the other hand does not feature this ROM section in the traditional way anymore, everything (code, data) is located in the 4MB main memory region. When you #include data in your program code, memory for it is always allocated and present in main memory, even when you don’t need it!

Let’s take a simple example to clearify: Imagine you create a platform game with several levels. Each level features a different set of graphics (sunny, snowy, rainy). If you would #include graphics in program code, memory for every single graphic would be allocated in main memory for every level in each level. So you would have graphics of level 1 (sunny) in main memory, even when you play level 2 (snowy). It might work for the first 1 1/2 graphic-sets, but then exceeds the main memory capacity of 4MB eventually.

As you already know, program code is located in main memory too, so you can’t even calculate with 4MB for resources. As far as my experience goes, real-world-projects use about 400-600 KB for program code, leaving you with only about 3.5MB for internal game management and resources.

#include or not #include, that’s the question

When you aim for a game with a decent quality, you can’t #include data as this would blow up memory, you have to load resources from the file system. To be more precise, you load only resources to memory, that you actually need in the particular level (file i/o handling in nds homebrew).

However, sometimes it’s necessary to have access to resources all the time, for example to display on-screen debug text. For such “low-level-system”, you want to have the resource in memory, because what happens when loading of the debug-font resource would fail?! You couldn’t even display an error at this time, because the font is missing! For this special case, I suggest you either convert the resource in question to an object file (.o) and link this one, or convert to assembler code and assemble as well as link, but don’t #include.

If you generate C code, rather than assembler, the C compiler must parse the code, then generate the corresponding assembler code, that is being transformed to object code afterwards. When you don’t like waiting for your compiler to complete, choose .o or .s as target!

Why did I just say “don’t #include”?

When you create a seperate source file that contains data and store it in your “source” directory, it’s being compiled only when the target object file (.o) is either missing or the source file is newer. As for the debug font resource, this would be once or at least very rarely. After the source file has been compiled and transformed to an object file, it only needs to get passed to the linker when you build the project. It removes the need to compile the file every time again.

When you #include data in a file where you currently work at, the compiler must parse the #included data every time you change something on the code and recompile. Performance-wise isn’t it really much of a problem anymore, since todays computers have several GB memory, multi-core processors and relatively fast hard-drives, but it makes the compile process slower anyway, which sums up when your project grows.

I think the error that occurs most often when dealing with #include data is multiple definitions of the same symbol. This occurs when you include a file in more than one source file, there are dozens of these posts on the palib forum!

How to embed data

You want to embed “debugfont.bmp” in your game, so it’s always in memory, how you doing that? It’s simple! Use your favorite graphics converter to ouput a “binary” version of the converted data, lets call it “output.bin”. In order to embed output.bin, all you have to do is to move it to the “build” directory of your project. The example makefiles of libnds/devkitARM include a target that takes every .bin file in the “build” directory and convert, compile and link it.

If you want to do it by hand, use devkitARM/bin/bin2s.exe to convert the .bin file to assembler code and move the output file to your “source” directory. Next time you build the project, the file gets compiled and linked.

Conclusion

• You can’t complete a game project by #include’ing data, which contains more than about 3.5MB resources.
• Sometimes you want to embed data, eg graphic for debug text, compile and link the file.
• #include data is error-prone, multiple definition of…
• #include data slows down compile process which can become significant as the project grows.

iDeaS is a Nintendo DS software emulator available for Linux and Microsoft Windows. Since version 1.0.2.8 (21 Dec 2008) iDeaS features program breakpoints and user messages that can be sent to the debug console:

# Added program breakpoint (SWI #0xFDFDFD).
# Added output on console for user’s messages (SWI #0xFCFCFC).

My alarm bells started to bang in the moment I saw the changelog. Both features have been implemented using software interrupts, that do not exist on the target hardware.

What does it mean? It means, those software interrupt ID’s make your NDS application incompatible with the NDS hardware. Whenever you accidentally print an iDeaS debug message, the application will crash on actual hardware.

no$gba implemented the breakpoint and debug message facility quite excellent. In no$gba a breakpoint is just a mov r11, r11 instruction (a nop), which can be executed by the hardware by all means. The no$gba debug message system is a bit more complicated. It uses a combination of harmless instructions to detect you want to print text. However, all instructions used for that feature do work on hardware as well, this is what we want, this is how it should be implemented.

The current scenario

Shortly after the release I contacted the iDeaS author and told him the current approach isn’t top notch and should be changed that it won’t break program execution on actual hardware. Unfortunalety he didn’t see the advantage, since you can:

• Remove all prints before you test on hardware

Obviously, this cannot be the way to handle it. Manually removing all prints would be enormously time consuming and error-prone.

• Use some sort of #ifdef blocks to automatically remove all prints

This might sound like a solid idea at first glance, but thinking about it a further minute, proves it isn’t. The problem is simple, it’s too time comsuning. When you use the pre-processor to remove all print calls, you have to rebuild the entire project every time you change the corresponding pre-processor switch. Depending on how many files the project has, can it take a significant amount of time to rebuild, which is no option for many people.

• Wrap print calls with some enabled/disabled mechanism

A function that wraps print calls with a surrounding if block, removes the need to rebuild the project. Just disable debug output at program initialization and there you go. However, I tend to forget things that are of no importance for me and this is something I would forget many times!

How to solve the puzzle

We developers want a debug text system that will not break program execution, no matter if the application runs in an emulator or on actual hardware. All those previous points do make some sense, but don’t remove the problem in a whole. What we have to create is a system that:

• Is able to print text to the debug console
• Can be switched on/off at runtime
• Automatically detects when it runs on hardware and discards all print calls in this case
• Remove all print calls when building a “release version” (pre-processor)

Everthing on the list should be quite clear, except for the hardware detection. When I had the requirements-list done and was wondering how to implement that, I remembered this post at gbadev.org. They used a hardware feature that no emulator seems to emulate correctly, instruction fetching:

1
2
3
4
5

detectGBA: ;returns 0 if emulated
mov r0,#0
str r0,_0
_0: mov r0,#1
mov pc,lr

The str r0, _0 instruction overwrites the following instruction, where r0 is set to 1. However, on real hardware the instruction would be fetched already and the change has no effect. Basically the memory at this address is being overwritten, but the instruction pipeline fetched the instruction before, so the original instruction is used. When you call the function more than once, it won’t work correctly anymore.

So I slightly changed the code to restore the original instruction to be able to call the function more than once. I also made it arm and thumb compatible. Here is the entire source code to print text to iDeaS debug console, that detects real hardware and discards all prints at runtime. You can also use a pre-processor #define to remove all print calls when you build a “release version”.

ideas.h

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28

#ifndef __ideas_h__
#define __ideas_h__
 
#if __cplusplus
  extern "C" {
#endif
 
#if !_RELEASE
 
int IdeasEnableDebugOutput(int enable);
int IdeasOutputDebugString(const char* format, ...);
 
#else
 
// in release mode use empty functions, so the compiler
// optimizes any call to them away.
inline int IdeasEnableDebugOutput(int /*enable*/) { return 0; }
inline int IdeasOutputDebugString(const char* /*format*/, ...) { return 0; }
 
#endif // _RELEASE
 
int IdeasIsEmulator();
 
#if __cplusplus
  } // extern "C"
#endif
 
#endif // __ideas_h__

ideas.c

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98

#include "ideas.h"
#include <stdio.h>
#include <stdarg.h>
 
#if !_RELEASE
 
static int IdeasDebugOutputEnabled=1;
 
// Gets if the program runs in an emulator.
__attribute__ ((noinline))
int IdeasIsEmulator()
{
// The idea behind the code is to overwrite
// the "mov r0, #0" instruction with "mov r0, r0" (NOP).
// On real hardware, the instruction would have been fetched,
// so the overwrite has no effect for the first time executed.
// In order to be able to call the function more than once,
// the original instruction is being restored.
// see http://forum.gbadev.org/viewtopic.php?t=910
#ifdef __thumb__
  int mov_r0_r0 = 0x1c00;   // mov r0, r0
  int mov_r0_0  = 0x2000;   // mov r0, #0
 
  asm volatile (
    "mov  r0, %1     \n\t"  // r0 = mov_r0_r0
    "mov  r2, %2     \n\t"  // r2 = mov_r0_0
    "mov  r1, pc     \n\t"  // r1 = program counter
    "strh r0, [r1]   \n\t"  // Overwrites following instruction with mov_r0_r0
    "mov  r0, #0     \n\t"  // r0 = 0
    "strh r2, [r1]   \n\t"  // Restore previous instruction
    : "=r"(mov_r0_r0)       // output registers
    : "r"(mov_r0_r0), "r"(mov_r0_0)    // input registers
    : "%r1","%r2"           // clobbered registers
    );
 
#else
  int mov_r0_r0 = 0xe1a00000; // mov r0, r0
  int mov_r0_0  = 0xe3a00000; // mov r0, #0
 
  asm volatile (
    "mov  r0, %1     \n\t"  // r0 = mov_r0_r0
    "mov  r2, %2     \n\t"  // r2 = mov_r0_0
    "mov  r1, pc     \n\t"  // r1 = program counter
    "str  r0, [r1]   \n\t"  // Overwrites following instruction with mov_r0_r0
    "mov  r0, #0     \n\t"  // r0 = 0
    "str  r2, [r1]   \n\t"  // Restore previous instruction
    : "=r"(mov_r0_r0)       // output registers
    : "r"(mov_r0_r0), "r"(mov_r0_0)    // input registers
    : "%r1","%r2"           // clobbered registers
    );
#endif
 
  return mov_r0_r0 != 0;
}
 
// This function must be noinline, because
// iDeaS expects the text to output in register r0.
// If this code is inlined somewhere, it's not guaranteed
// that text is located in r0 anymore, thus will not work.
static __attribute__ ((noinline))
void IdeasOutputDebugStringInternal(const char* text)
{
#ifdef __thumb__
  asm volatile ("swi #0xfc");
#else
  asm volatile ("swi #0xfc000");
#endif
}
 
 
// Prints formatted output to the iDeaS debug console
// Returns false when text has not been printed, true otherwise.
int IdeasOutputDebugString(const char* format, ...)
{
  va_list args;
  char    buffer[128]; // increase to support more characters
 
  if(!IdeasDebugOutputEnabled || !IdeasIsEmulator())
    return 0;
 
  va_start(args,0);
  vsnprintf(buffer, sizeof(buffer), format, args);
  va_end(args);
 
  IdeasOutputDebugStringInternal(buffer);
  return 1;
}
 
// Enables or disables debug output.
// Returns the previous enabled state.
int IdeasEnableDebugOutput(int enable)
{
  int old = IdeasDebugOutputEnabled;
  IdeasDebugOutputEnabled = enable;
  return old;
}
 
#endif // !_RELEASE

Download the files here.

A commercial game that is being shipped on its own cartrige can rely on the fact all game assets are available. There is no way to delete files from it. In this case, you don’t need to keep attention if fopen or fread succeded, because they always do.

With nds homebrew it’s a whole different. The user of your game has to copy related game assets and the .nds executable itself to his/her flashcard. Everything can go wrong during this process. Beside missing files, it could be also a flashcard compatibility problem, such as an invalid DLDI driver. In these cases, many games just die silently, rather that displaying an error-message to let the user know what happened.

You often find file loading code such as (don’t copy!):

1
2
3
4
5
6
7
8
9
10
11

uint8_t* LoadFile(const char* filename)
{
  FILE* file = fopen(filename, "rb");
  fseek(file, 0, SEEK_END);
  size_t size = ftell(file);
  rewind(file);
  uint8_t* buffer = (uint8_t*)malloc(size);
  fread(buffer, size, sizeof(uint8_t), file);
  fclose(file);
  return buffer;
}

All those functions have return values, which must not be ignored. If any function call fails here, it’s very unlikely the program will continue to operate correctly.

We do have to find a better way to ensure loading either works “in all cases” or does not. The very first thing that comes to mind is to use some sort of file archive. If you bundle all assets in one archive, there is a good chance everything is available and there is no way to delete files from it. At application startup, test if the archive is available, open it and read a test file to confirm the system works. If it does not, display a human readable error message, numerical error codes don’t count.

Another critical part is deployment. If there is more than one file to deploy, there is also more than one chance to fail. So, it would be quite good if the user needs to deploy only one file for the entire game, rather than the .nds file and the archive seperately. This can be done by appending the archive to the .nds file!

I’ve created my own tool set for all this, which I won’t share,  but there is a free public solution for it called Embedded File System Library as well. I’ve never used EFS, but it should do exactly what we need if we trust its documentation. If we use this approach, an archive appended to the .ds file, we can be pretty sure assets are available as long as the appended archive is available. Only two problems left

• Incompatible DLDI driver.

To detect an incompatible DLDI driver, check the return value of fatInitDefault. This function is part of libfat. If it returns false, display an appropriate message and make the user aware it could be due to an incompatible DLDI driver. This gives him/her a chance to solve the problem rather than just being frustrated, deleting your game and sending you hate-mails.

• User aborted the copy process before the whole file was written to the flashcard.

To check if the whole file was written to the flashcard could be done by appending a magic value at the end of the archive. At application startup seek to this position, read the value and compare it with what it should be. If it’s different, display a warning that the attached data seems to be corrupted. You could let the user continue with the game, he/she will know what could have caused the problem when the game crashes eventually.

What I do to ensure file i/o works

Create an archive of all assets, append it to the .nds file as well as a magic value.

The magic value can be a string like “I FEEL GOOD” at the end of the file. At application startup, initialize libfat and repsond to its return value. Display an error message when fatInitDefault failed.

Open the application file where the archive is appened. Check if this operation succeded and display an error message if anything went wrong.

Seek to and read the “I FEEL GOOD” magic value. If it’s different, display a “data is corrupted” warning. Open and read a file from the archive to verify the system works, display an error if it fails.

From now on assume the file i/o system will work and continue with further initialization.

If all checks last longer than a few milliseconds, display a “Please wait, initializing file system…” message before.

Source code snippet to verify that at the end of the program file is “I FEEL GOOD” located:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50

// opens the file specified by filename and checks if at the
// end of the file is "I FEED GOOD" located.
// returns true on success, false otherwise. If return value
// is false, further error information is stored at errorString.
bool VerifyProgramFile(const char* filename, char* errorString)
{
  const char* const MAGICVALUE="I FEEL GOOD";
 
  // open file
  FILE *file = fopen(filename, "rb");
  if(file == NULL)
  {
    sprintf(errorString, "Cannot open file '%s'.", filename);
    return false;
  }
 
  // seek to magic value offset
  fseek(file, 0, SEEK_END);
  size_t magicOffset = ftell(file) - strlen(MAGICVALUE);
  if(fseek(file, magicOffset, SEEK_SET) != 0)
  {
    // could not seek to magic
    sprintf(errorString, "Data in file '%s' seems to be corrupted.", filename);
    fclose(file);
    return false;
  }
 
  // read magic value
  char magicValue[64];
  if(fread(magicValue, sizeof(char), strlen(MAGICVALUE), file) != strlen(MAGICVALUE))
  {
    // could not read magic
    sprintf(errorString, "Data in file '%s' seems to be corrupted.", filename);
    fclose(file);
    return false;
  }
 
  // compare magic value
  if(memcmp(magicValue, MAGICVALUE, strlen(MAGICVALUE)) != 0)
  {
    // magic is different
    sprintf(errorString, "Data in file '%s' seems to be corrupted.", filename);
    fclose(file);
    return false;
  }
 
  // all tests successfully passed
  fclose(file);
  return true;
}

Source code snippet of application startup code:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21

int main()
{
  if(!fatInitDefault())
  {
    DisplayMessage("libfat file system initialization failed. Did you apply the correct DLDI driver?");
    Halt();
  }
 
  char errorString[1024];
  if(!VerifyProgramFile(pathToFileArchive, errorString))
  {
    DisplayMessage(errorString);
    WaitForUserConfirmation();
  }
 
  // open archive and test if a file
  // can be read from the archive. if all tests
  // succeed, continue with application initialization...
 
	return 0;
}

However, even compiling a .c file containing only the application entry point, without any external libraries, already creates a .nds file that is 54848 bytes:

int main(void)
{
  return 0;
}

After fiddling around for “some” time, I had a .nds file that not only shows a simple quad, I had one that shows hundreds of lit textured cubes that create a tunnel where the camera flies through with a fullscreen distortion effect in less than 4096 bytes.

Once we’ve released it at pouet.net, I figured from the comments I could had shrink the filesize further 480 bytes by just compressing it – I feel so dumb!

Obviously it can not compete with the breakpoint 4k entries, but I felt good that I was able to do it anyway, especially because it seems to be the worlds first 4k intro for the Nintendo DS! Unfortunalety, it features no music.

You can download the binary at: http://pouet.net/prod.php?which=53081