First of all, the vector::insert should finally be fixed. And there was much rejoicing. I’ve also added an option for forcing the map palette-index (-mp <num>, which should help with NDS backgrounds that use ext-palettes.

Also – and this one is pretty big – I’ve completely replaced the tile-mapping routines for something more general. The new method should be able to handle variable-sized tiles (-tw <n> and -th <n>) and is mostly independent of bitdepth. Specifically, bitdepths over 8 bpp can be handled as well, at least in principle. It also means that the external tileset can be a metatile-set as well now, which is good if you’re using metatiles.

With this new method also comes a way to create a custom bitformat for maps (-mB flag). I’m not entirely sure how this can be used yet, but using more than 10 bits for the tile-index, or a 1bpp collision map should be possible now.

Since this is a fairly major change, I kinda expect there’s still some bugs in the system. I have tested it for a number of options, but you know how it is with multi-platform stuff. In particular, if any of you big-endian-system users have trouble now, this will probably be the cause.

And now I will leave you with a …

• link to grit

From xkcd (obviously)

T_T

Okay, so it’s only a card game; but a game nonetheless.

The game in question is an NDS implementation of
SET. Set is a card-matching game
with 81 cards (see below). The figures on the cards have four
properties and 3 possibilities for each property. The key is to find
three cards for which the values of each property are either all
equal or all different. Looking at the color property,
for example, a “Red Red Red” combination could (yes “could”; there
are still three other properties to consider) form a set.
“Red Green Blue” would also work, but “Red Green Green” would not.

Further details can be found in the readme and the game itself.

The game is mostly finished. There may be some tweaking to do
here and there, but right now I don’t want to get bogged down in a
massive fine-tuning-fest – especially since I’m not sure what
parts need fine-tuning … and because there’s
other stuff I really should get
back to.

That said, all important aspects work … with one
exception: hiscore saving. Yes, that. I’ve seen the
multitude of threads on the subject but sofar I’m unsure of what would
work on both hardware and emulator, so I’m leaving it as is for now.
If anyone has a tidy hw+emu solution, please do tell.

Links

• binary: setds-bin.zip (146k)
• source: setds-src.zip (354k)
• readme: setds-readme.txt

Oh, and merry Christmas everybody.

The short story is that officially new should throw an
exception when it can’t allocate new memory. Exceptions come with about
60 kb worth of baggage. Yes, this is more or less the same stuff that
goes into vector and string.

The long story, including a detailed look at a minimal binary,
a binary that uses new and a solution to the exception overhead (in this particular case anyway) can be read below the fold.

• 1 Minimal project
• 2 Standard C++ new/delete
• 3 Custom new/delete
• 4 Other considerations and conclusions.

1 Minimal project

The following is essentially an empty project. It should represent
the smallest binary you can get with the current DKA (r26) and
libnds (1.3.7). This is the primary reference case.

This actually already leads to a binary of 53.5 kb. To analyze what
goes on in there, we can look at the map file. Not the mapfile
generated by the linker, mind you, but by the arm-eabi-nm tool,
whose generated files are considerably easier to read. To use this tool,
add the following line to $(BUILD) rule in the makefile,
so that it looks like below. If you want to know what the flags mean,
please RTFM.

And this is the resulting mapfile, in full.

Now, I expect you can’t really tell much from this, so here’s a summary.

The 0100:xxxx and 0B00:xxxx ranges belong to
ITCM and DTCM, so those are irrelevant when looking at main RAM size.
The libc, impure and misc bookkeeping sections are stuff related to the
C library and C overhead, accounting for about 1.5 kb. The boot code,
crt0.S also covers close to 1.0 kb. As expected, the code for
main.c –the actual project– is more or less
nothing.

The rest, about 7 kb, is libnds. Now, you may say that this is quite a bit
of overhead, but it really isn’t. Pretty much all of it relates to
interrupts and the fifo system, which takes care of ARM7-ARM9
communication. You need to have these parts. Okay, you could try
to roll your own to shrink this down to the bare essentials, but in all
likelihood that’s more trouble than it’s worth.

The observant of you should have noticed something: we’re only at 9.5 kb,
but the file size is 53.5 kb. So what the hell happened to the other 44 kb?
Well, I don’t know, to be honest. It doesn’t appear in MWRAM to be sure.
It’s probably the stuff ndstool adds. My guess it that that’s
where the ARM7 binary goes, along with the icon, titles and possibly
DLDI interfaces, but I really can’t say right now.

2 Standard C++ new/delete

And now, let’s look at what happens when you invoke new.

Just this small thing increases the file size to 117 kb! And remember,
that’s not merely a doubling of the size, as 44 kb of the binary is not
put in memory. The memory load has gone from about 10 kb to over 70 kb.
What causes this increase? Well, let’s see:

Well, I did say this was going to be the long story, didn’t I?
Everything that was in the base project is in here as well. The
additional parts can summarized as follows.

There are three main areas to discern:

• d_*() routines, presumably for debug printing.
  (size: 12k)
• Stdio formatting and related. This includes file handling, device
  handling and many forms of printf, which brings a
  whole lot of bagage (some allocation, format parsing and
  math/floating point routines). There’s also some abort and
  signalling routines. (size: 28k).
• Exception handling. Not just routines for handling them, but also
  the typeinfo stuff required, the output strings and the output
  string pointers. (size: 18k)

These roughly 60k of stuff is the overhead of exceptions –
any potential exception. In this case, it’s because
new requires a bad_alloc exception when it’s
unable to allocate more.

The problem is that exceptions have
many dependencies: to do exception handling, you keep track and unwind
the stack. You also need to be able to tell the type of exception
thrown, which requires RTTI. And then you say which exception was
thrown, so you need error messages, and a list of pointers to
those messages, and a way to format and write those messages,
hence the d_*() routines and all the stdio stuff.

3 Custom new/delete

There is a way around this, though: redefine new and
related functions. Technically speaking, this is a bad idea
if you don’t know what you’re doing, but it can be done. Note that
you would need overload four operators: new,
delete and their array counterparts.

This way, you just incur the cost of malloc() and
free(), which are only about 3k. But again, this is going
against the standard and you’ll really have to ask yourself if the
(at best) 2% of main RAM you save with this is really worth it.

More on this can be read at

http://brewforums.qualcomm.com/showthread.php?t=2033.

4 Other considerations and conclusions.

The binary size is not the same as the main RAM footprint.
About 44 kb other stuff.

The overhead of the standard new is 60 kb, which is all
due to exceptions. You cannot remove it by
using the compiler options -fno-exceptions and
-fno-rtti, because that only affects your own code, not the
standard libraries. You can remove this overhead by using overloading
new and related functions, but you have to be really
careful with this.

I’ve also done a little bit of testing with vector, and
it seems that vector’s overhead also comes from
new and can be removed the same way. However, other parts
of vector (and STL) may use other exceptions, so it’s
quite possible it won’t work in all cases.

Note that roughly 28 kb of the exception overhead is actually
stdio related – specifically formatted printing:
*printf. If you’re using printf anyway, the
effective overhead of exceptions is reduced considerably.

Finally, remember that the exception overhead amounts to roughly 2% of
main RAM at most. In most homebrew cases it won’t matter that much.
When it does start to affect your app, you will likely have other parts
that are easier and safer to optimize out.

Test project + notes.

Of course, for those that aren’t normal and that would like to write
in raw HTML, these things are somewhat annoying. Fortunately, though,
WordPress allows you to disable these kinds of filters. The catch is
that you need to find out which filters to disable, namely,
wptexturize (which converts HTML entities) and
wpautop (which does newline control). WordPress also makes
it easy add additional filters, like the
rel="pingback">CodeSnippet plugin that I use for code highlighting.

However, with the amount of filters available, sometimes things will
clash. A good example of this is comments that have source code
in them. Part of what CodeSnippet does is convert certain characters
(specifically: ‘<’, ‘>’,
‘&’) to printable characters
(&lt;, &gt;, &amp;) and aren’t considered special HTML
characters anymore. However, there are several other filters that
have a similar task, so that when you write this:

Oh hai! This is a useful bitfield function.

 

[code lang="cpp"]

template<class T>

inline void bfInsert(T &y, u32 x, int start, int len)

{

   u32 mask= ((1<<len)-1) << start;

   y &= ~mask;

   y |= (x<<start) & mask;

}

[/code]

what it becomes is:

Oh hai! This is a useful bit function.

template
inline void bfInsert(T &amp;y, u32 x, int start, int len)
{
    u32 mask= (1&lt;&lt;len) &lt;&lt; start;
    y &amp;= ~mask;
    y |= (x&lt;&lt;start) &amp; mask;
}

Not exactly pretty. Note that the template class is simply removed
because it's seen as an illicit HTML tag, and all the special
characters are doubly converted. This is still a mild example; I think
if you place the angle brackets wrong, whole swaths of code can
simply be eaten by the sanitizer.

Unfortunately, finding out where the problem lies is tricky. Not
only are there dozens of potential functions doing the conversion,
they can be called from anywhere and PHP isn't exactly rich in the
debugger department. You also have no idea where to start, because
the filters can be called from everywhere. Worse still, in this
particular case the place where the bad happens is actually before
the comment is even saved to the database (but only for unregistered
people; for me the code comments would work fine), and because comments
are handled on a page that you don't actually ever see, random
echo/print statements are useless as well.

But I think I finally got it: it was
wp_kses() using (in a roundabout way)
wp_specialchars() in the wp-includes/kses.php
roomfile. The contractor is actually
wp_filter_comment() from wp-includes/comment.php,
using the pre_comment_content filter as a middleman.

The trick now is to keep it from happening. What I've done is define
not one but two pre_comment_content filters: one that
pre-mangles the brackets and ampersand before wp_kses,
and one that de-mangles them afterwards. Of course, this will only
be of importance between [code] tags. Exactly how to do this will
depend on the plugin you're using, but in the case of
CodeSnippet it goes like this:

Notice that both methods are under the same filter group. The trick
is that they have different priorities, which makes one act before
wp_kses(), and one after. Also note how the regexps work
in the replacement part of preg_replace(). This particular
feature of preg_replace() allows for shorter code, but is
very fragile; it may be better to use
preg_replace_callback() instead. In any case, written like
this it seems to work:

Oh hai! This is a useful bit function.

template<class T>
inline void bfInsert(T &y, u32 x, int start, int len)
{
   u32 mask= ((1<<len)-1)<<start;
   y &= ~mask;
   y |= (x<<start) & mask;
}

Comment preview

The code-comment mangling is just part of the issues one can
encounter in blog comments. It's usually impossible to see beforehand
what will be accepted and what not. Is HTML allowed? Are all tags
allows, or just some or none at all? What about whitespace? Or
BB-like tags? Basically, you'll never know what a comment will look
like until you submitted it, and by then it's too late to change it.

You know what'd be really helpful? A comment preview!

You'd think this'd be a fairly obvious feature for a blogging
system to have, but apparently not.
I was thinking of making by own preview functionality, but when
attempting to do so several items within WP thwarted my efforts.
Fortunately, it seems plugins of this sort exist already. The plugin
I'm now using is rel="pingback">ajax-comment-preview, which works pretty darn well.

So anyway, comments should be able to handle code properly now and
there's a comment-preview to show you what the comment will look
like in the end. And there was much rejoicing.

The integer datatypes in C can be either signed or unsigned. Sometimes,
it’s obvious which should be used; for negative values you clearly
should use signed types, for example. In many cases there is no obvious
choice – in that case it usually doesn’t matter which you use.
Usually, but not always. Sometimes, picking the wrong kind
can introduce subtle bugs in your program that, unless you know what
to look out for, can catch you off-guard and have you searching for
the problem for hours.

I’ve mentioned a few of these occasions in Tonc
here and
there, but I think
it’s worth going over them again in a little more detail. First, I’ll
explain how signed integers work and what the difference between signed
and unsigned and where potential problems can come from. Then I’ll
discuss some common pitfalls so you know what to expect.

• 1 Basics
• 2 Potential problems
• 3 Summary

1 Basics

The signedness of a variable refers to whether it can be
used to represent negative values or not. Unsigned variables can only
have positive values; signed values can be both positive or negative.

In the computer world, signedness is mostly a matter of interpretation.
Say you have a variable that is N bits long. This is enough
room for 2^N distinct numbers, but it says nothing about
which range of numbers you should be using them for. Interpreted as
unsigned integers, its range would be [0, 2^N−1].
Under a signed interpretation, you’d use some bit-patterns for negative
numbers. There are actually several ways of doing this, but the most
commonly used is known as two’s complement which leads to
a [−2^N−1, 2^N−1−1] range:
half positive and half negative.

1.1 Two’s complement theory

Two’s complement is sometimes seen as an awkward system, but it
actually follows quite naturally when you only have a fixed number
of digits to write down numbers with.
Consider the whole line of positive and negative integers. As you
move away from zero, the numbers will grow larger and larger.
Now suppose you have an
counting device
composed of a limited number of digits, each of which can only display
numbers 0 through 10−1. With N digits, you only have
room for 10^N different numbers, and once those are used up
(at 10^N−1), the counter returns to 0 and counting
effectively resets. In essence, the number on the counter works in
modulo 10^N.

The key is that this works in both positive and negative directions.
As far as the counter is concerned, 0 and 10^N are the
same thing. This being the case, you can argue that −1
(that is, the number before zero) is equivalent to 10^N−1;
and −2 ≡ 10^N−2, and so on.
Note that this works regardless of what 10 actually is; it can be
ten (decimal), two (binary) or sixteen (hexadecimal).

The 10^N possible numbers form a window over the number line,
but where the window starts is up to the user. For signed numbers,
you can move the window so that the upper half of the 10^N
range is interpreted as negative numbers.

Fig 1 shows how this works for 8-bit numbers
(written in hex for convenience). The black numbers represent the
entire number line, where numbers can have as many digits as you
need. With only two nybbles, the counter repeats every
100h = 256 values. FFh, 1FFh, but also −1 all reduce to the same
symbol, namely FFh. In Fig 2 you can see
how the available symbols are mapped to either signed or unsigned
values. In the unsigned case, numbers simply count from 0 to FFh;
for signed, the top half of the symbol range is put on the left side
of zero and are used for negative numbers.

Fig 1.

Fig 2.

The mathematical reason behind all this like this. Assume for
convenience that N = 1, so that 0 is equivalent to 10 and
in fact every multiple of 10. By definition, subtracting a value
from itself gives 0. Because subtraction is merely addition by its
negative value, you get the following:

(1)	title="\begin{eqnarray} x &-& x &=& 0& \\ x &+& (-x) &=& 0 & \\ x &+& (-x) &\equiv& 10 & \\ & & (-x) &=& 10 &- x \end{eqnarray}" alt="\begin{eqnarray} x &-& x &=& 0& \\ x &+& (-x) &=& 0 & \\ x &+& (-x) &\equiv& 10 & \\ & & (-x) &=& 10 &- x \end{eqnarray}" />

The term −x in the last step should be seen as a unit,
call it C. Numerically, C is the number that, when added
to x, gives 10. In decimal, if x = 1, then C = 9.
C is called the 10’s complement of x, because
it’s what’s needed to complete the 10. It’s called the two’s
complement in binary, because then 10 equals two.

In binary, there is an alternative to calculate the twos complement
of a number. Subtracting a number from 2^N is equivalent to
inverting all its bits, so you get:

(2)	title="\begin{eqnarray} (-x) &=& 2^N - x \\ &=& 2^N -1 - x + 1 \\ (-x) &=& \sim x + 1 \\ \end{eqnarray}" alt="\begin{eqnarray} (-x) &=& 2^N - x \\ &=& 2^N -1 - x + 1 \\ (-x) &=& \sim x + 1 \\ \end{eqnarray}" />

Using two’s complement(1)
for negative numbers has some interesting
properties. First, subtraction and addition are basically the same
thing. This is nice for arithmetic implementers for two reasons:
the same hardware can be used for both operations, and it can be used
for both positive and negative numbers.

Second, because the top half
is now used for negative numbers, the most significant bit can be
seen as a sign bit. Note: a sign bit, not the sign bit.
There is a subtle linguistic difference here. When talking about
the
sign bit, one may thing of it as a single bit that indicates the sign.
For example, 8-bit +1 and −1 could be `0000 0001‘
and `1000 0001‘, respectively. In two’s complement,
however, +1 and −1 are actually `0000 0001‘
and `11111111‘ (the sum of which is
`1,00000000‘ ≡ 0, as
it should be).

1.2 Declaring signed or unsigned

In the end, whether a particular group of bits is signed or unsigned is
a matter of interpretation. For example, the 8-bit group
`1111 1111‘ can be either 255 or −1, depending on
how you want to look at it. You can’t determine the signedness
from just the bits themselves.

Also, when you’ve decided you’re going to use a signed interpretation,
whether the group forms negative number or not depends on the size of
the group. for example, consider the two bytes `01 FF‘.
As separate bytes, these would form +1 and −1, respectively.
However, if you view them as a single 16-bit integer
(‘short”), it forms 0×01FF, which is a positive number.

In C, you specify signedness when you declare a variable. The general
rule is that an integer is signed unless the keyword
`unsigned‘ is used. The exception to the rule is
`char‘, whose default signedness is platform and
compiler-dependent! Be careful with this particular datatype.

Because they’re shorter and more descriptive, the following typedefs
are often used for variable declarations. Basically, it’s
‘s’ or ‘u’ for signed
or unsigned, respectively, followed by the size of the type in bits.
Unsigned variants are also sometimes indicated by
‘u”+typename.

In assembly, you can’t declare the signedness of variables, because
there’s no such thing as variables. There’s only labels and how you
use those labels determines what the related data are. Technically,
there is only one datatype: the 32-bit word, corresponding to C’s
int or long. The other datatypes are essentially emulated, or
defined by how which memory instructions you use:
LDRB/LDRSB/STRB for bytes and
LDRH/LDRSH/STRH for halfwords. For most data
operations, signedness is irrelevant and as such mostly ignored.
Only in a few cases does the sign actually matter and as these are
essentially the topic of the rest of the article, we’ll get
to those eventually.

2 Potential problems

The following sections are cases where signedness may become
problematic. I say “may”, because often it just works
out. But that’s just the thing: it can work most of the time and then
things can go horribly wrong all of a sudden. The root of the problem
comes down to one thing: negative numbers; usually, negative numbers
becoming large positive numbers when interpreted as unsigned values.

For example, 32-bit signed −1 = 0xFFFFFFFF = unsigned 4294967295
(= 2³²−1). If nothing else, remember that part.

2.1 Sign extension, casting and shifting

When you go from a small datatype to a larger one, you’re essentially
adding a new set of bits at the top, and these bits have to be
initialized in a meaningful way. The addition of these bits should have
no effect on the value itself. For example, +1 should remain +1 and
−1 should remain −1. What this boils down to for two’s
complement is that the new bits need to be filled with the sign-bit
of the old value. This is called sign extension, because
the top-bit (the sign-bit) is extended into all the higher bits. There
is also zero-extension, which is when the higher bits are
zeroed out. These two forms effectively correspond to signed
and unsigned casting. (2).

Fig 3.

Conversions of this kind actually happen all the time,
without any kind of direct intervention from the programmer. Data
operations are always done in CPU words and any time you use a smaller
datatype, there is the need to sign- or zero-extend.
This also brings forth the question of which type of extension will be
used: sign- or zero-extension. As the following bit of code shows, it
depends on the signedness of the variable you’re converting from.
8-bit variables sc and uc are both initialized
by 0xFF, which is either −1 or 255 (you can use either of those too,
by the way). After that, these are used to initialize signed or unsigned
words.

As you can see from the output, the value in the words correspond
to the signedness of the bytes, not the words. Also note that printing
sc (the signed byte) gives 0xFFFFFFFF and not the 0xFF you
initialized it with, and which are in fact its actual contents since
0xFFFFFFFF is too large to fit into a byte. However, when using it with
anything, it’s automatically extended to word-size. This becomes great
fun when you later compare it to 0xFF again.

Sign- and zero-extension also play a role in right-shifts. When using
shifts for arithmetic (shift-right is short-hand for a division by
power of two), you want the sign preserved. For example, when dividing
−16 = 0xFFFF:FFF0 by 16 (shift-right by 4), you want the result
to be −1 (=0xFFFF:FFFF), and not 268435455 (=0×0FFF:FFFF).
The right-shift that preserves the sign is the
arithmetic right-shift, and is used for signed numbers.
For unsigned numbers, or if the variable is considered a set of bits
instead of a single number, a logical right-shift is
appropriate, since that uses zero-extension.

In assembly, arithmetic and logical right-shift are called
ASR and LSR, respectively. In Java and other
languages where the keyword unsigned does not exist
the difference is indicated by >> (sign-extend)
and >>> (zero-extend). In C, however,
both types use the same symbol: >>. As such,
you cannot tell which type of extension is used from just the
expression; you’d have to look at the signedness of the operands
(including temporaries) to see if it’s a logical or arithmetic
right-shift.

Table 2: Right-shifts for different languages. Language Signed Unsigned ARM asm asr lsr C >> >> Java(script) >> >>>

Fig 4.

This ambivalence of shift symbols in C can be a major source of pain in
fixed-point calculations. Since unsigned has precedence over signed, if
you have an unsigned variable at any point of the calculation,
all subsequent calculations are unsigned too and you can kiss negative
numbers goodbye. If everything starts going wrong as soon as you move in
another direction or if rotations aren’t calculated properly, this will
be the cause.

The code below illustrates the problem in a very common situation. You
have a position p, and a directional vector for movement,
u. Since you want sub-pixel control of these, you use
fixed-point notation for both (I’m assuming non-FPU system
here). The u vector is a unit vector
(say, cos(α), sin(α)); to get to the full velocity vector,
we have to multiply u by some speed. The procedure comes down to
something like this:

pnew = pold + speed·u

In the example, I’m only considering the x-component for
convenience. Now, because position and direction can have negative
components, those would be signed. The speed, however, is a length
and therefore always positive, so it makes sense to make it unsigned,
right? Well, yes and no. As you can see from the result, mostly no.

With speed = +1 and u_x = −1, the
end result should be +1*−1 = −1, which would be 0xFFFFFF00
in Q8 fixed-point notation. However, it isn’t, thanks to the
unsignedness of speed, which makes subsequent arithmetic
unsigned so the right-shift does not sign-extend. So instead of the
small step you intended, you get a giant leap into no man’s land.

This mistake is depressingly easy to make, even for those who generally
think about which datatype to use. Especially those people, as
they’re prone to optimize prematurely and automatically pick unsigned
for a variable that will never be negative. The danger is that
unsigned arithmetic has precedence, which can screw up at later
right-shifts.

Bottom line: variables used in fixed-point calculations should be
signed. Always.

2.2 Division

This isn’t really a signed-vs-unsigned item per se, but integer
division behaves in a peculiar way for negative numbers. It becomes
one, however when you throw right-shift in the fray, which doesn’t
quite work like a division equivalent anymore for negative numbers.
To discriminate between integer and normal division, I will use
‘\ ’ for integer division in this section. Note also the
modulo operation is intimately tied to division, so this section applies
to that as well.

What integer division comes down to is taking a normal division and throwing
away the remaining fraction. For example, 7 / 4 = 1¾. The
integer division is just 1. This is also true for negative numbers:
−7 / 4 = −1¾, so 7 \ 4 = −1. In short,
integer division rounds towards zero. With bit-shifting, however, you get
something slightly different. Theoretically, x>>n is
equivalent to x \ 2ⁿ. For positive numbers, this is
true: 7>>2 in binary is
00000111>>2 = 00000001.
But with −7>>2 you get
11111001>>2 = 11111110 = −2.
Division-by-right-shift always rounds to negative infinity.

The upshot of this difference is that for negative numbers, the results
of x \ 2ⁿ and x>>n will be out
of sync, as Table 3 illustrates. They still give
identical results for positive numbers though.

There are some other consequences besides the obvious difference in
results. First, there’s how compilers deal with it. Compilers are very
well aware that a bit-shift is faster than division and one of the
optimizations they perform is replacing divisions by shifts
where appropriate(3). For
unsigned numerals the division will be replaced by a single shift.
However, for signed variables some extra instructions have to added to
correct the difference in rounding.

Second, note that the standard integer division does not give an equal
distribution of results: there are more results in the zero-bin.
Shift-division spreads the results around evenly. In some cases, you
will want to use the shift version for that reason. One clear example
of this would be tiling: using the ‘proper’ integer
division would give you odd-looking results.

2.3 Comparisons

The last area where signedness can be a factor is comparisons. The
next bit of code is from my implementation of a filled circle renderer
with boundary clipping. The circle is centered on
(x₀, y₀). Variables
x and y are local variables that
keep track of where we are on the circle, because these
can be negative, they must be signed. Variables dstW
and dstH are the destination image’s width and
height. Since width and height are unsigned by definition,
it’d make sense to make these unsigned, right? Right?

Well, apparently not. When I tested this, right and bottom edge
clipping went fine, but when the circle went over the top or
left edge, it disappeared completely.

The problem lies with the line in bold, which does the trivial rejection
test. Variables left and right are the left and
right-most edges of the scanline of the circle. If this is completely
to the left of the screen (right < 0) or to
the right of the screen (left ≥ dstW)
then there’s nothing to do.

Technically, the tests on that line are correct, so the code
should work.
The reason it doesn’t actually occurs a few lines earlier: the
definition of dstW as an unsigned variable. Because of
this, the second condition is an unsigned comparison. Now think of
what happens when left moves over the left of the
screen. left becomes becomes a (small) negative number,
which is converted to postive number for the comparison.
A large positive number for that matter – one that’s
quite a bit larger than the width of the image and as a result
the routine thinks the circle is out of bounds.

So again, a routine went all wonky because I assumed that, since
a width is always positive, using an unsigned variable would be
a good idea.

The worst part of this particular bit, however, is that I should have
known this. The compiler actually issues a warning for this type of
thing:

warning: comparison between signed and unsigned integer expressions

Or at least it would have if I hadn’t disabled the warning
because the message was cropping up everywhere in my normal and sign-safe
for-loops. Let this be a lesson: disable warnings at your own risk
and for Offler’s sake do not ignore them.

2.4 Well, duh

The problems covered above are the subtle ones, where you have to be
aware of some of the details that go into the C language itself. There
are also a few issues where the programmer really should have known
they were going to be a problem from the start.

The first example is, again, one that can occur when optimizing
prematurely. You may have heard that loops work better when you count
down instead of count up, because in machine code a subtraction is an
automatic comparison to zero. So, a clever programmer may turn this:

into this:

There are two problems with this code. First, the change probably will
not matter with modern compilers because they are aware of the
equivalence and can do this conversion themselves(4), so there’s nothing to gain from this.

The real problem, however, is the terminating condition:
`i>=0‘. Since i is unsigned, it can
never be negative, and therefore the condition is always true.

The second example involves bitfields. As it happens, bitfields can be
signed or unsigned as well. For the most part, handling this is like
handling normal signedness, but there is one situation where you have
to be careful.

In the code above I’ve created a bif-fielded struct with both
signed and unsigned members. There are two 7-bit fields and two
1-bit fields, and these are initialized to −1 and +1,
respectively. The values are then printed.

The 7-bit fields work as you might expect. f.s7 is
−1, as it’s signed, and f.u7 is 127, which is the
7-bit equivalent of −1. The interesting case is for
f.s1. This is initialized to 1, but comes out as
−1, because for a single signed bit the possibilities
are 0 and −1, and not 0 and +1! Without this knowledge,
a later test like `f.s1==1‘ might give unexpected results.

3 Summary

So, summarizing:

• Unsigned variables only represents positive numbers; signed ones
  can have positive or negative values.
  Negative numbers are usually represented via two’s complement,
  which is based on the cyclical nature of counters when you have
  a limited number of digits.
• In C, integers are signed unless specified otherwise, except
  for char, whose signedness is compiler dependent.
• Careless use of signed and unsigned types can result in subtle
  runtime bugs with not-so-subtle results. Usually, what happens
  is that a negative number is reinterpreted as a very large
  positive number and everything goes banana-shaped.
• Unsigned has a higher operator precedence than signed. If
  one of the operands is unsigned, the operation will use unsigned
  arithmetic. This can cause problems for divisions, modulos,
  right-shifts and comparisons.
• For negative numbers, division/modulo by 2ⁿ is not
  quite the same as right-shifts/ANDs. Analyse which is best for
  your situation, then act accordingly.
• Ignore compiler warnings at your own peril.
• The place where a bug manifests is not always the place where it
  originates. The declaration of variables matters! Do not forget
  this when debugging or when asking for assistance.

There isn’t really a hard rule on when to use which signedness, but
here are a few guidelines nonetheless.

• If a variable can, in principle, have negative values, make it
  signed. If it represents a physical quantity (position, velocity,
  mass, etc), make it signed.
• A variable that represents logical values (bools, pixels, colors,
  raw data) should probably be unsigned.
• And now the big one: just because a variable will always be
  positive doesn’t mean it should be unsigned. Yes, you may waste
  half the range, but using signed variables is usually safer. If
  you must have the larger range (for the smaller datatypes, for
  example), consider defining the storage variables unsigned, but
  convert them to local signed ints when you’re really going to
  use them.
• If mathematical symbols were gods, the minus sign would be
  Loki. Be extra careful when you encounter them.
  If there are minus signs anywhere in the algorithm, or even
  the potential for negative numbers, everything should be
  done with signed numbers.

The Geshi that came with it was … 1.0.7.20, I think. In any case, Geshi’s
is now at 1.0.8.3, so I figured it was time for an upgrade. Most notable was
that the way numbers were parsed has been greatly modified, with different
types of representations now being parsed separately – and correctly
to boot. Right now, it’s almost fully correct, as you can see from the list
below:

Only some of the more special integer literals aren’t parsed correctly,
specifically the unsigned (-U) and long long
(-LL) suffixes aren’t accepted. I don’t suppose hex floats will
work either, but that’s a GCC extension anyway.

To fix this, you need to modify geshi a little; specifically the
GESHI_NUMBER_INT_CSTYLE regular expression:

… yeah. I’m not sure why it’s formulated like that either. I’d have thought
‘\b‘ would have worked just as well, but alright. Anyway, notice the single ‘l‘ character in there? That needs to be extended to something
that matches a potential single ‘u‘, possibly followed by one or two
‘l’s. In other words: ‘u?l{0,2}‘.

HTML in code

An astute readed may have noted the bold in the previous snippet. Normally,
you can’t do that in Geshi.. One of the things that Geshi does is translate
HTML entities like ‘<‘ into things like “<”
so that it’ll turn up right on the resulting page. This, of course, is one of the
things Geshi is expected to do. However, in this case it also makes it impossible
to add HTML parts in the code snippet, which at times can be very useful.

So what do we do now? Well, we can use escaped HTML tags. Much like
“\n” doesn’t actually mean backslash + ‘n‘ but a
newline character, “\<” can be used for the actual
‘<‘. And to unescape that, a double backslash can be used,
much like it is in C.

There are several ways to implement this. One would be to modify it in the geshi
code. I haven’t tried that route yet because I expect it could get messy. That’s
arguably how it should be done, but it’s easier to do it after the fact:
when all the conversions have been done. Basically, you need something like this:

There are three sets of items to search & replace here. The first two are
the basic escaped tag delimiters, so that they’ll actually result in HTML tags,
and unescaped delimiters, so that you can print the combination itself. The
third category are for HTML in string literals. Since the backslash has a
specific meaning there as well, Geshi puts some highlighting stuff around it
that would make the standard search fail. So that whole thing would need
to be searched for and destroyedreplaced.

It’s ugly, I know, but it seems to work. It’d be nicer if this could be done
in the parser itself, but I have a feeling that’d take changes in multiple
places. Since I don’t know the code that well yet, I’m not touching that
one with a ten-foot pole.

Lastly, let’s test the ARM asm highlighter:

Still works too. Bitchin’.