New tonc-code.zip

The basic premise behind SpaceChem is chemical engineering: in each assignment, you get some input molecules and you have to turn them into the requested output molecules via bonding and unbonding their atoms, and the occasional nuclear fusion. A simple (and decidedly sub-optimal) example:

The chemistry part is just a front, though. What the game really is, is a programming simulator. You have a tiny board (10×8) board with input and output zones and two ‘waldos’ that trace a path on which you place and execute instructions: basically a tiny dual-threaded CPU with a select instruction set. I know it sounds simple, and that’s exactly right: it sounds simple. However, working out a workable design hard, VERY hard. Especially when you try to optimize a design for least time or components.

There are in-game leaderboards, showing you how much a design that you have toiled away at for hours actually sucks sweaty donkey balls, and a more detailed scoreboard at spacechem.net. You can also upload your solutions to youtube. Here’s one of mine for “Gas Works Park”, where you turn some random carbon strings (C₃ / C₄) and water (H₂O) and turn them into methane (CH₄) and carbondioxide (CO₂). Solution details can be found in the video description.

• Publisher site (with other engineering games): Zachatronics Industries
• Game site: http://www.spacechemthegame.com/
• Review: http://www.rockpapershotgun.com/2011/01/10/wot-i-think-spacechem

If you like programming and games (which probably describes readers here pretty well), you need to give this a try.

Normally, I don’t really do this kind of stuff, but …

Holy crap, that was a tight game. First a beautiful goal by van Bronckhorst. Then the equalizer by Forlán just before the break. Then a hideous goal by not-Sneijder, but he gets the point anyway, quickly followed by Robben’s header. And then in overtime they still almost manage to blow it when Pereira makes it 2–3 and then drama in the rest of the overtime! OMFGWTFBBQ!!111!!ELEVENTYONE!!!

Netherlands – Germany finals incoming!

Linky:

http://store.steampowered.com/freeportal/

• 1 Introduction
• 2 Probability and its rules
• 3 Combinatorics : counting evolved
• 4 Examples
• 5 Summary

1 Introduction

When it comes to math, one of the things that people are phenomenally
bad at – aside from the obvious everything – is
probabilities. Specifically, calculating probabilities.
People are bad enough at
estimating chances, but properly calculating then has its own
hurdles.

In this particular case, I’d like to take a look at the probability of
a Simple random sample: getting a particular distribution
of results from a number of samples, like getting 3 sixes in a
certain number of rolls. Specifically, I’ll look at the case
with replacement. In this case, the sample pool remains the same
after each individual sample. The case without replacement
(like picking colored marbles from a vase without putting them back)
is a bit trickier so I won’t cover that now.

The binomial distribution

One of the equations that is relevant here is the
binomial distribution
(see Eq 1), the scourge of math students
everywhere. What I’d like to do is explain why it looks the way it does,
what the terms mean and how you can generalize it to cover more than
two outcomes (that’s what the “bi-” stands for: two).
Basically, I’d like to show you how to “read” the equation
and to interpret the terms in it, which will make it easier to deal
with the subject.

(1)	title="P(W=k) = {n\choose k} p^k (1-p)^{n-k}" alt="P(W=k) = {n\choose k} p^k (1-p)^{n-k}" />

First, a few words about the Eq 1 itself. Say you have
a random event that can have two outcomes: coin-flip, true/false question,
but also a die-roll if you just look at an X vs non-X result. Or, in
the parlance of our times, whether you have a win (W) or a
fail (F). What Eq 1 gives you is the probability
of getting exactly k wins in n trials.

That’s all well and good, of course, but unless you actually understand
how the equation is built up and what all the terms in it mean, it’ll
just be yet another magic formula that someone dreamed up to torture you
with. In practical terms, it also means that it’s harder to apply it
correctly. So let’s look at what it means, starting with what each
individual symbol means.

Sigh. Gaddammit >_<

This is pretty much where I wanted to go with this game, so it’s now version 1.0. I may add some other things later, but for now I guess it’s done.

Changes

• new: FAT saving enabled :D. It’s intended for
  argv-capable
  cards, though. You can still save without one of those, but you’ll have to
  put in a little work.
• new: Reset/Back buttons for high-score screen.
• fix: Bug in shuffle code. Fixes the occasional hang at start of a new game.
• tweak: I’ve altered the decay constant slightly; it should be closer to
  the original tatset now. The upshot is that scores will be a little higher.
• tweak: Slightly bigger buttons.

Links

• setds project page

A while ago, I wrote
an article about NDS caching
, how it can interfere with DMA transfers and what you can do about them.
A little later I got a
pingback from ant512,
who had tried the “safe” DMA routines I made and found they
weren’t nearly as safe as I’d hoped. I’m still not sure what the actual
problem was, but this incident did make me think about one possible
reason, namely the one that will be discussed in this post: problematic
cache invalidation.

1 Test base

But first things first. Let’s start with some simple test code, see below.
We have a simple struct definition, two arrays using this struct, and
some default data for both arrays that we’ll use later.

And now we’re going to do some simple things with these arrays that
we always do: some reads, some writes, and a struct copy. And for the
copying, I’m going to use DMA, because DMA-transfers are fast,
amirite(1)? The specific actions I will do are the following:

Initialization

• Zero out g_src and g_dst.
• Initialize the arrays with some data, in this case from
  c_fooIn.
• Cache-Flush both arrays to ensure they’re uncached.

Testing

• Modify element in g_src, namely g_src[0].
• Modify an element in g_dst, namely g_dst[3].
• DMA-copy g_src[0] to g_dst[3].

In other words, this:

Note that there is nothing spectacularly interesting going on here.
There’s just your average struct definition, run of the mill array
definitions, and boring old accesses without even any pointer magic
that might hint at something tricky going on. Yes, alignment is forced,
but that just makes the test more reliable. Also, the fact that I’m
incrementing Foo.id rather than just reading from it is
only because ARM9 cache is read-allocate, and I need to have these
things end up in cache. The main point is that the actions in
test_dmaCopy() are very ordinary.

2 Results

It should be obvious what the outcome of the test should be. However,
when you run the test (on hardware! not emulator), the result seems to
be something different.

Notice that the changed values of g_src[0] never
end up in g_dst[0]. Not only that, not even the
original values g_src[0] have been copied.
It’s as if the transfer never happened at all.

The reason for this was covered in detail in the original article.
Basically, it’s because cache is invisible to DMA. Once a part of
memory is cached, the CPU only looks to the contents of the cache and
not the actual addresses, meaning that DMA not only reads out-of-date
(stale) source data, but also puts it where the CPU won’t look.
Two actions allow you to remedy this. The first is the
cache flush, which write the cache-lines back to RAM and
frees the cache-line. Then there’s cache invalidate, which
just frees the cache-line. Note that in both cases, the cache is
dissociated from memory.

With this information, it should be obvious what to do. When DMA-ing
from RAM, you need to flush the cache before the transfer to update
the source’s memory. When DMA-ing to RAM, you need to invalidate
after the transfer because now it’s actually the cache’s data that’s
stale.
When you think about it a little this makes perfect sense, and it’s easy
enough to implement:

Unfortunately, this doesn’t work right either. And if you think about
it a lot instead of merely a little, you’ll see why. Maybe showing the
results will make you see what I mean. The transfer seems to work now,
but the earlier changes to g_dst[3] have been erased. How
come?

The problem is that a cache-invalidate invalidates entire
cache-lines, not just the range you supply. If the start or
end of the data you want invalidate does not align to a cache-line,
the adjacent data contained in that line is also thrown away. I hope
you can see that this is bad.

This is exactly what’s happening here. The ARM9′s cache-lines are 32
bytes in size. Because of the alignment I gave the arrays, elements
0 through 3 lie on the same cache-line. The changes to
g_dst[3] occur in cache (but only because I read from it
through +=). The invalidate of g_dst[0]
also invalidates g_dst[3], which throws out the
perfectly legit data and you’re left in a bummed state. And again,
I’ve done nothing spectacularly interesting here; all I did was
modify something and then invalidated data that just happened to be
adjacent to it. But that was enough. Very, very bad.

Just to be sure, this is not due to a bad implementation of
DC_InvalidateRange(). The function does exactly what it’s
supposed to do. The problem is inherent in the hardware. If your
data does not align correctly to cache-lines, an invalidate will apply
to the adjacent data as well. If you do not want that to happen, do
not invalidate.

3 Solutions

So what to do? Well, there is one thing, but I’m not sure how foolproof
this is, but instead of invalidating the destination afterwards, you
can also flush it before the transfer. This frees up the cache-lines
without loss of data, and then it should be safe to DMA-copy to it.

Alternatively, you can also disable the underlying reason behind the
problem with invalidation: the write-buffer. The ARM9 cache allows
two modes for writing: write-through, which also updates
the memory related to the cache-line; and write-back, which
doesn’t. Obviously, the write-back is faster, so that’s how libnds
sets things up. I know that putting the cache in write-through mode
fixes this problem, because in libnds 1.4.0 the write-buffer had been
accidentally disabled and my test cases didn’t fail. This is probably
not the route you want to take, though.

4 Conclusions

So what have we learned?

• Cache – DMA interactions suck and can cause really subtle bugs.
  Ones that will only show up on hardware too.
• Cache-flushes and invalidates cover the cache-lines of the requested
  ranges, which exceed the range you actually wanted.
• To safely DMA from cachable memory, flush the source range first.
• Contrary to what I wrote earlier, to DMA to cachable memory,
  do not cache-invalidate – at least not when
  the range is not properly aligned to cache-lines. Instead, flush
  the destination range before the transfer (at which time
  invalidation should be unnecessary). That said, invalidate should
  still be safe if the write-buffer is disabled.

Link to test code.

In any case, the two versions should be identical again (with one small exception, namely that my version emits a .size directive for assembly, but that’s a minor something that should not affect anyone.)

First of all, the vector::insert should finally be fixed. And there was much rejoicing. I’ve also added an option for forcing the map palette-index (-mp <num>, which should help with NDS backgrounds that use ext-palettes.

Also – and this one is pretty big – I’ve completely replaced the tile-mapping routines for something more general. The new method should be able to handle variable-sized tiles (-tw <n> and -th <n>) and is mostly independent of bitdepth. Specifically, bitdepths over 8 bpp can be handled as well, at least in principle. It also means that the external tileset can be a metatile-set as well now, which is good if you’re using metatiles.

With this new method also comes a way to create a custom bitformat for maps (-mB flag). I’m not entirely sure how this can be used yet, but using more than 10 bits for the tile-index, or a 1bpp collision map should be possible now.

Since this is a fairly major change, I kinda expect there’s still some bugs in the system. I have tested it for a number of options, but you know how it is with multi-platform stuff. In particular, if any of you big-endian-system users have trouble now, this will probably be the cause.

And now I will leave you with a …

• link to grit