Does a mini-ITX motherboard exist that supports more than 8 GB of memory

Simple answer: It either will only see the max supported memory or it will not work at all.

My gut feeling says two things:

If it does work, you just will not see the extra memory, only the max the motherboard/chipset supports. If it doesn't work, it usually will just beep at you with a memory error and you will get no video responce. All depends on the motherboard depending on how it handles memory errors.

Someone on Yahoo answers says an interesting bit about getting a blue screen due to this:

In my experience, putting in more than the max amount of RAM as specified by the manufacturer will cause the computer to not boot up.

others claim:

Some won't POST, some will and simply BSOD (kernel panic, etc) with PFN_LIST_CORRUPT.

That said, the "max" memory isn't always the actual max. Case in point, Intel states the GL960 chipset (such as in my laptop) supports a max of 2GB of memory. 4GB is a no-go, but 3GB works.

If your board does not already have a power converter from nominal 12v, you can use what's known as a DC-DC power supply. Some are available here:

http://www.mini-box.com/DC-DC

They are meant to convert generic nominal 12v power to consistent 12v, 5v, and 3.3v power for computers. Common applications are car-computers, where source voltage can be anywhere from 10-15VDC and be variable and somewhat noisy in normal conditions.

As you mentioned, some mini ITX boards already have this circuitry on the board itself and accept a single 12v input. If you're using one of those and concerned about the stability or input voltage being too high, you can use a simple 12v voltage regulator.