Postgresql WAL Archiving, How to do it properly

You can't split WAL archiving out so one database goes in one WAL archive stream and other DBs go in another WAL archive stream. A single write-ahead log is shared by all databases in a PostgreSQL server ("cluster" in PostgreSQL parlance). All databases managed by a postmaster have the same WAL. This is a sometimes unfortunate limitation of the current replication/PITR/archiving design and is true for the in-development-at-time-of-writing 9.3 and all and prior releases.

If you want to separate the WAL archiving, you have to separate the databases into different clusters. You can do this by running more than one instance of PostgreSQL on a server; they simply have to have different data directories and different listening ports. Some packagings of PostgreSQL provide built-in tools to do this, like the pg_wrapper tool used on Debian and Ubuntu, which has pg_createcluster. Alternately, you can always just use initdb to create a cluster and pg_ctl to start the cluster after editing the new cluster's postgresql.conf to set an appropriate port.

Tools like pgbouncer may be used, if desired, to multiplex the multiple clusters transparently, so pgbouncer listens on port 5432 (the default Pg port) and decides which server to send the connection to based on which database it asks for. Generally it's easier to just specify a port in the connection setup of the applications that use each database, though.

On PostgreSQL 9.2 and below you must allocate each server an exclusive, non-overlapping shared memory region for its shared_buffers. One server cannot make use of the other server's shared_buffers space when the other server is not busy. That means you are wasting some system resources. PostgreSQL 9.3 improves this by allocating shared_buffers out of anonymous memory, giving the OS more freedom about how it manages it.

If you don't want to run multiple clusters, you can use alternative replication or backup solutions based on out-of-tree tools like Slony-I, Bucardo, etc to manage your data replication and archival.

In order to restore a backup, you need to have the base archive of all the data files, plus a sequence of xlogs. An "incremental backup" can be made, of just some more xlogs in the sequence. Note that if you have any missing xlogs, then recovery will stop early.

So it's not clear here exactly what you've done, unless you changed the level of detail you're mentioning part way through your list. When you make a copy of more segments that have been put into the archive directory after adding more data, you need to ensure that all the data has been archived: using pg_start_backup and pg_stop_backup usually does this for you, but you don't mention it the second time. You need to at least do a pg_switch_xlog to have the current xlog segment immediately archived.

If you think that recovery is not consuming enough xlog segments, look at the recovery log to see if it tried to take them all. And have your recovery command make some sort of mark on which xlog files were taken.