The system keychain is stored in /Library/Keychains/System.keychain and the key to unlock it is stored in /var/db/SystemKey (its default file permissions are readable by root only). The location of these files is referenced in the security-checksystem script (from the security_systemkeychain source). It is even possible to test to automatic locking/unlocking of the system keychain by using
systemkeychain -vt
The keychain security framework allows non-privileged programs to make requests for information provided they are in the ACL stored within the keychain entry. Obviously if a user has root they on a system they can directly access both the file storing the system keychain and the key to unlock it, thus they do not have make requests via the security framework and are not beholden to the ACLs stored within the keychain itself.
(I didn't actually answer the original questions so let's give this another go)
How are the keys architected such that any administrative user can unlock the System Keychain?
The libsecurity keychain framework allows regular processes to interact with the system keychain in an authenticated manner using Apple's XPC interprocess communication framework (IPC).
Program A sends a request to access the system keychain information using IPC. A check is made that the requesting user is already in the wheel group and also knows the password of a user in the wheel group. Once authorization is confirmed, the privileged kcproxy daemon can be used to access material in /var/db/SystemKey, unlock the system keychain and return the requested information.
Are there cryptographic restrictions that limit what an administrative user can do with information in the System Keychain in any way?
No - an administrative user is allowed to access/change anything in the system keychain. Even if they couldn't, they could copy the underlying files to another machine on which they have complete control and just unlock/access it there.
Given an unencrypted system backup without /Users, how would you gain access to the keys in the System Keychain?
If the backup contained copies of /Library/Keychains/System.keychain and /var/db/SystemKey then I would copy them to their respective locations on a new OS X system and use systemkeychain to make the later unlock the former and dump the keychain database using security dump-keychain.
The way I achieve this is what you suggested:
Make a secondary Admin account and then take away your admin privileges.
I'm almost certain that it is the only way to achieve this.
Best Answer
It is not currently possible to get the system dialogs for TouchID to also work with a security key, such as for example the popular Yubikey product. Third party applications can however provide support for products such as the Yubikey through similar popups windows.
One method of using security keys in a manner that resembles TouchID is to set your system up to them as a kind of "virtual smart card". This allows you to use for example a Yubikey as a means of logging in to your Mac, and for authenticating with Keychain, etc.
The specific method of achieving that depends on the type of security key you have. In general you need to configure the system and/or install drivers to let the security key function as a "smart card".
For the Yubikey product you can follow this guide from the manufacturer:
https://support.yubico.com/support/solutions/articles/15000006468-using-your-yubikey-as-a-smart-card-in-macos