MacBook – Connecting multiple screens to the 2016 MacBook Pro with Touch Bar

There are USB 3.x male to male cables that should work for you. Here's one example. -> https://www.startech.com/Networking-IO/USB-PS2/usb-file-transfer-cable-windows-mac~USB3LINK

I have not used such a cable myself but there are several such cables on the market and it appears that they all use the same chip inside. It sounds like they can be put in a "serial mode" or an "Ethernet mode". Each mode is much like it sounds, a serial mode will look like having two USB to serial adapters back to back, and Ethernet mode will look like two Ethernet adapters back to back. Cables like this are generally preferable to pairing up serial or Ethernet adapters because this is faster, 5 gigabits instead of 100 megabits per second, and is cheaper, because if someone were to track down a couple gigabit Ethernet adapters the cost would be far higher.

Take great care in selecting cables because there are still many USB 2.0 devices like this which will only get you 480 Mbps. Even worse are a number of cables that do not comply with the USB standard and will SHORT OUT THE POWER PINS!

There are passive, and USB 3.x compliant, crossover cables (compliant because they do not connect the power pins) that could work if someone could find drivers. Cables like this -> https://www.datapro.net/products/usb-3-0-super-speed-a-a-debugging-cable.html

Both laptops do not have to be connected by the same means. A Thunderbolt cable between the Mac Mini and one laptop will create a very fast network, up to 20 Gbps with Thunderbolt 2. Cables like this -> https://eshop.macsales.com/shop/Thunderbolt/OWC/Thunderbolt-Cables

Connecting everything by Ethernet is a "safe" choice as setting up a network this way is quite simple, low cost, well documented, etc. All you need is a couple Thunderbolt to Ethernet adapters and a hub, or three Ethernet adapters (one for each computer, plus using the one Ethernet port already in the Mac Mini) and connect the two laptops to the Mini by Ethernet. Thunderbolt to Ethernet adapters like these should do -> https://eshop.macsales.com/item/Apple/MD463LLAU/

Here's what I'd do in this case, Thunderbolt to Thunderbolt from Mini to one laptop and the USB 3.x transfer cable to the other. The Thunderbolt connection should practically set itself up, it's things like this that the port was made for. The USB transfer cable is not something I've tried before but the people that make them say they work. If not then return it and get a gigabit Ethernet adapter instead.

My guess is that it's in the DisplayPort to VGA not supporting a high enough resolution–although 800x600 is rather low, there's not a hard rule for VGA. Have you tried switching which monitor uses VGA and seen if the resolution tracks?