As I mentioned in my final edit, the reason that I am not able to get higher bandwidth using round-robin bonding when the switch has Link Aggregation Groups set is that switch Link Aggregation Groups do not do round-robin striping of packets on a single TCP connection, whereas the linux bonding does. This is mentioned in the kernel.org docs:
12.1.1 MT Bonding Mode Selection for Single Switch Topology
This configuration is the easiest to set up and to understand, although you will have to decide which bonding mode best suits your
needs. The trade offs for each mode are detailed below:
balance-rr: This mode is the only mode that will permit a single TCP/IP connection to stripe traffic across multiple interfaces. It is
therefore the only mode that will allow a single TCP/IP stream to
utilize more than one interface's worth of throughput. This comes at a
cost, however: the striping generally results in peer systems
receiving packets out of order, causing TCP/IP's congestion control
system to kick in, often by retransmitting segments.
It is possible to adjust TCP/IP's congestion limits by altering the net.ipv4.tcp_reordering sysctl parameter. The usual default value
is 3. But keep in mind TCP stack is able to automatically increase
this when it detects reorders.
Note that the fraction of packets that will be delivered out of order is highly variable, and is unlikely to be zero. The level of
reordering depends upon a variety of factors, including the networking
interfaces, the switch, and the topology of the configuration.
Speaking in general terms, higher speed network cards produce more
reordering (due to factors such as packet coalescing), and a "many to
many" topology will reorder at a higher rate than a "many slow to one
fast" configuration.
Many switches do not support any modes that stripe traffic (instead choosing a port based upon IP or MAC level addresses); for
those devices, traffic for a particular connection flowing through the
switch to a balance-rr bond will not utilize greater than one
interface's worth of bandwidth.
If you are utilizing protocols other than TCP/IP, UDP for example, and your application can tolerate out of order delivery, then this
mode can allow for single stream datagram performance that scales near
linearly as interfaces are added to the bond.
This mode requires the switch to have the appropriate ports configured for "etherchannel" or "trunking."
The last note about having ports configured for "trunking" is odd, since when I make the ports in a LAG, all outgoing Tx from the switch go down a single port. Removing the LAG makes it send and receive half and half on each port, but results in many resends, I assume due to out-of-order packets. However, I still get an increase in bandwidth.
Best Answer
I would assume most distros accept individual private donations (they may also accept free hosting). However, that is probably not the bulk of their financing in most cases.
Note that some of the major distros may have some paid staff, and possibly also office space, the cost of which likely exceeds that of hosting the repos1. This should not be taken to mean that they are not primarily volunteer based (except for the commercial variants, they are), just that they do have operating budgets.
Fedora is owned by Redhat, and the latter is a publicly traded, billion dollar annual business. I would presume they do quite a bit to help support the former.
According to wikipedia, CentOS is now also owned by Redhat and earlier this year Redhat announced their ongoing sponsorship of CentOS development.
Ubuntu is owned by Canonical, which I do not think is on a par with Redhat, but they probably still have revenues into the tens of millions USD per year. Last time I downloaded an image, Ubuntu was pretty aggressive about encouraging you to make a small donation at the same time. $5 a year would I think cover the costs of repo hosting associated with the average installation.
The Debian project has been around for nearly 20 years and surely has a substantial core of users willing to help support it. They also have a list of "partners" here which provide them with resources. I would think Canonical helps out significantly, since Ubuntu is reliant upon Debian, but judging from this link provided in Kiwi's answer, they are still having to beg publicly for $250K to cover meeting costs, which is pretty disappointing.
Arch is likely much poorer than the other distros mentioned here, but they may still collect enough money from various sources to support some development staff and hosting. They do not appear to obviously solicit on their site, so I would guess this funding comes mostly from industry (and possibly, government) grants.
1. To get some idea of how much this hosting would actually cost, consider that GNU/Linux systems probably account for 1-2% of desktop systems worldwide and at least 40% of web servers. If we then assume this might amount to ~25 million systems, if a large (theoretical) distro accounted for 10% of those and each user accounted for 4 MB a day averaged out over time, this would amount to 10 TB/day. I would think if you know the right people, you could perhaps get 3000 TB/month for <$5000 US.