How to fake the source-port of a TCP packet

You can use SO_REUSEADDR

int optval = 1;
/* create socket using socket */
setsockopt(s1, SOL_SOCKET, SO_REUSEADDR, &optval, sizeof optval);
/* bind socket */

You get this error because the TCP protocol forces the server to put the socket that you just closed into the status TIME_WAIT for the time defined in net.ipv4.tcp_fin_timeout. This is to ensure that really every packet which the other peer might have sent after your server closed the socket is still being handled correctly.

Here is a nice description of this problem in the top answer of the thread: What is the meaning of SO_REUSEADDR (setsockopt option) - Linux?.

Just to make sure we're on the same page (your question is ambiguous this way), asking to bind TCP on port 0 indicates a request to dynamically generate an unused port number. In other words, the port number you're actually listening on after that request is not zero. There's a comment about this in [linux kernel source]/net/ipv4/inet_connection_sock.c on inet_csk_get_port():

/* Obtain a reference to a local port for the given sock,
 * if snum is zero it means select any available local port.
 */

Which is a standard unix convention. There could be systems that will actually allow use of port 0, but that would be considered a bad practice. This behaviour is not officially specified by POSIX, IANA, or the TCP protocol, however.¹ You may find this interesting.

That's why you cannot sensibly make a TCP connection to port zero. Presumably nc is aware of this and informs you you're making a non-sensical request. If you try this in native code:

int fd = socket(AF_INET, SOCK_STREAM, 0);
struct sockaddr_in addr;
addr.sin_family = AF_INET;
addr.sin_port = 0;
inet_aton("127.0.0.1", &addr.sin_addr);
if (connect(fd, (const struct sockaddr*)&addr, sizeof(addr)) == -1) {
    fprintf(stderr,"%s", strerror(errno));
}    

You get the same error you would trying to connect to any other unavailable port: ECONNREFUSED, "Connection refused". So in reply to:

Where in the system is this handled? In the TCP stack of the OS kernel?

Probably not; it doesn't require special handling. I.e., if you can find a system that allows binding and listening on port 0, you could presumably connect to it.

^{1. But IANA does refer to it as "Reserved" (see here). Meaning, this port should not be used online. That makes it okay with regard to the dynamic assignment convention (since it won't actually be used). Stipulating that specifically as a purpose would probably be beyond the scope of IANA; in essence operating systems are free to do what they want with it, including nothing.}