TCP UDP Socket – How to Reserve Ports for Applications

I believe that the idea of the socket being unavailable to a program is to allow any TCP data segments still in transit to arrive, and get discarded by the kernel. That is, it's possible for an application to call close(2) on a socket, but routing delays or mishaps to control packets or what have you can allow the other side of a TCP connection to send data for a while. The application has indicated it no longer wants to deal with TCP data segments, so the kernel should just discard them as they come in.

I hacked out a little program in C that you can compile and use to see how long the timeout is:

#include <stdio.h>        /* fprintf() */
#include <string.h>       /* strerror() */
#include <errno.h>        /* errno */
#include <stdlib.h>       /* strtol() */
#include <signal.h>       /* signal() */
#include <sys/time.h>     /* struct timeval */
#include <unistd.h>       /* read(), write(), close(), gettimeofday() */
#include <sys/types.h>    /* socket() */
#include <sys/socket.h>   /* socket-related stuff */
#include <netinet/in.h>
#include <arpa/inet.h>    /* inet_ntoa() */
float elapsed_time(struct timeval before, struct timeval after);
int
main(int ac, char **av)
{
        int opt;
        int listen_fd = -1;
        unsigned short port = 0;
        struct sockaddr_in  serv_addr;
        struct timeval before_bind;
        struct timeval after_bind;

        while (-1 != (opt = getopt(ac, av, "p:"))) {
                switch (opt) {
                case 'p':
                        port = (unsigned short)atoi(optarg);
                        break;
                }
        }

        if (0 == port) {
                fprintf(stderr, "Need a port to listen on\n");
                return 2;
        }

        if (0 > (listen_fd = socket(AF_INET, SOCK_STREAM, 0))) {
                fprintf(stderr, "Opening socket: %s\n", strerror(errno));
                return 1;
        }

        memset(&serv_addr, '\0', sizeof(serv_addr));
        serv_addr.sin_family      = AF_INET;
        serv_addr.sin_addr.s_addr = htonl(INADDR_ANY);
        serv_addr.sin_port        = htons(port);

        gettimeofday(&before_bind, NULL);
        while (0 > bind(listen_fd, (struct sockaddr *)&serv_addr, sizeof(serv_addr))) {
                fprintf(stderr, "binding socket to port %d: %s\n",
                        ntohs(serv_addr.sin_port),
                        strerror(errno));

                sleep(1);
        }
        gettimeofday(&after_bind, NULL);
        printf("bind took %.5f seconds\n", elapsed_time(before_bind, after_bind));

        printf("# Listening on port %d\n", ntohs(serv_addr.sin_port));
        if (0 > listen(listen_fd, 100)) {
                fprintf(stderr, "listen() on fd %d: %s\n",
                        listen_fd,
                        strerror(errno));
                return 1;
        }

        {
                struct sockaddr_in  cli_addr;
                struct timeval before;
                int newfd;
                socklen_t clilen;

                clilen = sizeof(cli_addr);

                if (0 > (newfd = accept(listen_fd, (struct sockaddr *)&cli_addr, &clilen))) {
                        fprintf(stderr, "accept() on fd %d: %s\n", listen_fd, strerror(errno));
                        exit(2);
                }
                gettimeofday(&before, NULL);
                printf("At %ld.%06ld\tconnected to: %s\n",
                        before.tv_sec, before.tv_usec,
                        inet_ntoa(cli_addr.sin_addr)
                );
                fflush(stdout);

                while (close(newfd) == EINTR) ;
        }

        if (0 > close(listen_fd))
                fprintf(stderr, "Closing socket: %s\n", strerror(errno));

        return 0;
}
float
elapsed_time(struct timeval before, struct timeval after)
{
        float r = 0.0;

        if (before.tv_usec > after.tv_usec) {
                after.tv_usec += 1000000;
                --after.tv_sec;
        }

        r = (float)(after.tv_sec - before.tv_sec)
                + (1.0E-6)*(float)(after.tv_usec - before.tv_usec);

        return r;
}

I tried this program on 3 different machines, and I get a variable time, between 55 and 59 seconds, when the kernel refuses to allow a non-root user to reopen a socket. I compiled the above code to an executable named "opener", and ran it like this:

./opener -p 7896; ./opener -p 7896

I opened another window and did this:

telnet otherhost 7896

That causes the first instance of "opener" to accept a connection, then close it. The second instance of "opener" tries to bind(2) to the TCP port 7896 every second. "opener" reports 55 to 59 seconds of delay.

Googling around, I find that people recommend doing this:

echo 30 > /proc/sys/net/ipv4/tcp_fin_timeout

to reduce that interval. It didn't work for me. Of the 4 linux machines I had access to, two had 30 and two had 60. I also set that value as low as 10. No difference to the "opener" program.

Doing this:

echo 1 > /proc/sys/net/ipv4/tcp_tw_recycle

did change things. The second "opener" only took about 3 seconds to get its new socket.

Local processes connecting to such port will create a unix pipe instead of an IP socket.

Whether a process uses a pipe or a socket does not depend on the interface per se. It depends on which system services it calls.

To create a named pipe, the program calls mkfifo(2). To get a descriptor on that file, it calls open(2). ls -l shows a "p" in the status bits of a fifo. Anonymous pipes, created with pipe(2), have no name and are invisible to processes without a common ancestor.

To create a TCP socket, it calls socket(2) and then bind(2) to assign it a name for other processes to connect to. If the argument to bind is an external address, it will be visible to the network, else not.

You may be thinking of the loopback address 127.0.0.1. If the program binds to that, it won't be visible or accessible from the network. Only processes running on the host machine can bind or connect to the loopback address.

It is also possible to create a TCP connection over something that looks a bit like a file, known as a UNIX-domain socket. In that case the argument to bind (and connect) has a filename. Because the name appears in the filesystem, it looks a little like a pipe, but ls -l shows an "s" among the status bits.

If you're interested in how all these strange distinctions came to be, NetBSD distributes two papers that date from their invention and are still relevant today. Look for An Advanced 4.4BSD Interprocess Communication Tutorial and An Introductory 4.4BSD Interprocess Communication Tutorial on the web.