linux dynamic-linking – How Do Shared Object (SO) Numbers Work?

Actually, you can install multiple versions of a shared library if it's done properly.

Shared libraries are usually named as follows:

lib<name>.so.<api-version>.<minor>

Next, there are symlinks to the library under the following names:

lib<name>.so
lib<name>.so.<api-version>

When a developer links against the library to produce a binary, it is the filename that ends in .so that the linker finds. There can indeed be only one of those installed at a time for any given <name> but that only means that a developer cannot target multiple different versions of a library at the same time. With package managers, this .so symlink is part of a separate -dev package which only developers need install.

When the linker finds a file with a name ending in .so and uses it, it looks inside that library for a field called soname. The soname advises the linker what filename to embed into the resulting binary and thus what filename will be sought at runtime. The soname is supposed to be set to lib<name>.so.<api-version>.

Therefore, at run time, the dynamic linker will seek lib<name>.so.<api-version> and use that.

The intention is that:

• <minor> upgrades don't change the API of the library and when the <minor> gets bumped to a higher version, it's safe to let all the binaries upgrade to the new version. Since the binaries are all seeking the library under the lib<name>.so.<api-version> name, which is a symlink to the latest installed lib<name>.so.<api-version>.<minor>, they get the upgrade.
• <api-version> upgrades change the API of the library, and it is not safe to let existing binary applications use the new version. In the case that the <api-version> is changed, since those applications are looking for the name lib<name>.so.<api-version> but with a different value for <api-version>, they will not pick up the new version.

Package managers don't often package more than one version of the same library within the same distribution version because the whole distribution, including all binaries that make use of the library, is usually compiled to use a consistent version of every library before the distribution is released. Making sure that everything is consistent and that everything in a distribution is compatible with everything else is a big part of the workload for distributors.

But you can easily end up with multiple versions of a library if you've upgraded your system from one version of your distritution to another and still have some older packages requiring older library versions. Example:

• libmysqlclient16 from an older Debian, contains libmysqlclient.so.16.0.0 and symlink libmysqlclient.so.16.
• libmysqlclient18 from current Debian, contains libmysqlclient.so.18.0.0 and symlink libmysqlclient.so.18.

Since the debug output from the ld dynamic linker/loader confirms that both the victim and spy programs load the correct input file, the next step would be to verify if the kernel has actually set up the physical pages where libmyl.so is loaded in memory to be shared between the victim and spy.

In Linux this is possible to verify since kernel version 2.6.25 via the pagemap interface in the kernel that allows userspace programs to examine the page tables and related information by reading files in /proc.

The general procedure for using pagemap to find out if two processes share memory goes like this:

1. Read/proc/<pid>/maps for both processes to determine which parts of the memory space are mapped to which objects.
2. Select the maps you are interested in, in this case the pages to which libmyl.so is mapped.
3. Open /proc/<pid>/pagemap. The pagemap consists of 64-bit pagemap descriptors, one per page. The mapping between the page's address and it's descriptors address in the pagemap is page address / page size * descriptor size. Seek to the descriptors of the pages you would like to examine.
4. Read a 64-bit descriptor as an unsigned integer for each page from the pagemap.
5. Compare the page frame number (PFN) in bits 0-54 of the page descriptor between the libmyl.so pages for victim and spy. If the PFNs match, the two processes are sharing the same physical pages.

The following sample code illustrates how the pagemap can be accessed and printed from within the process. It uses dl_iterate_phdr() to determine the virtual address of each shared library loaded into the processes memory space, then looks up and prints the corresponding pagemap from /proc/<pid>/pagemap.

#define _GNU_SOURCE
#include <stdio.h>
#include <stdlib.h>
#include <unistd.h>
#include <string.h>
#include <inttypes.h>
#include <sys/stat.h>
#include <fcntl.h>
#include <link.h>
#include <errno.h>
#include <error.h>

#define E_CANNOT_OPEN_PAGEMAP 1
#define E_CANNOT_READ_PAGEMAP 2

typedef struct __attribute__ ((__packed__)) {
    union {
        uint64_t pmd;
        uint64_t page_frame_number : 55;
        struct {
            uint64_t swap_type: 5;
            uint64_t swap_offset: 50;
            uint64_t soft_dirty: 1;
            uint64_t exclusive: 1;
            uint64_t zero: 4;
            uint64_t file_page: 1;
            uint64_t swapped: 1;
            uint64_t present: 1;
        };
    };
} pmd_t;


static int print_pagemap_for_phdr(struct dl_phdr_info *info,
                                  size_t size, void *data)
{
    struct stat statbuf;

    size_t pagesize = sysconf(_SC_PAGESIZE);

    char pagemap_path[BUFSIZ];
    int pagemap;

    uint64_t start_addr, end_addr;

    if (!strcmp(info->dlpi_name, "")) {
        return 0;
    }

    stat(info->dlpi_name, &statbuf);

    start_addr = info->dlpi_addr;
    end_addr = (info->dlpi_addr + statbuf.st_size + pagesize) & ~(pagesize-1);

     printf("\n%10p-%10p %s\n\n",
            (void *)start_addr,
            (void *)end_addr,
            info->dlpi_name);

     snprintf(pagemap_path, sizeof pagemap_path, "/proc/%d/pagemap", getpid());

     if ((pagemap = open(pagemap_path, O_RDONLY)) < 0) {
         error(E_CANNOT_OPEN_PAGEMAP, errno, 
               "cannot open pagemap: %s", pagemap_path);
     }

     printf("%10s %8s %7s %5s %8s %7s %7s\n",
            "", "", "soft-", "", "file /", "", "");
     printf("%10s %8s %7s %5s %11s %7s %7s\n",
            "address", "pfn", "dirty", "excl.",
            "shared anon", "swapped", "present");

     for (unsigned long i = start_addr; i < end_addr; i += pagesize) {
          pmd_t pmd;

          if (pread(pagemap, &pmd.pmd, sizeof pmd.pmd, (i / pagesize) * sizeof pmd) != sizeof pmd) {
              error(E_CANNOT_READ_PAGEMAP, errno,
                    "cannot read pagemap: %s", pagemap_path);
          }

          if (pmd.pmd != 0) {
              printf("0x%10" PRIx64 " %06" PRIx64 " %3d %5d %8d %9d %7d\n", i,
                     (unsigned long)pmd.page_frame_number,
                     pmd.soft_dirty,
                     pmd.exclusive,
                     pmd.file_page,
                     pmd.swapped,
                     pmd.present);
          }
    }

    close(pagemap);

    return 0;
}

int main()
{
    dl_iterate_phdr(print_pagemap_for_phdr, NULL);

    exit(EXIT_SUCCESS);
}

The output of the program should look similar to the following:

$ sudo ./a.out

0x7f935408d000-0x7f9354256000 /lib/x86_64-linux-gnu/libc.so.6

                      soft-         file /                
   address      pfn   dirty excl. shared anon swapped present
0x7f935408d000 424416   1     0        1         0       1
0x7f935408e000 424417   1     0        1         0       1
0x7f935408f000 422878   1     0        1         0       1
0x7f9354090000 422879   1     0        1         0       1
0x7f9354091000 43e879   1     0        1         0       1
0x7f9354092000 43e87a   1     0        1         0       1
0x7f9354093000 424790   1     0        1         0       1
...

where:

• address is the virtual address of the page
• pfn is the pages page frame number
• soft-dirty indicates the if soft-dirty bit is set in the pages Page Table Entry (PTE).
• excl. indicates if the page is exclusively mapped (i.e. page is only mapped for this process).
• file / shared anon indicates if the page is a file pages or a shared anonymous page.
• swapped indicates if the page is currently swapped (implies present is zero).
• present indicates if the page is currently present in the processes resident set (implies swapped is zero).

(Note: I run the example program with sudo as since Linux 4.0 only users with the CAP_SYS_ADMIN capability can get PFNs from /proc/<pid>/pagemap. Starting from Linux 4.2 the PFN field is zeroed if the user does not have CAP_SYS_ADMIN. The reason for this change is to make it more difficult to exploit another memory related vulnerability, the Rowhammer attack, using the information on the virtual-to-physical mapping exposed by the PFNs.)

If you run the example program several times, you should notice that the virtual address of the page should change (due to ASLR), but the PFN for shared libraries that are in use by other processes should stay the same.

If the PFNs for libmyl.so match between the victim and spy program, I would start looking for a reason for why the attack fails in the attack code itself. If the PFNs don't match, the additional bits may give some hint why the pages are not set up to be shared. The pagemap bits indicate the following:

present file exclusive state:
   0      0     0      non-present
   1      1     0      file page mapped somewhere else
   1      1     1      file page mapped only here
   1      0     0      anonymous non-copy-on-write page (shared with parent/child)
   1      0     1      anonymous copy-on-write page (or never forked)

Copy-on-write pages in (MAP_FILE | MAP_PRIVATE) areas are anonymous in this context.

Bonus: To obtain the number of times a page has been mapped, the PFN can be used to look up the page in /proc/kpagecount. This file contains a 64-bit count of the number of times each page is mapped, indexed by PFN.