Centos – bin/ld: cannot find -l install BigARTM – CentOS v7

ccentosgcclibrariesmake

I am trying to install BigARTM according to this tutorial on Centos 7.
But I am getting this error:

/bin/ld: cannot find -lboost_thread-mt
/bin/ld: cannot find -lboost_program_options-mt
/bin/ld: cannot find -lboost_date_time-mt
/bin/ld: cannot find -lboost_filesystem-mt
/bin/ld: cannot find -lboost_iostreams-mt
/bin/ld: cannot find -lboost_system-mt
/bin/ld: cannot find -lboost_chrono-mt
/bin/ld: cannot find -lboost_timer-mt
/usr/lib/gcc/x86_64-redhat-linux/4.8.5/../../../../lib64/libpthread.a(libpthread.o): In function 'sem_open':
(.text+0x6823): warning: the use of 'mktemp' is dangerous, better use 'mkstemp'
collect2: error: ld returned 1 exit status
make[2]: [bin/bigartm] Error 1
make[1]:  [src/bigartm/CMakeFiles/bigartm.dir/all] Error 2
make:  [all] Error 2

I installed the 'Development Tools' using yum, and I found the libraries in /usr/lib64 

rpm -ql boost-devel | grep '_program-\|_date_time-\|_thread-\|_filesystem-\|_iostreams-\|_system-\|_chrono-\|_timer-' 
/usr/lib64/libboost_chrono-mt.so
/usr/lib64/libboost_date_time-mt.so
/usr/lib64/libboost_filesystem-mt.so
/usr/lib64/libboost_iostreams-mt.so
/usr/lib64/libboost_system-mt.so
/usr/lib64/libboost_thread-mt.so
/usr/lib64/libboost_timer-mt.so

I also installed boost-devel sudo yum install boost-devel

For libboost_thread lib I ran the following commands:

[oshri@analytics build]$ rpm -ql boost-devel | grep  libboost_thread
/usr/lib64/libboost_thread-mt.so
/usr/lib64/libboost_thread.a
/usr/lib64/libboost_thread.so
[oshri@analytics build]$  locate thread | grep libboost_thread
/home/oshri/PycharmProjects/playground/libboost_thread-mt.so
/usr/lib/libboost_thread-mt.so
/usr/lib/libboost_thread.a
/usr/lib/libboost_thread.so
/usr/lib/gcc/x86_64-redhat-linux/4.8.2/libboost_thread-mt.so
/usr/lib/gcc/x86_64-redhat-linux/4.8.2/libboost_thread.a
/usr/lib/gcc/x86_64-redhat-linux/4.8.2/libboost_thread.so
/usr/lib64/libboost_thread-mt.so
/usr/lib64/libboost_thread-mt.so.1.53.0
/usr/lib64/libboost_thread.a
/usr/lib64/libboost_thread.so
/usr/lib64/libboost_thread.so.1.59.0
/usr/lib64 (copy)/libboost_thread-mt.so.1.53.0
[oshri@analytics build]$  locate -e thread | grep libboost_thread
/usr/lib/libboost_thread.a
/usr/lib/gcc/x86_64-redhat-linux/4.8.2/libboost_thread.a
/usr/lib64/libboost_thread-mt.so
/usr/lib64/libboost_thread-mt.so.1.53.0
/usr/lib64/libboost_thread.a
/usr/lib64/libboost_thread.so
/usr/lib64/libboost_thread.so.1.59.0
/usr/lib64 (copy)/libboost_thread-mt.so.1.53.0

I tried three ways:

  1. To create a link (using ln) from /usr/lib64/libboost_<name>-mt.so to /usr/lib64/lboost_<name>-mt.so and to add the /usr/lib64 to LD_LIBRARY_PATH.
  2. To create a link from /usr/lib64/libboost_<name>-mt.so to
    /usr/lib/lboost_<name>-mt.so.
  3. To add LIBS = -L /usr/lib64 to the MakeFile.

For example:

[oshri@analytics build]$ ls -l /usr/lib/\*program\*
lrwxrwxrwx 4 root root 37 Jan 19 17:38 /usr/lib/lboost_program_options-mt -> libboost_program_options-mt.so.1.53.0
lrwxrwxrwx 4 root root 37 Jan 19 17:38 /usr/lib/lboost_program_options-mt.so -> libboost_program_options-mt.so.1.53.0
[oshri@analytics build]$ ls -l /usr/lib64/\*program\*
lrwxrwxrwx 4 root root     37 Jan 19 17:38 /usr/lib64/lboost_program_options-mt.so -> libboost_program_options-mt.so.1.53.0
lrwxrwxrwx 4 root root     37 Jan 19 17:38 /usr/lib64/libboost_program_options-mt.so -> libboost_program_options-mt.so.1.53.0
-rwxr-xr-x 1 root root 468952 Nov  5 20:58 /usr/lib64/libboost_program_options-mt.so.1.53.0
lrwxrwxrwx 1 root root     34 Jan 19 17:38 /usr/lib64/libboost_program_options.so -> libboost_program_options.so.1.53.0
-rwxr-xr-x 1 root root 468944 Nov  5 20:58 /usr/lib64/libboost_program_options.so.1.53.0
[oshri@analytics build]$ printenv LD_LIBRARY_PATH
/home/oshri/PycharmProjects/EuroSalesOptimization/runtime/:/usr/lib64/

Best Answer

On a 64-bit Centos7 virt with no particularly special modifications (lib versus lib64 directories have been problematical when building on Linux in the past, hence my mentioning the arch):

# yum -y install git make cmake '@Development Tools' boost-devel
# git clone --branch=stable https://github.com/bigartm/bigartm.git
# cd bigartm
# mkdir build && cd build
# cmake ..
# make
... jeopardy music ...

Okay, it takes until 99% through the build to fail. With a verbose build we can discover the precise command (with some linefeeds manually added to avoid horizontal scrolling insanity):

# make VERBOSE=1
...
/usr/bin/c++    -Wall -pthread -fPIC -std=c++11 -O3 -DNDEBUG  
  -static CMakeFiles/bigartm.dir/srcmain.cc.o
  CMakeFiles/bigartm.dir/__/artm/cpp_interface.cc.o  -o ../../bin/bigartm
  -rdynamic ../../lib/libmessages_proto.a ../../lib/libartm-static.a
  ../../lib/libprotobuf.a ../../lib/libgoogle-glog.a -lboost_thread-mt
  -lboost_program_options-mt -lboost_date_time-mt -lboost_filesystem-mt
  -lboost_iostreams-mt -lboost_system-mt -lboost_chrono-mt -lboost_timer-mt
  ../../lib/libmessages_proto.a ../../lib/libinternals_proto.a ../../lib/libgflags.a
  -lpthread 
/usr/bin/ld: cannot find -lboost_thread-mt
/usr/bin/ld: cannot find -lboost_program_options-mt
/usr/bin/ld: cannot find -lboost_date_time-mt
/usr/bin/ld: cannot find -lboost_filesystem-mt
/usr/bin/ld: cannot find -lboost_iostreams-mt
/usr/bin/ld: cannot find -lboost_system-mt
/usr/bin/ld: cannot find -lboost_chrono-mt
/usr/bin/ld: cannot find -lboost_timer-mt
/usr/bin/ld: cannot find -lpthread
/usr/bin/ld: cannot find -lstdc++
/usr/bin/ld: cannot find -lm
/usr/bin/ld: cannot find -lpthread
/usr/bin/ld: cannot find -lc
collect2: error: ld returned 1 exit status
make[2]: *** [bin/bigartm] Error 1

Not finding the C library, well now that's problematical. A relevant next question is what precise files was ld looking for that it could not find, easily solved with sysdig:

# sysdig -p '%fd.name' proc.name contains ld

And elsewhere re-run make and lo! static *.a files are being looked for according to sysdig (which, as ld already told us, do not exist). So there are at least two possible solutions here, on the one hand to not compile static, or the other to get all those various static libraries installed. Let's go with the not-static option, given handy the BUILD_STATIC_LIBS flag in the CMakeLists.txt file.

# cd .. && rm -rf build
# mkdir build && cd build
# cmake -D BUILD_STATIC_LIBS=OFF ..

Well that didn't help. Okay, let's manually edit the CMakeLists.txt file and turn off STATIC builds...

# cd .. && rm -rf build
# vi CMakeLists.txt
...
... make stuff again ...
[ 91%] Built target artm-static
Linking CXX shared library ../../lib/libartm.so
/usr/bin/ld: cannot find -lgflags-static
collect2: error: ld returned 1 exit status

Whoops, looks like this project really really really wants its static libraries. Some

yum whatprovides '*libboost_thread-mt.a'
yum whatprovides '*libc.a'
...
yum -y install glibc-static boost-static

churn later and then

cd ..
git checkout CMakeLists.txt
... build again ...
...
SystemError: Cannot compile 'Python.h'. Perhaps you need to install python-dev|python-devel.

Good to know, after all that other work...

# yum -y install git make cmake '@Development Tools' boost-devel glibc-static \
  boost-static python-devel
# git clone --branch=stable https://github.com/bigartm/bigartm.git
# cd bigartm
# mkdir build && cd build
# cmake ..
# make

This sort of stuff should be moved into an RPM file, so those dependencies can all be listed instead of requiring folks to flail around with git clones and so-so docs...

Related Question