R CMD SHLIB外部库链接问题(在Ubuntu/Linux上)

发布于 2025-01-31 01:35:19 字数 6805 浏览 2 评论 0原文

我有一个C文件,我试图将其变成一个共享对象,以读取为R。我想将其链接到OpenBlas。我正在从源中编译R(并将其链接到OpenBlas),因为我读到使用r cmd shlib时,需要设置一个环境变量以允许外部库链接进行。 (即pkg_cflags及其用于C ++和fortran的变体),如下所示(例如):

r cmd shlib从需要外部库的源.c文件中创建一个dll(在Windows上x64)

我从源安装和编译OpenBlas,然后运行以下安装脚本以配置和编译R(以及链接OpenBlas):

#####################################################
### How to compile R and link it against OpenBLAS ###
#####################################################

# Assuming no root priviledges on the destination machine,
# so we have to compile our software locally

# set default installation directory:
INSTDIR=$HOME/opt

# NOTE: NOT INSTALLING TO /opt but $HOME/opt to avoid sudo requirement

# place to store the source packages
SOURCES=$HOME/SourceBuilds

# make dirs (if they dont exist)
mkdir $INSTDIR
mkdir $SOURCES

# make sure GNU compiler is being used
CC="gcc"
CFLAGS="-O3"
CXX="g++"
CXXFLAGS="-O3"
CPICFLAGS="-fPIC"
FPICFLAGS="-fPIC"
CXXPICFLAGS="-fPIC"

BLAS_LIBS="-L${INSTDIR}/OpenBLAS/lib -lopenblas"
DYLIB_LDFLAGS="-L${INSTDIR}/OpenBLAS/lib -lopenblas"

# THIS IS CRITICAL, AS THESE FLAGS SHOULD BE SET FOR USE WITH "R CMD SHLIB"
PKG_CFLAGS='-O3 -lopenblas'
PKG_CXXFLAGS='-O3 -lopenblas'
PKG_CPPFLAGS='-O3 -lopenblas'
PKG_FFLAGS='-O3 -lopenblas'

# NOTE: Trying 'march=znver2' is not recognized as a valid option. Likely OS is too old.

###################
### Compiling R ###
###################

cd $SOURCES

# downlad tarball 
wget https://cran.rstudio.com/src/base/R-4/R-4.2.0.tar.gz

# extract tarball
tar -xf R-4.2.0.tar.gz

# change to source directory
cd R-4.2.0

# We want all R-packages to be compiled with optimization flags
mkdir ${HOME}/.R
echo CC=${CC} > ${HOME}/.R/Makevars
echo CXX=${CXX} >> ${HOME}/.R/Makevars
echo CFLAGS= -std=gnu99 ${CFLAGS} >> ${HOME}/.R/Makevars
echo CXXFLAGS=${CXXFLAGS} >> ${HOME}/.R/Makevars

# configure with local prefix and shared-blas
./configure --prefix=${INSTDIR}/R_OpenBLAS \
    --with-x --with-cairo \
    --with-pic \
    --enable-R-profiling \
    --enable-memory-profiling \
    --enable-R-shlib \
    --disable-java \
    --with-blas \
    --with-lapack \
    --with-readline \
    --with-ICU \
    --enable-static \
    --disable-shared \
    --with-libpng \
    --with-jpeglib \
    --with-libtiff \
    --disable-prebuilt-html \
    --with-recommended-packages=no
#   --enable-BLAS-shlib \
#    2>&1 | tee configure.log

# compile
make -j8
#2>&1 | tee build.log

# install
make install

### now we manually link R's Blas against our newly
### compiled OpenBLAS

# sometimes the 'lib'-dir appears as 'lib64'
LIBSUFFIX=lib/R/lib

if [ -d ${INSTDIR}/R_OpenBLAS/lib64 ]
then
  LIBSUFFIX=lib64/R/lib
fi

# move old blas-library
mv ${INSTDIR}/R_OpenBLAS/${LIBSUFFIX}/libRblas.so ${INSTDIR}/R_OpenBLAS/${LIBSUFFIX}/libRblas.so.backup

# create a symbolic link to the OpenBLAS library
#ln -s ${INSTDIR}/OpenBLAS/lib/libopenblas.so ${INSTDIR}/R_OpenBLAS/${LIBSUFFIX}/libRblas.so
#ln -s ${INSTDIR}/OpenBLAS/lib/libopenblas.so /usr/lib/libRblas.so

ln -s ${INSTDIR}/OpenBLAS/lib/libopenblas_zenp-r0.3.20.so ${INSTDIR}/R_OpenBLAS/${LIBSUFFIX}/libRblas.so
ln -s ${INSTDIR}/OpenBLAS/lib/libopenblas_zenp-r0.3.20.so /usr/lib/libRblas.so

### R is now linked against OpenBLAS.
### In order to use this R-Version over the old one, do this:

echo export PATH=${INSTDIR}/R_OpenBLAS/bin:${PATH} >> ${HOME}/.bashrc

# do this for current session:
export PATH=${INSTDIR}/R_OpenBLAS/bin:${PATH}

# check that correct R is being called now
which R

# The libraries installed by the new install script are not found by call to '.libPaths()'
export R_LIBS_SITE=/usr/local/lib/R/site-library:$R_LIBS_SITE
export R_LIBS=/usr/local/lib/R/site-library:$R_LIBS
export R_LIBS_USER=/usr/local/lib/R/site-library:$R_LIBS_USER

# NOTE: TO VALIDATE INSTALL, OPEN RSTUDIO AND TYPE IN: sessionInfo()
# RESULT SHOULD BE: Matrix products: default
# BLAS/LAPACK: /home/<username>/opt/OpenBLAS/lib/libopenblas_zenp-r0.3.20.so

然后,我在有问题的C文件上运行R CMD SHLIB像这样:

gcc -I"/home/<username>/opt/R_OpenBLAS/lib/R/include" -DNDEBUG   -I/usr/local/include   -fpic  -std=gnu99 -O3 -c hergmod.c -o hergmod.o
gcc -shared -L/home/<username>/opt/R_OpenBLAS/lib/R/lib -L/usr/local/lib -o hergmod.so hergmod.o -L/home/<username>/opt/R_OpenBLAS/lib/R/lib -lR

显然那里有一些冗余,但我不相信任何会引起问题的东西。从这个编译线看来,一切看起来都不错。但是,我没有看到任何性能提高,在R cmd shlib创建的共享对象上运行ldd后,我看到以下内容:

$ ldd hergmod.so 
    linux-vdso.so.1 (0x00007ffeff3f0000)
    libR.so => not found
    libc.so.6 => /lib/x86_64-linux-gnu/libc.so.6 (0x00007f429dc3e000)

libr.so不是发现显然是一个问题,但是此列表也应该比这更全面。例如,在未从源(而不是尝试链接openblas)编译的R安装上,当我在LDD上运行r cmd shlib时,是这样:

$ ldd hergmod.so 
    linux-vdso.so.1 (0x00007ffcb55ce000)
    libR.so => /usr/lib/libR.so (0x00007f8ee3ec8000)
    libc.so.6 => /lib/x86_64-linux-gnu/libc.so.6 (0x00007f8ee3ad7000)
    libblas.so.3 => /usr/lib/x86_64-linux-gnu/libblas.so.3 (0x00007f8ee386a000)
    libm.so.6 => /lib/x86_64-linux-gnu/libm.so.6 (0x00007f8ee34cc000)
    libreadline.so.7 => /lib/x86_64-linux-gnu/libreadline.so.7 (0x00007f8ee3283000)
    libpcre.so.3 => /lib/x86_64-linux-gnu/libpcre.so.3 (0x00007f8ee3012000)
    liblzma.so.5 => /lib/x86_64-linux-gnu/liblzma.so.5 (0x00007f8ee2dec000)
    libbz2.so.1.0 => /lib/x86_64-linux-gnu/libbz2.so.1.0 (0x00007f8ee2bdc000)
    libz.so.1 => /lib/x86_64-linux-gnu/libz.so.1 (0x00007f8ee29bf000)
    librt.so.1 => /lib/x86_64-linux-gnu/librt.so.1 (0x00007f8ee27b7000)
    libdl.so.2 => /lib/x86_64-linux-gnu/libdl.so.2 (0x00007f8ee25b3000)
    libicuuc.so.60 => /usr/lib/x86_64-linux-gnu/libicuuc.so.60 (0x00007f8ee21fb000)
    libicui18n.so.60 => /usr/lib/x86_64-linux-gnu/libicui18n.so.60 (0x00007f8ee1d5a000)
    libgomp.so.1 => /usr/lib/x86_64-linux-gnu/libgomp.so.1 (0x00007f8ee1b2b000)
    libpthread.so.0 => /lib/x86_64-linux-gnu/libpthread.so.0 (0x00007f8ee190c000)
    /lib64/ld-linux-x86-64.so.2 (0x00007f8ee474e000)
    libtinfo.so.5 => /lib/x86_64-linux-gnu/libtinfo.so.5 (0x00007f8ee16e2000)
    libicudata.so.60 => /usr/lib/x86_64-linux-gnu/libicudata.so.60 (0x00007f8edfb39000)
    libstdc++.so.6 => /usr/lib/x86_64-linux-gnu/libstdc++.so.6 (0x00007f8edf7b0000)
    libgcc_s.so.1 => /lib/x86_64-linux-gnu/libgcc_s.so.1 (0x00007f8edf598000)

我高度怀疑问题是配置和缺少的配置变量,或者可能需要删除其中一个或几个。我不确定问题在哪里,希望有人在R中有一个想法。任何技巧/指示都将不胜感激。谢谢!

I have a C file that I am trying to turn into a shared object to be read into R. I want to link that SO with OpenBLAS. I am compiling R from source (and linking it to OpenBLAS) since I have read that there is an environment variable that needs to be set in order to allow external library linking to take place when using R CMD SHLIB (namely PKG_CFLAGS and their variants for C++ and Fortran), as suggested here (for example):

R CMD SHLIB to create a DLL from a source .c file which needs an external library (on Windows x64)

I install and compile OpenBLAS from source, and then run the following install script for configuring and compiling R (along with linking OpenBLAS):

#####################################################
### How to compile R and link it against OpenBLAS ###
#####################################################

# Assuming no root priviledges on the destination machine,
# so we have to compile our software locally

# set default installation directory:
INSTDIR=$HOME/opt

# NOTE: NOT INSTALLING TO /opt but $HOME/opt to avoid sudo requirement

# place to store the source packages
SOURCES=$HOME/SourceBuilds

# make dirs (if they dont exist)
mkdir $INSTDIR
mkdir $SOURCES

# make sure GNU compiler is being used
CC="gcc"
CFLAGS="-O3"
CXX="g++"
CXXFLAGS="-O3"
CPICFLAGS="-fPIC"
FPICFLAGS="-fPIC"
CXXPICFLAGS="-fPIC"

BLAS_LIBS="-L${INSTDIR}/OpenBLAS/lib -lopenblas"
DYLIB_LDFLAGS="-L${INSTDIR}/OpenBLAS/lib -lopenblas"

# THIS IS CRITICAL, AS THESE FLAGS SHOULD BE SET FOR USE WITH "R CMD SHLIB"
PKG_CFLAGS='-O3 -lopenblas'
PKG_CXXFLAGS='-O3 -lopenblas'
PKG_CPPFLAGS='-O3 -lopenblas'
PKG_FFLAGS='-O3 -lopenblas'

# NOTE: Trying 'march=znver2' is not recognized as a valid option. Likely OS is too old.

###################
### Compiling R ###
###################

cd $SOURCES

# downlad tarball 
wget https://cran.rstudio.com/src/base/R-4/R-4.2.0.tar.gz

# extract tarball
tar -xf R-4.2.0.tar.gz

# change to source directory
cd R-4.2.0

# We want all R-packages to be compiled with optimization flags
mkdir ${HOME}/.R
echo CC=${CC} > ${HOME}/.R/Makevars
echo CXX=${CXX} >> ${HOME}/.R/Makevars
echo CFLAGS= -std=gnu99 ${CFLAGS} >> ${HOME}/.R/Makevars
echo CXXFLAGS=${CXXFLAGS} >> ${HOME}/.R/Makevars

# configure with local prefix and shared-blas
./configure --prefix=${INSTDIR}/R_OpenBLAS \
    --with-x --with-cairo \
    --with-pic \
    --enable-R-profiling \
    --enable-memory-profiling \
    --enable-R-shlib \
    --disable-java \
    --with-blas \
    --with-lapack \
    --with-readline \
    --with-ICU \
    --enable-static \
    --disable-shared \
    --with-libpng \
    --with-jpeglib \
    --with-libtiff \
    --disable-prebuilt-html \
    --with-recommended-packages=no
#   --enable-BLAS-shlib \
#    2>&1 | tee configure.log

# compile
make -j8
#2>&1 | tee build.log

# install
make install

### now we manually link R's Blas against our newly
### compiled OpenBLAS

# sometimes the 'lib'-dir appears as 'lib64'
LIBSUFFIX=lib/R/lib

if [ -d ${INSTDIR}/R_OpenBLAS/lib64 ]
then
  LIBSUFFIX=lib64/R/lib
fi

# move old blas-library
mv ${INSTDIR}/R_OpenBLAS/${LIBSUFFIX}/libRblas.so ${INSTDIR}/R_OpenBLAS/${LIBSUFFIX}/libRblas.so.backup

# create a symbolic link to the OpenBLAS library
#ln -s ${INSTDIR}/OpenBLAS/lib/libopenblas.so ${INSTDIR}/R_OpenBLAS/${LIBSUFFIX}/libRblas.so
#ln -s ${INSTDIR}/OpenBLAS/lib/libopenblas.so /usr/lib/libRblas.so

ln -s ${INSTDIR}/OpenBLAS/lib/libopenblas_zenp-r0.3.20.so ${INSTDIR}/R_OpenBLAS/${LIBSUFFIX}/libRblas.so
ln -s ${INSTDIR}/OpenBLAS/lib/libopenblas_zenp-r0.3.20.so /usr/lib/libRblas.so

### R is now linked against OpenBLAS.
### In order to use this R-Version over the old one, do this:

echo export PATH=${INSTDIR}/R_OpenBLAS/bin:${PATH} >> ${HOME}/.bashrc

# do this for current session:
export PATH=${INSTDIR}/R_OpenBLAS/bin:${PATH}

# check that correct R is being called now
which R

# The libraries installed by the new install script are not found by call to '.libPaths()'
export R_LIBS_SITE=/usr/local/lib/R/site-library:$R_LIBS_SITE
export R_LIBS=/usr/local/lib/R/site-library:$R_LIBS
export R_LIBS_USER=/usr/local/lib/R/site-library:$R_LIBS_USER

# NOTE: TO VALIDATE INSTALL, OPEN RSTUDIO AND TYPE IN: sessionInfo()
# RESULT SHOULD BE: Matrix products: default
# BLAS/LAPACK: /home/<username>/opt/OpenBLAS/lib/libopenblas_zenp-r0.3.20.so

I then run R CMD SHLIB on the C file in question and the compile line looks like so:

gcc -I"/home/<username>/opt/R_OpenBLAS/lib/R/include" -DNDEBUG   -I/usr/local/include   -fpic  -std=gnu99 -O3 -c hergmod.c -o hergmod.o
gcc -shared -L/home/<username>/opt/R_OpenBLAS/lib/R/lib -L/usr/local/lib -o hergmod.so hergmod.o -L/home/<username>/opt/R_OpenBLAS/lib/R/lib -lR

Clearly some redundancy there, but I don't believe anything that will cause a problem. It seems from this compile line that everything looks good. However, I did not see any performance increase, and after running ldd on the shared object created by R CMD SHLIB I see the following:

$ ldd hergmod.so 
    linux-vdso.so.1 (0x00007ffeff3f0000)
    libR.so => not found
    libc.so.6 => /lib/x86_64-linux-gnu/libc.so.6 (0x00007f429dc3e000)

The fact that libR.so is not found is clearly a problem, but this list should be much more comprehensive than this as well. For example, on an R install that was not compiled from source (and not trying to link OpenBLAS), when I run ldd on this same file after R CMD SHLIB the result is this:

$ ldd hergmod.so 
    linux-vdso.so.1 (0x00007ffcb55ce000)
    libR.so => /usr/lib/libR.so (0x00007f8ee3ec8000)
    libc.so.6 => /lib/x86_64-linux-gnu/libc.so.6 (0x00007f8ee3ad7000)
    libblas.so.3 => /usr/lib/x86_64-linux-gnu/libblas.so.3 (0x00007f8ee386a000)
    libm.so.6 => /lib/x86_64-linux-gnu/libm.so.6 (0x00007f8ee34cc000)
    libreadline.so.7 => /lib/x86_64-linux-gnu/libreadline.so.7 (0x00007f8ee3283000)
    libpcre.so.3 => /lib/x86_64-linux-gnu/libpcre.so.3 (0x00007f8ee3012000)
    liblzma.so.5 => /lib/x86_64-linux-gnu/liblzma.so.5 (0x00007f8ee2dec000)
    libbz2.so.1.0 => /lib/x86_64-linux-gnu/libbz2.so.1.0 (0x00007f8ee2bdc000)
    libz.so.1 => /lib/x86_64-linux-gnu/libz.so.1 (0x00007f8ee29bf000)
    librt.so.1 => /lib/x86_64-linux-gnu/librt.so.1 (0x00007f8ee27b7000)
    libdl.so.2 => /lib/x86_64-linux-gnu/libdl.so.2 (0x00007f8ee25b3000)
    libicuuc.so.60 => /usr/lib/x86_64-linux-gnu/libicuuc.so.60 (0x00007f8ee21fb000)
    libicui18n.so.60 => /usr/lib/x86_64-linux-gnu/libicui18n.so.60 (0x00007f8ee1d5a000)
    libgomp.so.1 => /usr/lib/x86_64-linux-gnu/libgomp.so.1 (0x00007f8ee1b2b000)
    libpthread.so.0 => /lib/x86_64-linux-gnu/libpthread.so.0 (0x00007f8ee190c000)
    /lib64/ld-linux-x86-64.so.2 (0x00007f8ee474e000)
    libtinfo.so.5 => /lib/x86_64-linux-gnu/libtinfo.so.5 (0x00007f8ee16e2000)
    libicudata.so.60 => /usr/lib/x86_64-linux-gnu/libicudata.so.60 (0x00007f8edfb39000)
    libstdc++.so.6 => /usr/lib/x86_64-linux-gnu/libstdc++.so.6 (0x00007f8edf7b0000)
    libgcc_s.so.1 => /lib/x86_64-linux-gnu/libgcc_s.so.1 (0x00007f8edf598000)

I highly suspect that the problem is with configure and either a missing configuration variable, or perhaps one or several of them need to be removed. I am not sure where the problem lies and was hoping someone more versed in R might have an idea. Any tips/pointers would be greatly appreciated. Thanks!

如果你对这篇内容有疑问,欢迎到本站社区发帖提问 参与讨论,获取更多帮助,或者扫码二维码加入 Web 技术交流群。

扫码二维码加入Web技术交流群

发布评论

需要 登录 才能够评论, 你可以免费 注册 一个本站的账号。

评论(1

风柔一江水 2025-02-07 01:35:19

您将需要链接R库(libr.so)。如果您运行r cmd config -ldflags您将获得一些输出,例如-wl, - export-dynamic -fopenmp-wl,-bsymbolic-firstions -wl,-z,relro- l/usr/lib/r/lib -lr -lpcre -llzma -lbz2 -lz -lz -lrt -llt -lm -lm -lm -lm -licuuc -licui18n。因此,例如,在我的情况下,libr.so在/usr/lib/r/lib中。

You will need to link the R library (libR.so). If you run R CMD config --ldflags you will get some output like -Wl,--export-dynamic -fopenmp -Wl,-Bsymbolic-functions -Wl,-z,relro -L/usr/lib/R/lib -lR -lpcre -llzma -lbz2 -lz -lrt -ldl -lm -licuuc -licui18n. So for example libR.so is in /usr/lib/R/lib in my case.

~没有更多了~
我们使用 Cookies 和其他技术来定制您的体验包括您的登录状态等。通过阅读我们的 隐私政策 了解更多相关信息。 单击 接受 或继续使用网站,即表示您同意使用 Cookies 和您的相关数据。
原文