Blas：无法加载实施的实现：com.github.fommil.netlib.nativeStemblas -CentOS 7

Question

最小的繁殖步骤：（ Centos 7）

scala> import com.github.fommil.netlib.BLAS
import com.github.fommil.netlib.BLAS

scala> println(BLAS.getInstance().getClass().getName())
22/07/03 20:09:42 WARN BLAS: Failed to load implementation from: com.github.fommil.netlib.NativeSystemBLAS
22/07/03 20:09:42 WARN BLAS: Failed to load implementation from: com.github.fommil.netlib.NativeRefBLAS
com.github.fommil.netlib.F2jBLAS

我做了什么。构建 source。 链接库：

ln -sf /opt/OpenBLAS/lib/libopenblas.so /opt/OpenBLAS/lib/libblas.so
ln -sf /opt/OpenBLAS/lib/libopenblas.so /opt/OpenBLAS/lib/libblas.so.3
ln -sf /opt/OpenBLAS/lib/libopenblas.so /opt/OpenBLAS/lib/liblapack.so
ln -sf /opt/OpenBLAS/lib/libopenblas.so /opt/OpenBLAS/lib/liblapack.so.3
ln -sf /opt/OpenBLAS/lib/libopenblas.so /usr/lib64/libblas.so
ln -sf /opt/OpenBLAS/lib/libopenblas.so /usr/lib64/libblas.so.3
ln -sf /opt/OpenBLAS/lib/libopenblas.so /usr/lib64/liblapack.so
ln -sf /opt/OpenBLAS/lib/libopenblas.so /usr/lib64/liblapack.so.3

更新的ld.conf：

include ld.so.conf.d/*.conf
/usr/local/lib
/usr/lib64
/opt/OpenBLAS/lib

ran ldconfig。 设置ld_library_path

echo $LD_LIBRARY_PATH
/usr/local/lib:/usr/lib64:/opt/OpenBLAS/lib

尝试通过 lib带有-d参数

spark-shell --conf 'spark.executor.extraJavaOptions=-Ddev.ludovic.netlib.blas.nativeLib=libopenblas.so' --conf 'spark.driver.extraJavaOptions=-Ddev.ludovic.netlib.blas.nativeLib=libopenblas.so' --conf 'spark.executor.extraJavaOptions=-Ddev.ludovic.netlib.blas.nativeLibPath=/opt/OpenBLAS/lib/' --conf 'spark.driver.extraJavaOptions=-Ddev.ludovic.netlib.blas.nativeLibPath=/opt/OpenBLAS/lib/'

：：

sudo update-alternatives --config libblas.so
sudo update-alternatives --config libblas.so.3
sudo update-alternatives --config liblapack.so
sudo update-alternatives --config liblapack.so.3

Answer 1

我正在使用旧文档和新文档的混合。

spark-shell -conf'spark.executor.extrajavaoptions = -ddev.ludovic.netlib.blas.blas.blas.nativelib = libopenblas.so'-conf'-conf'spark.driver.extrajavajavaoptions =-dddev.ludovic.ludovic.netlib.blas.blas.blas.blas.blas.blas.blas.blas.blas.blas.blas。blas。blas。blas。blas。blas。 nativelib = libopenblas.so' - conf 'spark.executor.extrajavaoptions = -ddev.ludovic.netlib.blas.blas.nativelibpath =/opt/opt/open/openblas/lib/libopenblas.so' - conf 'spark.driver.extrajavaoptions = -ddev.ludovic.netlib.blas.blas.nativelibpath =/opt/opt/open/openblas/lib/libopenblas.so'

这对我有用：

scala> import dev.ludovic.netlib.NativeBLAS
scala> NativeBLAS.getInstance()
res0: dev.ludovic.netlib.NativeBLAS = dev.ludovic.netlib.blas.JNIBLAS@de4bee9

spark 3.2迁移到fommil offommil over fommil offommil。