scylladb源码安装过程

1.安装ubuntu

http://mirrors.163.com/ubuntu-releases/18.04.4/

使用以上的国内地址下载镜像速度更快，使用的ubuntu版本是ubuntu-18.04.5-desktop-amd64.iso。

虚拟机使用vmware，安装过程略过，可能遇到卡在安装vmtools，参考https://blog.csdn.net/qq_34415586/article/details/80347813解决。

安装完成后配置为国内源，参考https://www.cnblogs.com/bug132294/p/12749192.html。

 

2.下载scylladb源码

安装git：sudo apt-get install git

拉取scylla源码，使用了国内的git源：

git clone https://github.com.cnpmjs.org/scylladb/scylla.git

切换到tag：

git checkout scylla-4.1.0 -b scylla4.1.0

拉取子项目代码:

git submodule update --init --recursive

 

3.安装依赖

增加scylladb依赖库的源：

sudo add-apt-repository -y ppa:scylladb/ppa

安装依赖包：

sudo ./install-dependencies.sh

 

4.configure

install-dependencies.sh运行完毕后会提示configure的语法：

./configure.py --enable-dpdk --mode=release --static-thrift --static-boost --static-yaml-cpp --compiler=/opt/scylladb/bin/g++-7 --cflags="-I/opt/scylladb/include -L/opt/scylladb/lib/x86-linux-gnu/" --ldflags="-Wl,-rpath=/opt/scylladb/lib"

 

4.1版本应该需要g++8，需要安装一下。更新的版本可能需要g++10，目前需要使用源码安装。

安装g++8:

sudo apt-get install g++-8

运行configure.py，修改了g++的路径：

./configure.py --enable-dpdk --mode=release --static-thrift --static-boost --static-yaml-cpp --compiler=/usr/bin/g++-8 --cflags="-I/opt/scylladb/include -L/opt/scylladb/lib/x86-linux-gnu/" --ldflags="-Wl,-rpath=/opt/scylladb/lib"

 

报错：

CMake Error at cmake/SeastarDependencies.cmake:107 (find_package):
  Could not find a configuration file for package "fmt" that is compatible
  with requested version "5.0.0".
需要安装更新版的fmt，先删除原来的：sudo apt remove libfmt-dev，再安装：

wget -c http://archive.ubuntu.com/ubuntu/pool/universe/f/fmtlib/libfmt-dev_5.2.1+ds-2_amd64.deb

sudo apt install ./libfmt-dev_5.2.1+ds-2_amd64.deb

 

重新config，又报错：

No package 'jsoncpp' found

安装：

sudo apt-get install libjsoncpp-dev

 

重新config，又报错：

No package 'libxxhash' found

安装：

wget http://ftp.cn.debian.org/debian/pool/main/x/xxhash/libxxhash0_0.8.0-1_amd64.deb

sudo apt install ./libxxhash0_0.8.0-1_amd64.deb

wget http://mirrors.ustc.edu.cn/debian/pool/main/x/xxhash/libxxhash-dev_0.8.0-1_amd64.deb

sudo apt install ./libxxhash-dev_0.8.0-1_amd64.deb\

 

重新config，又报错：

 

Traceback (most recent call last):
  File "./configure.py", line 1308, in <module>
    proc_res = subprocess.run(["thrift", "-version"], stdout=subprocess.PIPE, stderr=subprocess.STDOUT)
  File "/usr/lib/python3.6/subprocess.py", line 423, in run
    with Popen(*popenargs, **kwargs) as process:
  File "/usr/lib/python3.6/subprocess.py", line 729, in __init__
    restore_signals, start_new_session)
  File "/usr/lib/python3.6/subprocess.py", line 1364, in _execute_child
    raise child_exception_type(errno_num, err_msg, err_filename)
FileNotFoundError: [Errno 2] No such file or directory: 'thrift': 'thrift'
安装thrift：

sudo apt install thrift-compiler

 

重新config，又报错：

No package 'lua' found

查看configure.py脚本，似乎需要5.3版本，因此安装：

sudo apt-get install liblua5.3-dev

 

重新config成功

 

5.编译

命令：

ninja release -j 6

 

执行报错：

  File "./idl-compiler.py", line 25, in <module>
    import pyparsing as pp
ModuleNotFoundError: No module named 'pyparsing'

这个是python的包，因此安装：

sudo apt-get install python-pip

使用-i清华源安装，也可以先配置到配置文件：

pip install pyparsing -i https://pypi.tuna.tsinghua.edu.cn/simple

 

再执行，下一个报错：

#include <thrift/Thrift.h>

安装：

sudo apt-get install scylla-libthrift010-dev

 

报错：

build/release/gen/alternator/expressionsLexer.hpp:55:10: fatal error: antlr3.hpp: No such file or directory
 #include <antlr3.hpp>

安装：

sudo apt-get install scylla-antlr35-c++-dev

可能版本不大对，依赖没安装全，问题比较多

 

报错：

/usr/include/boost/icl/type_traits/type_to_string.hpp:56:12: error: partial specialization of ‘struct boost::icl::type_to_string<Unary<Type> >’ after instantiation of ‘struct boost::icl::type_to_string<std::__cxx11::basic_string<char> >’ [-fpermissive]

修改/usr/include/boost/icl/type_traits/type_to_string.hpp，调整了代码顺序：

namespace boost{ namespace icl
{
    //--------------------------------------------------------------------------
    template<class Type>
    struct type_to_string
    {
        /** Convert the type to it's typestring */
        static std::string apply();
    };


    //-------------------------------------------------------------------------
    template<template<class> class Templ>
    struct unary_template_to_string
    {
        static std::string apply();
    };

    template <template<class>class Unary, class Type>
    struct type_to_string<Unary<Type> >
    {
        static std::string to_string()
        {
            return unary_template_to_string<Unary>::apply()+"<"+type_to_string<Type>::apply()+">";
        }
    };

    // ---------------------------------------------------------------------------
    template<template<class,class>class Templ>
    struct binary_template_to_string
    {
        static std::string apply();
    };

    template <template<class Type1, class Type2>class Binary, class Type1, class Type2>
    struct type_to_string<Binary<Type1, Type2> >
    {
        static std::string apply()
        {
            return binary_template_to_string<Binary>::apply()+
                "<"+type_to_string<Type1>::apply()+","+type_to_string<Type2>::apply()+">";
        }
    };

    //--------------------------------------------------------------------------
    template<>inline std::string type_to_string<bool>::apply() { return "bool"; }
    template<>inline std::string type_to_string<char>::apply() { return "char"; }
    template<>inline std::string type_to_string<short>::apply(){ return "short"; }
    template<>inline std::string type_to_string<int>::apply()  { return "int"; }
    template<>inline std::string type_to_string<long>::apply() { return "long"; }
    template<>inline std::string type_to_string<long long>::apply(){ return "Long"; }

    template<>inline std::string type_to_string<unsigned char>::apply(){ return "char+"; }
    template<>inline std::string type_to_string<unsigned short>::apply(){ return "short+"; }
    template<>inline std::string type_to_string<unsigned int>::apply()  { return "int+"; }
    template<>inline std::string type_to_string<unsigned long>::apply() { return "long+"; }
    template<>inline std::string type_to_string<unsigned long long>::apply(){ return "Long+"; }

    template<>inline std::string type_to_string<float>::apply() { return "flt"; }
    template<>inline std::string type_to_string<double>::apply() { return "dbl"; }
    template<>inline std::string type_to_string<std::string>::apply() { return "string"; }

}} // namespace boost icl

 

报错：

compress.cc:24:10: fatal error: snappy-c.h: No such file or directory
 #include <snappy-c.h>

安装：

sudo apt-get install libsnappy-dev

 

报错：

g++-8: fatal error: Killed signal terminated program cc1plus

原因应该是虚拟机内存配了8G不大够用，减少线程数：ninja release -j 2

 

报错：

./alternator/rjson.hh:65:10: fatal error: rapidjson/document.h: No such file or directory
 #include <rapidjson/document.h>

安装：

sudo apt-get install rapidjson-dev

 

报错一堆gnutls相关未定义：

In file included from alternator/auth.cc:26:
/usr/include/gnutls/crypto.h:35:10: error: ‘gnutls_cipher_algorithm_t’ has not been declared
          gnutls_cipher_algorithm_t cipher,
          ^~~~~~~~~~~~~~~~~~~~~~~~~

修改alternator/auth.cc：增加#include <gnutls/gnutls.h>

 

报错：

g++-8: error: /usr/lib/x86_64-linux-gnu/libunistring.so: No such file or directory

安装：

sudo apt-get install libunistring-dev

 

报错：

build/release/gen/Cassandra.cpp:16366: error: undefined reference to 'apache::thrift::TApplicationException::write(apache::thrift::protocol::TProtocol*) const'
collect2: error: ld returned 1 exit status

修改：

sudo cp /opt/scylladb/lib/libthrift* /usr/lib

 

报错：

/usr/lib/gcc/x86_64-linux-gnu/8/../../../x86_64-linux-gnu/libboost_regex.a(icu.o):function boost::basic_regex<int, boost::icu_regex_traits>::imbue(icu_60::Locale): error: undefined reference to 'icu_60::Collator::createInstance(icu_60::Locale const&, UErrorCode&)'
/usr/lib/gcc/x86_64-linux-gnu/8/../../../x86_64-linux-gnu/libboost_regex.a(icu.o):function boost::basic_regex<int, boost::icu_regex_traits>::imbue(icu_60::Locale): error: undefined reference to 'icu_60::Collator::createInstance(icu_60::Locale const&, UErrorCode&)'
/usr/lib/gcc/x86_64-linux-gnu/8/../../../x86_64-linux-gnu/libboost_regex.a(icu.o):function boost::basic_regex<int, boost::icu_regex_traits>::imbue(icu_60::Locale): error: undefined reference to 'icu_60::Collator::createInstance(icu_60::Locale const&, UErrorCode&)'
/usr/lib/gcc/x86_64-linux-gnu/8/../../../x86_64-linux-gnu/libboost_regex.a(icu.o):function boost::basic_regex<int, boost::icu_regex_traits>::imbue(icu_60::Locale): error: undefined reference to 'icu_60::Collator::createInstance(icu_60::Locale const&, UErrorCode&)'
/usr/lib/gcc/x86_64-linux-gnu/8/../../../x86_64-linux-gnu/libicuuc.a(udata.ao):function openCommonData(char const*, int, UErrorCode*): error: undefined reference to 'icudt60_dat'
/usr/lib/gcc/x86_64-linux-gnu/8/../../../x86_64-linux-gnu/libicuuc.a(udata.ao):function openCommonData(char const*, int, UErrorCode*): error: undefined reference to 'icudt60_dat'

搜这两个函数搜到两个库

./x86_64-linux-gnu/libicuuc.so.60

./x86_64-linux-gnu/libicui18n.so.60.2

编辑build.ninja，修改以下内容：

libs = -Wl,-Bstatic -lyaml-cpp -Wl,-Bdynamic -latomic -llz4 -lz -lsnappy -ljsoncpp  -lstdc++fs  -lcrypt  -lcryptopp  -lpthread -Wl,-Bstatic -lboost_date_time -lboost_regex -licuuc -Wl,-Bdynamic -lxxhash -llua5.3 -lsystemd

修改为：

libs = -licuuc -licui18n -Wl,-Bstatic -lyaml-cpp -Wl,-Bdynamic -latomic -llz4 -lz -lsnappy -ljsoncpp  -lstdc++fs  -lcrypt  -lcryptopp  -lpthread -Wl,-Bstatic -lboost_date_time -lboost_regex -licuuc -Wl,-Bdynamic -lxxhash -llua5.3 -lsystemd

 

报错：

/usr/lib/gcc/x86_64-linux-gnu/8/../../../x86_64-linux-gnu/Scrt1.o:function _start: error: undefined reference to 'main'

此过程出错：

[18/149] LINK (stripped) build/release/test/boost/log_heap_test
FAILED: build/release/test/boost/log_heap_test 

不知道为啥没有main函数，因为是test程序，应该不影响功能，在ninja.build中注释到这个编译。还有挺多这种的，同样处理。

 

修改后编译通过。

 

6.安装

sudo apt-get install patchelf

sudo apt-get install gawk

sudo ln -s /bin/gzip /usr/bin/gzip

sudo ln -s /sbin/ifconfig /usr/sbin/ifconfig

sudo apt-get install ethtool;sudo ln -s /sbin/ethtool /usr/sbin/ethtool

sudo ln -s /bin/netstat /usr/bin/netstat

sudo apt-get install hwloc

sudo apt-get install pigz

sudo mkdir /usr/lib/lib64;sudo ln -s /usr/lib/x86_64-linux-gnu/libthread_db.so /usr/lib/lib64/libthread_db.so

sudo ln -s /lib/x86_64-linux-gnu/libthread_db-1.0.so /lib64/libthread_db-1.0.so

#这个不知道是什么配置文件，touch了一个，可能有问题

sudo mkdir /etc/crypto-policies;sudo mkdir /etc/crypto-policies/back-ends;sudo touch /etc/crypto-policies/back-ends/gnutls.config

 

python3 -m compileall ./dist/common/scripts/ ./seastar/scripts/perftune.py ./tools/scyllatop

ninja -j 3 build/release/scylla-package.tar.gz

 

生产文件：./build/release/scylla-package.tar.gz

考到其他目录解压缩后，修改一下install.sh后运行：sudo ./install.sh

adjust_bin() {
    local bin="$1"
    # We could add --set-rpath too, but then debugedit (called by rpmbuild) barfs
    # on the result. So use LD_LIBRARY_PATH in the thunk, below.
    strip $root/$prefix/libexec/$bin
    patchelf \
        --set-interpreter "$prefix/libreloc/ld.so" \
        "$root/$prefix/libexec/$bin"
    cat > "$root/$prefix/bin/$bin" <<EOF
#!/bin/bash -e
export GNUTLS_SYSTEM_PRIORITY_FILE="\${GNUTLS_SYSTEM_PRIORITY_FILE-$prefix/libreloc/gnutls.config}"
export LD_LIBRARY_PATH="$prefix/libreloc"
exec -a "\$0" "$prefix/libexec/$bin" "\$@"
EOF
    chmod +x "$root/$prefix/bin/$bin"
}

 

可以运行了：

~/software/scylla-package$ scylla --version
4.1.0-0.20201128.b443b2574