YCSB负载工具测试leveldb

郭兴平
2023-12-01

网上对有关YCSB怎么连接leveldb的讲解简直是太少了。我弄了好久才弄好分享下自己的经验,也为之后自己不用再查一遍。

首先下载需要的一些包:(我的系统版本是centos 7)

1.首先安装thrift:

wget http://mirrors.tuna.tsinghua.edu.cn/apache/thrift/0.12.0/thrift-0.12.0.tar.gz
tar zxvf thrift-0.12.0.tar.gz
yum install -y automake libtool flex bison pkgconfig 
yum install -y gcc-c++ libevent-devel zlib-devel python-devel ruby-devel openssl-devel

2.安装boost:(仅限于boost_1_67_0版本,因为老版本安装方式好像和这个有差别,我试过make一直报错)

wget https://sourceforge.net/projects/boost/files/boost/1.67.0/boost_1_67_0.tar.bz2
tar jxvf boost_1_67_0.tar.bz2
yum install gcc gcc-c++ bzip2 bzip2-devel bzip2-libs python-devel zlib-devel -y
sudo ./bootstrap.sh --prefix=/usr/local/include/boost
sudo ./b2 install

进入boost_1_67_0目录下的tools/build目录,执行:

sudo ./bootstrap.sh
sudo ./b2 install --prefix=/usr/local/boost
cp -rf /usr/local/include/boost/lib/* /usr/lib64 #一定要把库拷贝到/usr/lib64中,否则会报找不到头文件的错误

嘤嘤嘤,没执行cp -rf /usr/local/include/boost/lib/* /usr/lib64的操作,导致一直报以下错误了好几天,软连接也不行。。。

主要是报以下错误,现在终于弄好了,走了好多弯路。

processor/ProcessorTest.o:/usr/include/boost/test/test_tools.hpp:523: more undefined references to `boost::test_tools::tt_detail::check_impl(boost::test_tools::predicate_result const&, boost::unit_test::lazy_ostream const&, boost::unit_test::basic_cstring<char const>, unsigned long, boost::test_tools::tt_detail::tool_level, boost::test_tools::tt_detail::check_type, unsigned long, ...)' follow
g++: error: /usr/local/lib/libboost_unit_test_framework.a: No such file or directory
make[5]: *** [processor_test] Error 1
make[5]: Leaving directory `/home/hsk/kvstore/software/thrift-0.12.0/lib/cpp/test'
make[4]: *** [all] Error 2
make[4]: Leaving directory `/home/hsk/kvstore/software/thrift-0.12.0/lib/cpp/test'
make[3]: *** [all-recursive] Error 1
make[3]: Leaving directory `/home/hsk/kvstore/software/thrift-0.12.0/lib/cpp'
make[2]: *** [all-recursive] Error 1
make[2]: Leaving directory `/home/hsk/kvstore/software/thrift-0.12.0/lib'
make[1]: *** [all-recursive] Error 1
make[1]: Leaving directory `/home/hsk/kvstore/software/thrift-0.12.0'
make: *** [all] Error 2

设置环境变量:

vim /etc/profile之后在文件中添加
    export BOOST_INCLUDE = /usr/local/include/boost/include
    export BOOST_LIB =/usr/local/include/boost/lib
source /etc/profile

如果显示这个报错:fatal error: boost/tokenizer.hpp: No such file or directory,尝试使用

yum install boost-devel

到目前为止,boost已经安装完成。判断是否boost安装成功。创建一个cpp文件。

#include <boost/thread/thread.hpp> //包含boost头文件
#include <iostream>
#include <cstdlib>
using namespace std;

volatile bool isRuning = true;

void func1()
{
    static int cnt1 = 0;
    while(isRuning)
    {
        cout << "func1:" << cnt1++ << endl;
        sleep(1);
    }
}

void func2()
{
    static int cnt2 = 0;
    while(isRuning)
    {
        cout << "\tfunc2:" << cnt2++ << endl;
        sleep(2);
    }
}

int main()
{
    boost::thread thread1(&func1);
    boost::thread thread2(&func2);

    system("read");
    isRuning = false;

    thread2.join();
    thread1.join();
    cout << "exit" << endl;
    return 0;
}

然后执行,如果不报错则表示成功:

g++ test.cpp -g -o test -lboost_thread -lboost_system

下面回到thrift路径进行以下操作:

./bootstrap.sh
./configure --with-boost=/usr/local/include/boost
make -j20 && make install
thrift -version #验证是否成功
Thrift version 0.12.0 #显示结果表示已经安装成功

在进行make &&install 指令的时候可能会出现一些错误:

错误1:如果make error和composer有关可能是没有安装php,需要先安装pip,在这里需要安装的是pip7否则会报以下错误:

Problem 1
phpunit/phpunit 5.7.5 requires php ^5.6 || ^7.0 -> your PHP version (5.4.16) does not satisfy that requirement.

解决方案:

yum remove php*
rpm -Uvh https://dl.fedoraproject.org/pub/epel/epel-release-latest-7.noarch.rpm
rpm -Uvh https://mirror.webtatic.com/yum/el7/webtatic-release.rpm
yum install php70w
curl -sS https://getcomposer.org/installer | php
mv composer.phar /usr/local/bin/composer
chmod +x /usr/local/bin/composer
然后在进行在有composer.ini的文件中composer install

 错误二:可能是没有安装php.xml的问题:我是使用的centos 7操作系统,如果用的是ubuntu请使用apt-get下载。

Problem 2:
Installation request for phpunit/phpunit ~4.8.36 -> satisfiable by phpunit/phpunit[4.8.36].

解决方案:

[root@localhost ~]# yum search php70w    查询可供安装的包
[root@localhost ~]# yum -y install php70w-xml.x86_64
[root@localhost ~]# systemctl restart httpd.service  
[root@localhost laravel_demo]# composer install #在含有composer.ini的路径下执行

 

3.snappy下载:

git clone https://github.com/google/snappy.git
mkdir build
cd build && cmake ../ && make
sudo cp build/libsnappy.a /usr/local/lib/

在安装snappy的时候进行cmake时有可能会显示cmake版本太老。需要进行以下操作,然后在进行cmake操作即可。

cd /home/wangrui
wget https://cmake.org/files/v3.10/cmake-3.10.2-Linux-x86_64.tar.gz
yum remove cmake
tar zxvf cmake-3.10.2-Linux-x86_64.tar.gz
cd cmake-3.10.2-Linux-x86_64
vi /etc/profile
在环境变量中添加:
export CMAKE_HOME=/home/wangrui/cmake-3.10.2-Linux-x86_64
export PATH=$PATH:$CMAKE_HOME/bin


source /etc/profile

4.levelDB安装:

git clone https://github.com/google/leveldb.git
cd leveldb
mkdir -p build && cd build
cmake -DCMAKE_BUILD_TYPE=Release .. && cmake --build 

然后再build目录下:

make
cd ..
cp -r include/leveldb /usr/local/include/
sudo cp build/libleveldb.a /usr/local/lib/

测试leveldb安装是否成功,创建demo.cc:

#include <cassert>
#include <iostream>
#include <string>
#include <leveldb/db.h>

int main() {
  leveldb::DB* db;
  leveldb::Options options;
  options.create_if_missing = true;
  leveldb::Status status = leveldb::DB::Open(options, "/tmp/testdb", &db);
  assert(status.ok());

  std::string key = "apple";
  std::string value = "A";
  std::string get;

  leveldb::Status s = db->Put(leveldb::WriteOptions(), key, value);

  if (s.ok()) s = db->Get(leveldb::ReadOptions(), key, &get);
  if (s.ok()) std::cout << "读取到的与(key=" << key << ")对应的(value=" << get << ")" << std::endl;
  else std::cout << "读取失败!" << std::endl;

  delete db;

  return 0;
}

命令行中输入:在makefile文件中添加-std=c++11这一行,如果版本g++不一致的话。

g++ -o demo demo.cc -pthread -lleveldb -std=c++11 -lsnappy
./demo
显示以下内容表示level安装成功:
读取到的与(key=apple)对应的(value=A)

5.下载libevent:

wget http://monkey.org/~provos/libevent-2.0.12-stable.tar.gz
tar xfvz libevent-2.0.12-stable.tar.gz
cd libevent-2.0.12-stable
./configure --prefix=/usr/local
make
sudo make install

下载mapkeeper:

git clone https://github.com/sears/mapkeeper.git
export MKROOT=/home/wangrui/downloads/mapkeeper
cd $MKROOT/thrift
make

返回到mapkeeper目录下

cd leveldb
make

make可能会报很多错误:

错误1,如果出现nullptr没有define的错误,错误显示如下:

/usr/local/include/leveldb/options.h:161:30: error: ‘nullptr’ was not declared in this scope
   const Snapshot* snapshot = nullptr;

改正1,应该在makefile文件中添加-std=c++11来解决这个问题。

错误2,如果出现LevelDbServer出现57行错误的话:

Error: LevelDbServer error: conversion from ‘boost::filesystem::path’ to non-scalar type ‘std::string {aka std::basic_string<char>}’ requested

改正2,修改第57行:

std::string mapName = boost::filesystem::path(itr->path().filename()).string();

错误3:

/usr/local/lib/libleveldb.a(table_builder.cc.o): In function `leveldb::TableBuilder::WriteBlock(leveldb::BlockBuilder*, leveldb::BlockHandle*)':
table_builder.cc:(.text+0x83d): undefined reference to `snappy::MaxCompressedLength(unsigned long)'
table_builder.cc:(.text+0x8a2): undefined reference to `snappy::RawCompress(char const*, unsigned long, char*, unsigned long*)'
/usr/local/lib/libleveldb.a(format.cc.o): In function `leveldb::ReadBlock(leveldb::RandomAccessFile*, leveldb::ReadOptions const&, leveldb::BlockHandle const
&, leveldb::BlockContents*)':format.cc:(.text+0x44e): undefined reference to `snappy::GetUncompressedLength(char const*, unsigned long, unsigned long*)'
format.cc:(.text+0x5bf): undefined reference to `snappy::RawUncompress(char const*, unsigned long, char*)'
collect2: error: ld returned 1 exit status

改正3,添加/usr/local/lib/libleveldb.a /usr/local/lib/libsnappy.a对应依赖,主要是因为libleveldb.a misses Snappy when being linked which would be probably in libsnappy.a in the same directory.,同时添加-std=c++11消除版本问题:

include ../Makefile.config
EXECUTABLE = mapkeeper_leveldb
all : thrift
        g++ -Wall -o $(EXECUTABLE) *cpp -I $(THRIFT_DIR)/include/thrift -I $(THRIFT_DIR)/include \
        /usr/local/lib/libleveldb.a /usr/local/lib/libsnappy.a -lboost_thread -lboost_filesystem -lthrift -lleveldb -I ../thrift/gen-cpp \
        -L $(THRIFT_DIR)/lib \
        -L ../thrift/gen-cpp -lmapkeeper \
           -Wl,-rpath,\$$ORIGIN/../thrift/gen-cpp                       \
           -Wl,-rpath,$(THRIFT_DIR)/lib \
        -std=c++11

thrift:
        make -C ../thrift

run:
        ./$(EXECUTABLE) --sync

clean :
        - rm -rf $(THRIFT_SRC) $(EXECUTABLE) *.o

wipe:
        - rm -rf data/*

错误4:如果报错是leveldbserver.cpp的255行和259行,修改相应行数的内容,修改如下:

std::shared_ptr<LevelDbServer> handler(new LevelDbServer("data"));
std::shared_ptr<TProcessor> processor(new MapKeeperProcessor(handler));
std::shared_ptr<TServerTransport> serverTransport(new TServerSocket(port));
std::shared_ptr<TTransportFactory> transportFactory(new TFramedTransportFactory());
std::shared_ptr<TProtocolFactory> protocolFactory(new TBinaryProtocolFactory());

错误5:

/usr/bin/ld: /tmp/cch6BDEI.o: undefined reference to symbol 'pthread_condattr_setclock@@GLIBC_2.3.3'
/usr/lib64/libpthread.so.0: error adding symbols: DSO missing from command line
collect2: error: ld returned 1 exit status
make: *** [all] Error 1

解决修改makefile文件,修改后文件内容如下:

include ../Makefile.config
EXECUTABLE = mapkeeper_leveldb
all : thrift
        g++ -Wall -o $(EXECUTABLE) *cpp -I $(THRIFT_DIR)/include/thrift -I $(THRIFT_DIR)/include \
        /usr/local/lib/libleveldb.a /usr/local/lib/libsnappy.a -lpthread -lboost_thread -lboost_system -lboost_filesystem -lthrift -lleveldb -I ../thrift/gen-cpp  \
        -L $(THRIFT_DIR)/lib \
        -L ../thrift/gen-cpp -lmapkeeper \
        -Wl,-rpath,\$$ORIGIN/../thrift/gen-cpp                       \
        -Wl,-rpath,$(THRIFT_DIR)/lib \
        -std=c++11
thrift:
        make -C ../thrift

run:
        ./$(EXECUTABLE) --sync

clean :
        -rm -rf $(THRIFT_SRC) $(EXECUTABLE) *.o

wipe:
        -rm -rf data/*

运行mapkeeper,成功运行表示成功:

./mapkeeper_leveldb 1 -d ~/mapkeeper/level/data

然后是下载ycsb加到mapkeeper/ycsb中(我试过好多版本都会make出错,我也很无奈是在没办法了,就选用特定这个0.1.4版本吧,能运行成功):如果想找这个版本可以到我的博客中进行下载。

wget https://github.com/downloads/brianfrankcooper/YCSB/ycsb-0.1.4.tar.gz
tar xfvz ycsb-0.1.4
cd ycsb-0.1.4
bin/ycsb load mapkeeper -s -P workloads/workloada -p "mapkeeper.host=127.0.0.1" -p "mapkeeper.port=9090"
bin/ycsb run mapkeeper -s -P workloads/workloada -p "mapkeeper.host=127.0.0.1" -p "mapkeeper.port=9090"

显示类似(截取一段):

[UPDATE], 296, 0
[UPDATE], 297, 0
[UPDATE], 298, 0
[UPDATE], 299, 0
[UPDATE], 300, 0
[UPDATE], 301, 0
[UPDATE], 302, 0
[UPDATE], 303, 0
[UPDATE], 304, 0
[UPDATE], 305, 0
[UPDATE], 306, 0
[UPDATE], 307, 0
[UPDATE], 308, 0
[UPDATE], 309, 0
[UPDATE], 310, 0
[UPDATE], 311, 0
[UPDATE], 312, 0
[UPDATE], 313, 0
[UPDATE], 314, 0
[UPDATE], 315, 0
[UPDATE], 316, 0
[UPDATE], 317, 0

表示成功。

在最后的最后,我重启了服务器发现/home文件夹下什么都没有了。哭了。。。

最后发现是重启之后没有挂载硬盘,执行下面命令就好了。

mount /dev/nvme0n1 /home/hsk/kvstore

批量测试leveldb在ycsb工作负载下运行情况,编写run.sh。(一定要在每一次load和run一个工作负载文件之后删除数据库路径“rm -rf /home/hsk/kvstore/software/mapkeeper/leveldb/data/*”下的所有文件)否则会报错。同时通过ulimit查看open file文件设置的上限,并把上限通过ulimit -n 65535指令调到64MB大小,否则会报如下错误。

insert not ok! IO error: While open a file for random read: /home/hsk/kvstore/software/mapkeeper/rocksdb/data/usertable/011619.sst: Too many open files
status: IO error: lock : /home/hsk/kvstore/software/mapkeeper/rocksdb/data/usertable/LOCK: No locks available

运行脚本如下,放在ycsb所在路径下: 

bin/ycsb load mapkeeper -s -P workloads/workloada -p "mapkeeper.host=127.0.0.1" -p "mapkeeper.port=9090" > /home/hsk/kvstore/test_data/YCSB_leveldb/1_load.txt 2>&1
bin/ycsb run mapkeeper -s -P workloads/workloada -p "mapkeeper.host=127.0.0.1" -p "mapkeeper.port=9090" > /home/hsk/kvstore/test_data/YCSB_leveldb/1_run.txt 2>&1
rm -rf /home/hsk/kvstore/software/mapkeeper/leveldb/data/*
bin/ycsb load mapkeeper -s -P workloads/workloadb -p "mapkeeper.host=127.0.0.1" -p "mapkeeper.port=9090" > /home/hsk/kvstore/test_data/YCSB_leveldb/2_load.txt 2>&1
bin/ycsb run mapkeeper -s -P workloads/workloadb -p "mapkeeper.host=127.0.0.1" -p "mapkeeper.port=9090" > /home/hsk/kvstore/test_data/YCSB_leveldb/2_run.txt 2>&1
rm -rf /home/hsk/kvstore/software/mapkeeper/leveldb/data/*
bin/ycsb load mapkeeper -s -P workloads/workloadc -p "mapkeeper.host=127.0.0.1" -p "mapkeeper.port=9090" > /home/hsk/kvstore/test_data/YCSB_leveldb/3_load.txt 2>&1
bin/ycsb run mapkeeper -s -P workloads/workloadc -p "mapkeeper.host=127.0.0.1" -p "mapkeeper.port=9090" > /home/hsk/kvstore/test_data/YCSB_leveldb/3_run.txt 2>&1
rm -rf /home/hsk/kvstore/software/mapkeeper/leveldb/data/*
bin/ycsb load mapkeeper -s -P workloads/workloadd -p "mapkeeper.host=127.0.0.1" -p "mapkeeper.port=9090" > /home/hsk/kvstore/test_data/YCSB_leveldb/4_load.txt 2>&1
bin/ycsb run mapkeeper -s -P workloads/workloadd -p "mapkeeper.host=127.0.0.1" -p "mapkeeper.port=9090" > /home/hsk/kvstore/test_data/YCSB_leveldb/4_run.txt 2>&1
rm -rf /home/hsk/kvstore/software/mapkeeper/leveldb/data/*
bin/ycsb load mapkeeper -s -P workloads/workloade -p "mapkeeper.host=127.0.0.1" -p "mapkeeper.port=9090" > /home/hsk/kvstore/test_data/YCSB_leveldb/5_load.txt 2>&1
bin/ycsb run mapkeeper -s -P workloads/workloade -p "mapkeeper.host=127.0.0.1" -p "mapkeeper.port=9090" > /home/hsk/kvstore/test_data/YCSB_leveldb/5_run.txt 2>&1
rm -rf /home/hsk/kvstore/software/mapkeeper/leveldb/data/*
bin/ycsb load mapkeeper -s -P workloads/workloadf -p "mapkeeper.host=127.0.0.1" -p "mapkeeper.port=9090" > /home/hsk/kvstore/test_data/YCSB_leveldb/6_load.txt 2>&1
bin/ycsb run mapkeeper -s -P workloads/workloadf -p "mapkeeper.host=127.0.0.1" -p "mapkeeper.port=9090" > /home/hsk/kvstore/test_data/YCSB_leveldb/6_run.txt 2>&1
rm -rf /home/hsk/kvstore/software/mapkeeper/leveldb/data/*

 

 类似资料: