TDengine3.0 基础操作

梁豪

2023-12-01

上周TDengine3.0 开了发布会，今天终于有时间全面的尝鲜体验一番。
感触颇多，见《TDengine3.0 踩坑实录》。
下面简单的介绍一下基础操作。
详细资料请查看官方文档：

环境：经典的3节点 (2C/4G CentOS 7.9)

192.168.0.14    c0-14
192.168.0.15    c0-15
192.168.0.16    c0-16

目录规划

目录	路径
dataDir	/taos/data
logDir	/taos/log
tempDir	/taos/tmp
软件目录	/taos/soft
core目录	/taos/core

1.安装

安装包下载地址：https://docs.taosdata.com/get-started/package/

安装包	介绍
TDengine-server-3.0.0.1-Linux-x64.tar.gz	服务端安装包，1个顶3个，包里啥都有
TDengine-client-3.0.0.1-Linux-x64.tar.gz	客户端安装包
taosTools-2.1.2-Linux-x64.tar.gz	工具包，包含备份恢复工具`taosdump`和压测工具`taosBenchmark`，不能单独使用，需要先安装客户端。

1.1.配置地址解析

vi /etc/hosts
192.168.0.14    c0-14
192.168.0.15    c0-15
192.168.0.16    c0-16

1.2.创建相关目录

mkdir -p /taos/{data,log,tmp,core,soft}

1.3.安装 TDengine Server

安装步骤和2.x一样，还是原来的三板斧。

cd /taos/soft/
tar xzf TDengine-server-3.0.0.1-Linux-x64.tar.gz
cd TDengine-server-3.0.0.1
./install.sh -e no

1.4.配置 taos.cfg

mv taos.cfg taos.cfg.bak
vi taos.cfg

因为只是尝鲜试用，配置几个参数就够了。
TDengine 3.0 很多参数都发生了变化，有兴趣的可以去看官方文档。
没啥兴趣的可以看《TDengine3.0 踩坑实录》

firstEp         c0-14:6030
secondEp        c0-15:6030
fqdn            c0-14
dataDir         /taos/data
logDir          /taos/log
tempDir         /taos/tmp

1.5.设置core目录

不管是生产还是测试环境，这个一定要设置的。不然把根目录撑爆了不要怪别人。

[root@c0-14 ]# set_core /taos/core
kernel.core_pattern = /taos/core/core-%e-%p
/taos/core/core-%e-%p

1.6.启动 TDengine 服务

systemctl start taosd

2.创建集群

添加节点的命令没有变化，和 2.6 一样。

CREATE DNODE "fqdn:port";

[root@c0-14 taos]# taos
Welcome to the TDengine Command Line Interface, Client Version:3.0.0.1
Copyright (c) 2022 by TDengine, all rights reserved.

Server is Community Edition.

taos> show dnodes;
     id      |            endpoint            | vnodes | support_vnodes |   status   |       create_time       |              note              |
=================================================================================================================================================
           1 | c0-14:6030                     |      0 |              4 | ready      | 2022-08-24 11:26:31.343 |                                |
Query OK, 1 rows in database (0.001536s)

taos> create dnode "c0-15:6030";
Query OK, 0 of 0 rows affected (0.000694s)

taos> create dnode "c0-16:6030";
Query OK, 0 of 0 rows affected (0.000651s)

taos> show dnodes;
     id      |            endpoint            | vnodes | support_vnodes |   status   |       create_time       |              note              |
=================================================================================================================================================
           1 | c0-14:6030                     |      0 |              4 | ready      | 2022-08-24 11:26:31.343 |                                |
           2 | c0-15:6030                     |      0 |              4 | ready      | 2022-08-24 11:26:55.594 |                                |
           3 | c0-16:6030                     |      0 |              4 | ready      | 2022-08-24 11:26:59.504 |                                |
Query OK, 3 rows in database (0.001595s)

细心的小朋友会发现show dnodes的输出里面多了个support_vnodes列，这个列说明了对应节点允许创建的 vnode 个数。

3.创建数据库

taos> create database test replica 3;
Query OK, 0 of 0 rows affected (8.369874s)

taos> show databases;
              name              |
=================================
 information_schema             |
 performance_schema             |
 test                           |
Query OK, 3 rows in database (0.001277s)

和 2.x 最大的区别就是多了information_schema和performance_schema两张表，对于熟悉 MySQL 的同学，是不是有点眼熟了。

TDengine 3.0 的建库语句也发生了很大变化，通过show create database 可要看到建库的参数越来越陌生了。

taos> show create database test\G;
*************************** 1.row ***************************
       Database: test
Create Database: CREATE DATABASE `test` BUFFER 96 CACHEMODEL 'none' COMP 2 DURATION 14400m WAL_FSYNC_PERIOD 3000 MAXROWS 4096 MINROWS 100 KEEP 5256000m,5256000m,5256000m PAGES 256 PAGESIZE 4 PRECISION 'ms' REPLICA 3 STRICT 'off' WAL_LEVEL 1 VGROUPS 2 SINGLE_STABLE 0
Query OK, 1 rows in database (0.000644s)

4.创建表/超级表

建表命令变化不大，不管是超级表还是子表都能用原来的命令创建。

taos> create stable stb(ts timestamp,v1 int) tags(t1 int,t2 nchar(100));
Query OK, 0 of 0 rows affected (0.007377s)

taos> show stables;
          stable_name           |
=================================
 stb                            |
Query OK, 1 rows in database (0.001822s)

taos> create table t1 using stb tags(1,'tag1');
Query OK, 0 of 0 rows affected (0.002435s)

taos> show tables;
           table_name           |
=================================
 t1                             |
Query OK, 1 rows in database (0.004632s)

5.写入/查询数据

还好，还是原来的味道。

taos> insert into t1 values(now,1);
Query OK, 1 of 1 rows affected (0.001821s)

taos> insert into t1 values(now,2);
Query OK, 1 of 1 rows affected (0.001061s)

taos> insert into t1 values(now,3);
Query OK, 1 of 1 rows affected (0.001234s)

taos> select * from t1;
           ts            |     v1      |
========================================
 2022-08-24 11:31:56.354 |           1 |
 2022-08-24 11:31:58.600 |           2 |
 2022-08-24 11:32:00.849 |           3 |
Query OK, 3 rows in database (0.001667s)

taos> select * from stb;
           ts            |     v1      |     t1      |               t2               |
=======================================================================================
 2022-08-24 11:31:56.354 |           1 |           1 | tag1                           |
 2022-08-24 11:31:58.600 |           2 |           1 | tag1                           |
 2022-08-24 11:32:00.849 |           3 |           1 | tag1                           |
Query OK, 3 rows in database (0.002319s)

6.taosBenchmark

[root@c0-14 taos]# taosBenchmark -y
[08/24 11:33:01.896648] INFO: taos client version: 3.0.0.1
[08/24 11:33:02.939009] INFO: create database: <CREATE DATABASE IF NOT EXISTS test precision 'ms';>
[08/24 11:33:04.941904] INFO: stable meters does not exist, will create one
[08/24 11:33:04.942265] INFO: create stable: <CREATE TABLE IF NOT EXISTS test.meters (ts TIMESTAMP,current float,voltage int,phase float) TAGS (groupid int,location binary(16))>
[08/24 11:33:04.944294] INFO: generate stable<meters> columns data with lenOfCols<80> * prepared_rand<10000>
[08/24 11:33:04.953617] INFO: generate stable<meters> tags data with lenOfTags<54> * childTblCount<10000>
[08/24 11:33:04.956969] INFO: start creating 10000 table(s) with 8 thread(s)
[08/24 11:33:04.959901] INFO: thread[0] start creating table from 0 to 1249
[08/24 11:33:04.960719] INFO: thread[1] start creating table from 1250 to 2499
[08/24 11:33:04.961634] INFO: thread[2] start creating table from 2500 to 3749
[08/24 11:33:04.962011] INFO: thread[3] start creating table from 3750 to 4999
[08/24 11:33:04.962637] INFO: thread[4] start creating table from 5000 to 6249
[08/24 11:33:04.963563] INFO: thread[5] start creating table from 6250 to 7499
[08/24 11:33:04.964074] INFO: thread[6] start creating table from 7500 to 8749
[08/24 11:33:04.966660] INFO: thread[7] start creating table from 8750 to 9999
[08/24 11:33:06.038676] INFO: Spent 1.0820 seconds to create 10000 table(s) with 8 thread(s), already exist 0 table(s), actual 10000 table(s) pre created, 0 table(s) will be auto created
[08/24 11:33:06.038705] INFO: record per request (30000) is larger than insert rows (10000) in progressive mode, which will be set to 10000
[08/24 11:33:06.051761] INFO: Estimate memory usage: 11.74MB

................

[08/24 11:37:14.542688] INFO: thread[3] completed total inserted rows: 12500000, 50808.82 records/second
[08/24 11:37:16.653293] INFO: thread[4] completed total inserted rows: 12500000, 50377.97 records/second
[08/24 11:37:17.576138] INFO: thread[1] completed total inserted rows: 12500000, 50185.57 records/second
[08/24 11:37:17.926826] INFO: thread[5] completed total inserted rows: 12500000, 50120.61 records/second
[08/24 11:37:18.003390] INFO: thread[6] completed total inserted rows: 12500000, 50100.36 records/second
[08/24 11:37:18.115691] INFO: thread[2] completed total inserted rows: 12500000, 50078.72 records/second
[08/24 11:37:18.904192] INFO: thread[0] completed total inserted rows: 12500000, 49922.57 records/second
[08/24 11:37:19.122251] INFO: thread[7] completed total inserted rows: 12500000, 49878.25 records/second
[08/24 11:37:19.124718] INFO: Spent 253.069134 seconds to insert rows: 100000000 with 8 thread(s) into test 395148.94 records/second
[08/24 11:37:19.124737] INFO: insert delay, min: 17.99ms, avg: 199.27ms, p90: 526.29ms, p95: 640.49ms, p99: 3429.31ms, max: 7109.14ms

taos> select count(*) from test.meters;
       count(*)        |
========================
             100000000 |
Query OK, 1 rows in database (0.415701s)

taos> use test;
Database changed.

taos> show stables;
          stable_name           |
=================================
 meters                         |
Query OK, 1 rows in database (0.001924s)

想查看超级表下有多少子表，show stables 已经不能胜任了。
TDengine 3.0 只能用以下两种方法：

taos> select count(*) from information_schema.ins_tables where stable_name='meters' and db_name='test';
       count(*)        |
========================
                 10000 |
Query OK, 1 rows in database (0.027683s)

taos> select count(*) from (select distinct tbname from meters);
       count(*)        |
========================
                 10000 |
Query OK, 1 rows in database (0.019055s)

7.数据备份恢复

TDengine 3.0 的备份恢复工具还是taosdump。
试用了一遍，效率还是那么喜人。

7.1.备份数据

创建备份目录

mkdir /taos/dump

备份数据库 test

taosdump -o /taos/dump/ -D test

[root@c0-14 taos]# cat dump_result.txt 
========== arguments config =========
taosdump version 2.1.2, commit: d237772
host: (null)
user: root
port: 0
outpath: /taos/dump//
inpath: 
resultFile: ./dump_result.txt
all_databases: false
databases: true
databasesSeq: test
schemaonly: false
with_property: true
answer_yes: false
avro codec: snappy
data_batch: 16383
thread_num: 8
allow_sys: false
escape_char: true
loose_mode: false
isDumpIn: false
arg_list_len: 0
debug_print: 0
========== DUMP OUT ========== 
# DumpOut start time:                   2022-08-24 11:41:13

#### database:                       test
# super table counter:               1

============================== TOTAL STATISTICS ============================== 
# total database count:     1
# total super table count:  1
# total child table count:  10000
# total row count:          100000000

7.2.删除数据库

taos> drop database test;
Query OK, 0 of 0 rows affected (0.913180s)

taos> show databases;
              name              |
=================================
 information_schema             |
 performance_schema             |
Query OK, 2 rows in database (0.001922s)

7.3.恢复数据

[root@c0-14 taos]# taosdump -i /taos/dump
==============================
========== arguments config =========
taosdump version 2.1.2
host: (null)
user: root
port: 0
outpath: 
inpath: /taos/dump
resultFile: ./dump_result.txt
all_databases: false
databases: false
databasesSeq: (null)
schemaonly: false
with_property: true
answer_yes: false
avro codec: snappy
data_batch: 16383
thread_num: 8
allow_sys: false
escape_char: true
loose_mode: false
isDumpIn: true
arg_list_len: 0
[0]: Restoring from test.3322624946919.1.avro-tbtags ...
............
[6]:100%
OK: [1] 10000 row(s) of file(test.3322625031319.6565.avro) be successfully dumped in!
.OK: [2] 10000 row(s) of file(test.3322625048330.7970.avro) be successfully dumped in!
.OK: [3] 10000 row(s) of file(test.3322625065322.623.avro) be successfully dumped in!
.OK: [1] 10000 row(s) of file(test.3322625031327.5310.avro) be successfully dumped in!
.OK: [2] 10000 row(s) of file(test.3322625048322.4216.avro) be successfully dumped in!
.OK: [3] 10000 row(s) of file(test.3322625065338.1873.avro) be successfully dumped in!
[3]:100%
OK: [1] 10000 row(s) of file(test.3322625031335.9059.avro) be successfully dumped in!
[1]:100%
OK: [2] 10000 row(s) of file(test.3322625048335.464.avro) be successfully dumped in!
.OK: [2] 10000 row(s) of file(test.3322625048339.9216.avro) be successfully dumped in!
[2]:100%
OK: 100000000 row(s) dumped in!

7.4.校验数据

taos> select count(*) from meters;
       count(*)        |
========================
             100000000 |
Query OK, 1 rows in database (19.131576s)

taos> select count(*) from (select distinct tbname from meters);
       count(*)        |
========================
                 10000 |
Query OK, 1 rows in database (0.173748s)