Apache IoTDB：一款面向物联网的数据库

发表于 更新于 分类于 阅读次数： 本文字数： 33k 阅读时长 ≈ 30 分钟

Apache IoTDB 是什么？

Apache IoTDB™ (Database for Internet of Things) is an IoT native database with high performance for data management and analysis, deployable on the edge and the cloud.

（图片来源：Apache IoTDB™ 官网）

特性

  • 高吞吐量读写
  • 高效的目录结构
  • 丰富的查询语义
  • 硬件成本低
  • 灵活的部署
  • 与开源生态系统的紧密集成

应用场景

  • 高端制造业
  • 本地控制器服务器
  • 云数据管理

部署

Docker 容器版

1
2
3
4
$ docker pull apache/iotdb
$ docker run -d -p 6667:6667 -p 31999:31999 -p 8181:8181 --name yuzhouwan-iotdb apache/iotdb:latest
$ docker exec -it `docker ps | grep iotdb | awk '{print $1}'` bash
$ start-cli.sh
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
IoTDB> login successfully
IoTDB> help
    <your-sql>                  execute your sql statment
    show timeseries             show how many timeseries are in iotdb
    set time_display_type=xxx   eg. long, default, ISO8601, yyyy-MM-dd HH:mm:ss.
    show time_display_type      show time display type
    set time_zone=xxx           eg. +08:00, Asia/Shanghai.
    show time_zone              show cli time zone
    set fetch_size=xxx          set fetch size when querying data from server.
    show fetch_size             show fetch size
    set max_display_num=xxx     eg. set max lines for cli to ouput, -1 equals to unlimited.

IoTDB> insert into root.ln.wf01.wt01(timestamp, status) values(100, true);
Msg: The statement is executed successfully.

IoTDB> show timeseries
+------------------------+-----+-------------+--------+--------+-----------+
|              timeseries|alias|storage group|dataType|encoding|compression|
+------------------------+-----+-------------+--------+--------+-----------+
|root.ln.wf01.wt01.status| null|      root.ln| BOOLEAN|     RLE|     SNAPPY|
+------------------------+-----+-------------+--------+--------+-----------+
Total line number = 1
It costs 0.008s

IoTDB> select * from root.ln.wf01.wt01;
+-----------------------------+------------------------+
|                         Time|root.ln.wf01.wt01.status|
+-----------------------------+------------------------+
|1970-01-01T08:00:00.100+08:00|                    true|
+-----------------------------+------------------------+
Total line number = 1
It costs 0.005s

IoTDB> exit

集群版

安装

1
2
3
4
$ git fetch upstream cluster_new:cluster_new
$ git checkout cluster_new
$ ./mvnw.sh clean package -T 1C -DskipTests
$ find . -name '*.zip' -type f -exec echo {} \;
1
2
3
4
./cluster/target/cluster-0.12.0-SNAPSHOT.zip
./distribution/target/apache-iotdb-0.12.0-SNAPSHOT-bin.zip
./server/target/iotdb-server-0.12.0-SNAPSHOT.zip
./cli/target/iotdb-cli-0.12.0-SNAPSHOT.zip
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
$ mkdir -p ~/apps/iotdb/cluster/node1
$ mkdir -p ~/apps/iotdb/cluster/node2
$ mkdir -p ~/apps/iotdb/cluster/node3
$ cp ./cluster/target/cluster-0.12.0-SNAPSHOT.zip ~/apps/iotdb/cluster/node1/
$ cp ./cluster/target/cluster-0.12.0-SNAPSHOT.zip ~/apps/iotdb/cluster/node2/
$ cp ./cluster/target/cluster-0.12.0-SNAPSHOT.zip ~/apps/iotdb/cluster/node3/
$ cd ~/apps/iotdb/cluster/node1
$ unzip cluster-0.12.0-SNAPSHOT.zip
$ cd ~/apps/iotdb/cluster/node2
$ unzip cluster-0.12.0-SNAPSHOT.zip
$ cd ~/apps/iotdb/cluster/node3
$ unzip cluster-0.12.0-SNAPSHOT.zip
$ cd ~/apps/iotdb/cluster
$ chmod +x node1/sbin/start-node.sh
$ chmod +x node2/sbin/start-node.sh
$ chmod +x node3/sbin/start-node.sh
$ sed -in-place -e 's/31999/31997/g' node1/conf/iotdb-env.sh
$ sed -in-place -e 's/31999/31998/g' node2/conf/iotdb-env.sh
$ node1/sbin/start-node.sh -internal_meta_port 9003 -internal_data_port 40010 -cluster_rpc_port 55560
$ node2/sbin/start-node.sh -internal_meta_port 9005 -internal_data_port 40012 -cluster_rpc_port 55561
$ node3/sbin/start-node.sh -internal_meta_port 9007 -internal_data_port 40014 -cluster_rpc_port 55562

使用

1
2
3
4
5
$ mkdir ~/apps/iotdb/standalone
$ cp ./cli/target/iotdb-cli-0.12.0-SNAPSHOT.zip ~/apps/iotdb/standalone/
$ cd ~/apps/iotdb/standalone
$ chmod +x sbin/start-cli.sh
$ sbin/start-cli.sh
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
---------------------
Starting IoTDB Cli
---------------------
 _____       _________  ______   ______
|_   _|     |  _   _  ||_   _ `.|_   _ \
  | |   .--.|_/ | | \_|  | | `. \ | |_) |
  | | / .'`\ \  | |      | |  | | |  __'.
 _| |_| \__. | _| |_    _| |_.' /_| |__) |
|_____|'.__.' |_____|  |______.'|_______/  version 0.12.0-SNAPSHOT


IoTDB> login successfully
IoTDB> show timeseries
+----------+-----+-------------+--------+--------+-----------+----+----------+
|timeseries|alias|storage group|dataType|encoding|compression|tags|attributes|
+----------+-----+-------------+--------+--------+-----------+----+----------+
+----------+-----+-------------+--------+--------+-----------+----+----------+
Empty set.
It costs 0.276s

IoTDB> set storage group to root.yuzhouwan
Msg: The statement is executed successfully.

IoTDB> create timeseries root.yuzhouwan.user with datatype=INT64,encoding=PLAIN
Msg: The statement is executed successfully.

IoTDB> insert into root.yuzhouwan(timestamp, user) values(1, 1)
Msg: The statement is executed successfully.

IoTDB> show timeseries
+-------------------+-----+--------------+--------+--------+-----------+----+----------+
|         timeseries|alias| storage group|dataType|encoding|compression|tags|attributes|
+-------------------+-----+--------------+--------+--------+-----------+----+----------+
|root.yuzhouwan.user| null|root.yuzhouwan|   INT64|   PLAIN|     SNAPPY|null|      null|
+-------------------+-----+--------------+--------+--------+-----------+----+----------+
Total line number = 1
It costs 0.015s

IoTDB> select * from root.yuzhouwan
+-----------------------------+-------------------+
|                         Time|root.yuzhouwan.user|
+-----------------------------+-------------------+
|1970-01-01T08:00:00.001+08:00|                  1|
+-----------------------------+-------------------+
Total line number = 1
It costs 0.005s

IoTDB> select * from root.*
+-----------------------------+-------------------+
|                         Time|root.yuzhouwan.user|
+-----------------------------+-------------------+
|1970-01-01T08:00:00.001+08:00|                  1|
+-----------------------------+-------------------+
Total line number = 1
It costs 0.004s

IoTDB> select * from root
+-----------------------------+-------------------+
|                         Time|root.yuzhouwan.user|
+-----------------------------+-------------------+
|1970-01-01T08:00:00.001+08:00|                  1|
+-----------------------------+-------------------+
Total line number = 1
It costs 0.005s

IoTDB> show child paths root
+--------------+
|   child paths|
+--------------+
|root.yuzhouwan|
+--------------+
Total line number = 1
It costs 0.003s

IoTDB> show child paths root.*
+-------------------+
|        child paths|
+-------------------+
|root.yuzhouwan.user|
+-------------------+
Total line number = 1
It costs 0.002s

停止

1
$ for p in `jps -ml | grep iotdb | awk '{print $1}'`; do echo $p && kill $p; done

待完善列表

  • 模拟集群故障频繁重启的情况，发现 RaftServer 会出现 OOM 问题，可能存在线程泄露问题
  • 可以考虑使用 Google 设计的有界一致性 Hash 算法，替换原生的一致性算法，以提高集群扩缩容时的平衡性

架构

总体架构图

Apache IoTDB Architecture

（图片来源：iotdb.apache.org™）

写入链路

sequenceDiagram

participant Thrift
participant TSIService
participant TSServiceImpl
participant Planner
participant PlanExecutor
participant StorageEngine
participant StorageGroupProcessor
participant TsFileProcessor
participant AbstractMemTable
participant InsertRowPlan
participant WritableMemChunk
participant MeasurementSchema
participant TVList
participant ExclusiveWriteLogNode
participant TsFileFlushPolicy

opt insert into root.yuzhouwan(timestamp, value) values(1,1)
Thrift ->>+ TSIService : getResult
TSIService ->>+ TSServiceImpl : executeStatement
end

opt deviceId: root.yuzhouwan, time: 1, measurements: [value], values: [1.0]
TSServiceImpl ->>+ Planner : parseSQLToPhysicalPlan
Planner ->>- TSServiceImpl : PhysicalPlan

TSServiceImpl ->>+ TSServiceImpl : executeUpdateStatement
TSServiceImpl ->>+ TSServiceImpl : executeNonQueryPlan
TSServiceImpl ->>+ TSServiceImpl : executeNonQuery

TSServiceImpl ->> PlanExecutor : processNonQuery
PlanExecutor ->> StorageEngine : insert
StorageEngine ->> StorageGroupProcessor : insert
StorageGroupProcessor ->> StorageGroupProcessor : insert

StorageGroupProcessor ->>+ StorageGroupProcessor : writeLock


opt MemTable
StorageGroupProcessor ->>+ StorageEngine : getTimePartition

alt enable_partition is true
StorageEngine ->> StorageEngine : time / timePartitionInterval
else
StorageEngine ->> StorageEngine : 0
end

alt !isSequence && time > lastFlushTime && enable_discard_out_of_order_data is true
StorageGroupProcessor -->> TSServiceImpl : return
end

StorageEngine ->>- StorageGroupProcessor : long timePartitionId

StorageGroupProcessor ->> StorageGroupProcessor : insertToTsFileProcessor
StorageGroupProcessor ->> TsFileProcessor : insert
TsFileProcessor ->> AbstractMemTable : insert
AbstractMemTable ->>+ InsertRowPlan : getIndex
InsertRowPlan ->>- AbstractMemTable : long index
AbstractMemTable ->> AbstractMemTable : updatePlanIndexes
AbstractMemTable ->> AbstractMemTable : write
AbstractMemTable ->>+ AbstractMemTable : createIfNotExistAndGet
AbstractMemTable ->>- AbstractMemTable : IWritableMemChunk
AbstractMemTable ->> WritableMemChunk : write(insertTime, objectValue)

WritableMemChunk ->>+ MeasurementSchema : getType
MeasurementSchema ->>- WritableMemChunk : TSDataType

alt BOOLEAN
WritableMemChunk ->> WritableMemChunk : putBoolean
else INT32
WritableMemChunk ->> WritableMemChunk : putInt
else INT64
WritableMemChunk ->> WritableMemChunk : putLong
else FLOAT
WritableMemChunk ->> WritableMemChunk : putFloat
else DOUBLE
WritableMemChunk ->> WritableMemChunk : putDouble
else TEXT
WritableMemChunk ->> WritableMemChunk : putBinary
end

WritableMemChunk ->> TVList : putFloat
end


alt enable_wal is true
opt WAL
TsFileProcessor ->>+ TsFileProcessor : getLogNode
TsFileProcessor ->>- TsFileProcessor : WriteLogNode
TsFileProcessor ->> ExclusiveWriteLogNode : write

ExclusiveWriteLogNode ->> ExclusiveWriteLogNode : putLog

alt >= flush_wal_threshold
ExclusiveWriteLogNode ->> ExclusiveWriteLogNode : sync
end

end
end


opt Flush
StorageGroupProcessor ->>+ TsFileProcessor : shouldFlush
TsFileProcessor ->>- StorageGroupProcessor : boolean

alt memSize of memtable >= memtable_size_threshold or pointNum of memtable >= avg_series_point_number_threshold
StorageGroupProcessor ->> TsFileFlushPolicy : apply
TsFileFlushPolicy ->>+ TsFileProcessor : shouldClose
TsFileProcessor ->>- TsFileFlushPolicy : boolean
alt shouldClose is true
TsFileFlushPolicy ->> StorageGroupProcessor : asyncCloseOneTsFileProcessor
else
TsFileFlushPolicy ->> TsFileProcessor : asyncFlush
end
end

end



StorageGroupProcessor ->>- StorageGroupProcessor : writeUnlock

StorageGroupProcessor -->> TSServiceImpl : may throw exception
end

TSServiceImpl ->>- TSServiceImpl : boolean succ
TSServiceImpl ->>- TSServiceImpl : TSStatus
TSServiceImpl ->>- TSServiceImpl : TSExecuteStatementResp
TSServiceImpl ->>- TSIService : TSExecuteStatementResp
TSIService ->>- Thrift : executeStatement_result
点击这里查看完整大图
写入数据时，时间粒度支持 ms / us / ns 三种，默认为 ms，且存储时间粒度统一为 ms
参数 enable_partition 控制是否 partition 分区，默认为 false
参数 enable_discard_out_of_order_data 控制是否丢弃乱序数据，默认为 false
参数 enable_wal 控制是否开启写 WAL 日志，默认为 true
参数 flush_wal_threshold 控制 MemTable 中缓存了多少条记录后，才会同步到 WAL 日志文件中，默认为 10000
参数 memtable_size_threshold 控制 MemTable 中缓存占用多大内存后，才会 Flush 到 TSFile 数据文件中，默认为 1G
参数 avg_series_point_number_threshold 控制 MemTable 中缓存了多少条记录后，才会 Flush 到 TSFile 数据文件中，默认为 100000

查询链路

sequenceDiagram

participant Thrift
participant TSIService
participant TSServiceImpl
participant Planner
participant PhysicalPlan
participant QueryPlan
participant RawDataQueryPlan
participant IPlanExecutor
participant IQueryRouter
participant RawDataQueryExecutor
participant RawQueryDataSetWithoutValueFilter
participant RawQueryDataSetWithoutValueFilter.ReadTask
participant IBatchReader
participant TSQueryDataSet
participant TSExecuteStatementResp

opt select * from root.*
Thrift ->>+ TSIService : getResult
TSIService ->>+ TSServiceImpl : requestStatementId
TSServiceImpl ->>- TSIService : long statementId

TSIService ->>+ TSServiceImpl : executeStatement


TSServiceImpl ->>+ Planner : parseSQLToPhysicalPlan
Planner ->>- TSServiceImpl : PhysicalPlan
TSServiceImpl ->>+ PhysicalPlan : isQuery
PhysicalPlan ->>- TSServiceImpl : boolean
alt isQuery
TSServiceImpl ->>+ TSServiceImpl : internalExecuteQueryStatement
TSServiceImpl ->>+ TSServiceImpl : getQueryResp
TSServiceImpl ->>+ TSServiceImpl : getQueryColumnHeaders

TSServiceImpl ->>+ TSServiceImpl : getWideQueryHeaders

opt root.yuzhouwan.value
TSServiceImpl ->>+ QueryPlan : getPaths
QueryPlan ->>- TSServiceImpl : list of PartialPath
end

TSServiceImpl ->>+ TSServiceImpl : getSeriesTypeByPath
TSServiceImpl ->>- TSServiceImpl : TSDataType

TSServiceImpl -->>- TSServiceImpl : list of respColumns and list of columnTypes
end


alt plan instanceof is true
TSServiceImpl ->>+ RawDataQueryPlan : getDeduplicatedPaths().size()
RawDataQueryPlan ->>- TSServiceImpl : long deduplicatedPathNum
end

TSServiceImpl ->>+ TSServiceImpl : generateQueryId
TSServiceImpl ->>- TSServiceImpl : long queryId

TSServiceImpl ->>+ TSServiceImpl : createQueryDataSet

alt execute query
TSServiceImpl ->>+ IPlanExecutor : processQuery
IPlanExecutor ->>+ IPlanExecutor : processDataQuery
IPlanExecutor ->>+ IQueryRouter : rawDataQuery
IQueryRouter ->>+ RawDataQueryExecutor : executeWithoutValueFilter
RawDataQueryExecutor ->>+ RawQueryDataSetWithoutValueFilter : new
RawQueryDataSetWithoutValueFilter ->> RawQueryDataSetWithoutValueFilter : init
RawQueryDataSetWithoutValueFilter ->>+ RawQueryDataSetWithoutValueFilter.ReadTask : new
RawQueryDataSetWithoutValueFilter.ReadTask ->>- RawQueryDataSetWithoutValueFilter : RawQueryDataSetWithoutValueFilter.ReadTask
RawQueryDataSetWithoutValueFilter.ReadTask ->> RawQueryDataSetWithoutValueFilter.ReadTask : runMayThrow
RawQueryDataSetWithoutValueFilter.ReadTask ->>+ IBatchReader : nextBatch
IBatchReader ->>- RawQueryDataSetWithoutValueFilter.ReadTask : BatchData
RawQueryDataSetWithoutValueFilter ->> RawQueryDataSetWithoutValueFilter : fillCache
RawQueryDataSetWithoutValueFilter ->>- RawDataQueryExecutor : QueryDataSet
RawDataQueryExecutor ->>- IQueryRouter : QueryDataSet
IQueryRouter ->>- IPlanExecutor : QueryDataSet
IPlanExecutor ->>- IPlanExecutor : QueryDataSet
IPlanExecutor ->>- TSServiceImpl : QueryDataSet
end

TSServiceImpl ->>- TSServiceImpl : QueryDataSet

alt build result
TSServiceImpl ->>+ TSServiceImpl : fillRpcReturnData
TSServiceImpl ->>+ RawQueryDataSetWithoutValueFilter : fillBuffer
RawQueryDataSetWithoutValueFilter ->> TSQueryDataSet : setTime
RawQueryDataSetWithoutValueFilter ->> TSQueryDataSet : setValueList
RawQueryDataSetWithoutValueFilter ->> TSQueryDataSet : setBitmapList
RawQueryDataSetWithoutValueFilter ->>- TSServiceImpl : TSQueryDataSet

TSServiceImpl ->> TSExecuteStatementResp : setQueryDataSet
TSServiceImpl ->> TSExecuteStatementResp : setQueryId

TSServiceImpl ->>- TSServiceImpl : TSQueryDataSet
end

TSServiceImpl ->>- TSServiceImpl : TSExecuteStatementResp
TSServiceImpl ->>- TSServiceImpl : TSExecuteStatementResp
TSServiceImpl ->>- TSServiceImpl : TSExecuteStatementResp
TSServiceImpl ->>- TSIService : TSExecuteStatementResp
end

TSIService ->>- Thrift : requestStatementId_result
点击这里查看完整大图

目录结构

1
$ tree
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
.
├── data
│   ├── sequence
│   │   └── root.ln
│   │       └── 0
│   │           └── 1606030145164-1-0.tsfile
│   └── unsequence
├── system
│   ├── roles
│   ├── schema
│   │   ├── mlog.txt
│   │   ├── system.properties
│   │   └── tlog.txt
│   ├── storage_groups
│   │   └── root.ln
│   │       ├── 0
│   │       │   └── Version-100
│   │       └── upgrade
│   │           └── Version-100
│   ├── upgrade
│   │   └── upgrade.txt
│   └── users
│       └── root.profile
└── wal
    └── root.ln-1606030145164-1-0.tsfile
        └── wal1

16 directories, 9 files
1
$ cat system/schema/mlog.txt
1
2
2,root.ln
0,root.ln.wf01.wt01.status,0,2,1,,,
1
$ cat system/schema/system.properties
1
2
3
4
5
6
7
8
9
#System properties:
#Sun Nov 22 15:28:58 CST 2020
partition_interval=9223372036854775807
timestamp_precision=ms
tsfile_storage_fs=LOCAL
enable_partition=false
max_degree_of_index_node=1024
tag_attribute_total_size=700
iotdb_version=UNKNOWN
1
$ cat system/users/root.profile
1
rootc���{��Pykd��HE%
1
$ cat wal/root.ln-1606030145164-1-0.tsfile/wal1
1
8droot.ln.wf01.wt01status�O�f%

压缩策略

  • LevelCompactionTsFileManagement
  • NoCompactionTsFileManagement

性能压测

1
2
3
4
5
6
7
$ git clone --depth 1 --single-branch --branch master https://github.com/thulab/iotdb-benchmark
$ cd iotdb-benchmark
$ mvn clean package -Dmaven.test.skip=true
$ mkdir ../lib
$ find . -name '*jar' -type f -exec echo {} \; -exec cp {} ../lib/ \;
$ mv ../lib .
$ ./benchmark.sh

踩过的坑

Cannot connect to node Node

描述

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
---------------------
Starting IoTDB (Cluster Mode)
---------------------
Maximum memory allocation pool = 4096MB, initial memory allocation pool = 800MB
If you want to change this configuration, please check conf/iotdb-env.sh(Unix or OS X, if you use Windows, check conf/iotdb-env.bat).
2020-12-01 17:12:58,126 [main] INFO  org.apache.iotdb.cluster.config.ClusterDescriptor:191 - Start to read config file sbin/../conf/iotdb-cluster.properties
2020-12-01 17:12:58,159 [main] INFO  org.apache.iotdb.db.conf.IoTDBDescriptor:148 - Start to read config file file:sbin/../conf/iotdb-engine.properties
2020-12-01 17:12:58,161 [main] INFO  org.apache.iotdb.db.conf.IoTDBDescriptor:803 - allocateMemoryForRead = 347078656
2020-12-01 17:12:58,161 [main] INFO  org.apache.iotdb.db.conf.IoTDBDescriptor:804 - allocateMemoryForWrite = 2429550592
2020-12-01 17:12:58,161 [main] INFO  org.apache.iotdb.db.conf.IoTDBDescriptor:805 - allocateMemoryForSchema = 347078656
2020-12-01 17:12:58,162 [main] INFO  org.apache.iotdb.db.conf.IoTDBDescriptor:247 - IoTDB enable memory control: true
2020-12-01 17:12:58,166 [main] INFO  org.apache.iotdb.tsfile.common.conf.TSFileDescriptor:108 - Start to read config file sbin/../conf/iotdb-engine.properties
2020-12-01 17:12:58,168 [main] INFO  org.apache.iotdb.cluster.ClusterMain:90 - Running mode -s
2020-12-01 17:12:58,250 [main] INFO  org.apache.iotdb.db.engine.version.SimpleFileVersionController:115 - Version file updated, previous: /Users/benedictjin/apps/iotdb/cluster/sbin/../data/system/raftLog/0/upgrade/Version-100, current: /Users/benedictjin/apps/iotdb/cluster/sbin/../data/system/raftLog/0/upgrade/Version-200
2020-12-01 17:12:58,252 [main] INFO  org.apache.iotdb.cluster.log.manage.serializable.SyncLogDequeSerializer:786 - Recovered log meta: LogManagerMeta{ commitLogTerm=-1, commitLogIndex=-1, lastLogIndex=-1, lastLogTerm=-1, maxHaveAppliedCommitIndex=-1}, availableVersion: [0,9223372036854775807], state: HardState{currentTerm=0, voteFor=null}
2020-12-01 17:12:58,253 [main] INFO  org.apache.iotdb.cluster.log.manage.serializable.SyncLogDequeSerializer:482 - Find log type =idx log files [sbin/../data/system/raftLog/0/0-9223372036854775807-101-idx]
2020-12-01 17:12:58,260 [main] INFO  org.apache.iotdb.cluster.log.manage.serializable.SyncLogDequeSerializer:482 - Find log type =data log files [sbin/../data/system/raftLog/0/0-9223372036854775807-101-data]
2020-12-01 17:12:58,261 [main] INFO  org.apache.iotdb.cluster.log.manage.serializable.SyncLogDequeSerializer:546 - no log index file to recover
2020-12-01 17:12:58,269 [main] INFO  org.apache.iotdb.cluster.server.member.MetaGroupMember:1267 - Recovered node identifier 356238364
2020-12-01 17:12:58,270 [main] INFO  org.apache.iotdb.cluster.server.member.MetaGroupMember:1294 - The identifier of this node has been set to 356238364
2020-12-01 17:12:58,307 [main] INFO  org.apache.iotdb.db.auth.authorizer.BasicAuthorizer$InstanceHolder:87 - Authorizer provider class: org.apache.iotdb.db.auth.authorizer.LocalFileAuthorizer
2020-12-01 17:12:58,313 [main] INFO  org.apache.iotdb.db.auth.user.BasicUserManager:79 - Admin initialized
2020-12-01 17:12:58,316 [main] INFO  org.apache.iotdb.db.auth.user.BasicUserManager:79 - Admin initialized
2020-12-01 17:12:58,316 [main] INFO  org.apache.iotdb.db.auth.authorizer.BasicAuthorizer:67 - Initialization of Authorizer completes
2020-12-01 17:12:58,329 [main] INFO  org.apache.iotdb.cluster.server.member.MetaGroupMember:1189 - No partition table file found
2020-12-01 17:12:58,334 [main] INFO  org.apache.iotdb.cluster.server.RaftServer:220 - Cluster node Node(ip:127.0.0.1, metaPort:9003, nodeIdentifier:356238364, dataPort:40010, clientPort:55560) begins to set up
2020-12-01 17:12:58,348 [main] INFO  org.apache.iotdb.cluster.server.RaftServer:231 - Cluster node Node(ip:127.0.0.1, metaPort:9003, nodeIdentifier:356238364, dataPort:40010, clientPort:55560) is up
2020-12-01 17:12:58,349 [main] INFO  org.apache.iotdb.cluster.server.heartbeat.HeartbeatServer:201 - Cluster node's heartbeat Node(ip:127.0.0.1, metaPort:9003, nodeIdentifier:356238364, dataPort:40010, clientPort:55560) begins to set up
2020-12-01 17:12:58,350 [main] INFO  org.apache.iotdb.cluster.server.heartbeat.HeartbeatServer:213 - Cluster node's heartbeat Node(ip:127.0.0.1, metaPort:9003, nodeIdentifier:356238364, dataPort:40010, clientPort:55560) is up
2020-12-01 17:12:58,361 [main] WARN  org.apache.iotdb.db.service.StartupChecks:35 - iotdb.jmx.port missing from iotdb-env.sh(Unix or OS X, if you use Windows, check conf/iotdb-env.bat)
2020-12-01 17:12:58,361 [main] INFO  org.apache.iotdb.db.service.StartupChecks:50 - JDK veriosn is 8.
2020-12-01 17:12:58,361 [main] INFO  org.apache.iotdb.db.service.IoTDB:95 - Setting up IoTDB...
2020-12-01 17:12:58,362 [main] INFO  org.apache.iotdb.db.service.IoTDB:99 - recover the schema...
2020-12-01 17:12:58,402 [main] INFO  org.apache.iotdb.db.service.IoTDB:151 - spend 40ms to recover schema.
2020-12-01 17:12:58,402 [main] INFO  org.apache.iotdb.db.service.IoTDB:152 - After initializing, tsFile threshold is 0, memtableSize is 134217728
2020-12-01 17:12:58,407 [main] INFO  org.apache.iotdb.db.concurrent.IoTDBThreadPoolFactory:46 - new fixed thread pool: Flush-ServerServiceImpl, thread number: 8
2020-12-01 17:12:58,409 [main] INFO  org.apache.iotdb.db.concurrent.IoTDBThreadPoolFactory:80 - new cached thread pool: Flush-SubTask-ServerServiceImpl
2020-12-01 17:12:58,409 [main] INFO  org.apache.iotdb.db.engine.flush.pool.FlushSubTaskPoolManager:57 - Flush sub task manager started.
2020-12-01 17:12:58,409 [main] INFO  org.apache.iotdb.db.engine.flush.pool.FlushTaskPoolManager:58 - Flush task manager started.
2020-12-01 17:12:58,414 [main] INFO  org.apache.iotdb.db.cost.statistic.Measurement:142 - start measurement stats module...
2020-12-01 17:12:58,415 [main] INFO  org.apache.iotdb.db.cost.statistic.Measurement:250 - start the consuming task in the measurement stats module...
2020-12-01 17:12:58,422 [main] INFO  org.apache.iotdb.db.concurrent.IoTDBThreadPoolFactory:46 - new fixed thread pool: Recovery-Thread-Pool, thread number: 8
2020-12-01 17:12:58,427 [main] INFO  org.apache.iotdb.db.concurrent.IoTDBThreadPoolFactory:63 - new single thread pool: Begin-Recovery-Pool
2020-12-01 17:12:58,430 [main] INFO  org.apache.iotdb.db.service.thrift.ThriftService:99 - IoTDB: start RPC ServerService...
2020-12-01 17:12:58,441 [RPC-ServerServiceImpl] INFO  org.apache.iotdb.db.service.thrift.ThriftServiceThread:125 - The RPC ServerService service thread begin to run...
2020-12-01 17:12:58,544 [main] INFO  org.apache.iotdb.db.service.thrift.ThriftService:117 - IoTDB: start RPC ServerService successfully, listening on ip 0.0.0.0 port 6667
2020-12-01 17:12:58,544 [main] INFO  org.apache.iotdb.db.service.IoTDB:118 - IoTDB is set up, now may some sgs are not ready, please wait several seconds...
2020-12-01 17:12:58,554 [main] INFO  org.apache.iotdb.db.service.UpgradeSevice:112 - finish counting upgrading files, total num:0
2020-12-01 17:12:58,554 [main] INFO  org.apache.iotdb.db.service.UpgradeSevice:73 - Waiting for upgrade task pool to shut down
2020-12-01 17:12:58,554 [main] INFO  org.apache.iotdb.db.service.UpgradeSevice:78 - Upgrade service stopped
2020-12-01 17:12:58,565 [main] INFO  org.apache.iotdb.db.engine.merge.manage.MergeManager:154 - MergeManager started
2020-12-01 17:12:58,566 [main] INFO  org.apache.iotdb.db.engine.compaction.CompactionMergeTaskPoolManager:57 - Compaction task manager started.
2020-12-01 17:12:58,566 [main] INFO  org.apache.iotdb.db.service.IoTDB:137 - Congratulation, IoTDB is set up successfully. Now, enjoy yourself!
2020-12-01 17:12:58,566 [main] INFO  org.apache.iotdb.db.service.IoTDB:91 - IoTDB has started.
2020-12-01 17:12:58,569 [main] INFO  org.apache.iotdb.cluster.server.member.RaftMember:282 - Meta started
2020-12-01 17:13:03,578 [pool-12-thread-1] WARN  org.apache.iotdb.cluster.server.member.MetaGroupMember:1005 - Start up exception. Cannot connect to node Node(ip:127.0.0.1, metaPort:9005, nodeIdentifier:0, dataPort:40012, clientPort:55561). Try again in next turn.
2020-12-01 17:13:03,578 [pool-12-thread-2] WARN  org.apache.iotdb.cluster.server.member.MetaGroupMember:1005 - Start up exception. Cannot connect to node Node(ip:127.0.0.1, metaPort:9007, nodeIdentifier:0, dataPort:40014, clientPort:55562). Try again in next turn.

解决

　检查 conf/iotdb-cluster.properties 配置文件中 seed_nodes 是否都正常启动

org.apache.maven.wagon.TransferFailedException: Transfer failed

描述

1
$ ./mvnw.sh clean install -P mac
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
org.apache.maven.wagon.TransferFailedException: Transfer failed for https://github.com/jt2594838/mvn-thrift-compiler/raw/master/thrift_0.12.0_0.13.0_mac.exe
    at org.apache.maven.wagon.providers.http.wagon.shared.AbstractHttpClientWagon.fillInputData (AbstractHttpClientWagon.java:1242)
    at org.apache.maven.wagon.providers.http.wagon.shared.AbstractHttpClientWagon.fillInputData (AbstractHttpClientWagon.java:1138)
    at org.apache.maven.wagon.StreamWagon.getInputStream (StreamWagon.java:126)
    at org.apache.maven.wagon.StreamWagon.getIfNewer (StreamWagon.java:88)
    at org.apache.maven.wagon.StreamWagon.get (StreamWagon.java:61)
    at com.googlecode.download.maven.plugin.internal.WGet.doGet (WGet.java:377)
    at com.googlecode.download.maven.plugin.internal.WGet.execute (WGet.java:282)
    at org.apache.maven.plugin.DefaultBuildPluginManager.executeMojo (DefaultBuildPluginManager.java:137)
    at org.apache.maven.lifecycle.internal.MojoExecutor.execute (MojoExecutor.java:210)
    at org.apache.maven.lifecycle.internal.MojoExecutor.execute (MojoExecutor.java:156)
    at org.apache.maven.lifecycle.internal.MojoExecutor.execute (MojoExecutor.java:148)
    at org.apache.maven.lifecycle.internal.LifecycleModuleBuilder.buildProject (LifecycleModuleBuilder.java:117)
    at org.apache.maven.lifecycle.internal.builder.multithreaded.MultiThreadedBuilder$1.call (MultiThreadedBuilder.java:190)
    at org.apache.maven.lifecycle.internal.builder.multithreaded.MultiThreadedBuilder$1.call (MultiThreadedBuilder.java:186)
    at java.util.concurrent.FutureTask.run (FutureTask.java:266)
    at java.util.concurrent.Executors$RunnableAdapter.call (Executors.java:511)
    at java.util.concurrent.FutureTask.run (FutureTask.java:266)
    at java.util.concurrent.ThreadPoolExecutor.runWorker (ThreadPoolExecutor.java:1149)
    at java.util.concurrent.ThreadPoolExecutor$Worker.run (ThreadPoolExecutor.java:624)
    at java.lang.Thread.run (Thread.java:748)
Caused by: javax.net.ssl.SSLHandshakeException: Remote host closed connection during handshake
    at sun.security.ssl.SSLSocketImpl.readRecord (SSLSocketImpl.java:994)
    at sun.security.ssl.SSLSocketImpl.performInitialHandshake (SSLSocketImpl.java:1367)
    at sun.security.ssl.SSLSocketImpl.startHandshake (SSLSocketImpl.java:1395)
    at sun.security.ssl.SSLSocketImpl.startHandshake (SSLSocketImpl.java:1379)
    at org.apache.maven.wagon.providers.http.httpclient.conn.ssl.SSLConnectionSocketFactory.createLayeredSocket (SSLConnectionSocketFactory.java:436)
    at org.apache.maven.wagon.providers.http.httpclient.conn.ssl.SSLConnectionSocketFactory.connectSocket (SSLConnectionSocketFactory.java:384)
    at org.apache.maven.wagon.providers.http.httpclient.impl.conn.DefaultHttpClientConnectionOperator.connect (DefaultHttpClientConnectionOperator.java:142)
    at org.apache.maven.wagon.providers.http.httpclient.impl.conn.PoolingHttpClientConnectionManager.connect (PoolingHttpClientConnectionManager.java:374)
    at org.apache.maven.wagon.providers.http.httpclient.impl.execchain.MainClientExec.establishRoute (MainClientExec.java:393)
    at org.apache.maven.wagon.providers.http.httpclient.impl.execchain.MainClientExec.execute (MainClientExec.java:236)
    at org.apache.maven.wagon.providers.http.httpclient.impl.execchain.ProtocolExec.execute (ProtocolExec.java:186)
    at org.apache.maven.wagon.providers.http.httpclient.impl.execchain.RetryExec.execute (RetryExec.java:89)
    at org.apache.maven.wagon.providers.http.httpclient.impl.execchain.RedirectExec.execute (RedirectExec.java:110)
    at org.apache.maven.wagon.providers.http.httpclient.impl.client.InternalHttpClient.doExecute (InternalHttpClient.java:185)
    at org.apache.maven.wagon.providers.http.httpclient.impl.client.CloseableHttpClient.execute (CloseableHttpClient.java:83)
    at org.apache.maven.wagon.providers.http.wagon.shared.AbstractHttpClientWagon.execute (AbstractHttpClientWagon.java:1002)
    at org.apache.maven.wagon.providers.http.wagon.shared.AbstractHttpClientWagon.fillInputData (AbstractHttpClientWagon.java:1160)
    at org.apache.maven.wagon.providers.http.wagon.shared.AbstractHttpClientWagon.fillInputData (AbstractHttpClientWagon.java:1138)
    at org.apache.maven.wagon.StreamWagon.getInputStream (StreamWagon.java:126)
    at org.apache.maven.wagon.StreamWagon.getIfNewer (StreamWagon.java:88)
    at org.apache.maven.wagon.StreamWagon.get (StreamWagon.java:61)
    at com.googlecode.download.maven.plugin.internal.WGet.doGet (WGet.java:377)
    at com.googlecode.download.maven.plugin.internal.WGet.execute (WGet.java:282)
    at org.apache.maven.plugin.DefaultBuildPluginManager.executeMojo (DefaultBuildPluginManager.java:137)
    at org.apache.maven.lifecycle.internal.MojoExecutor.execute (MojoExecutor.java:210)
    at org.apache.maven.lifecycle.internal.MojoExecutor.execute (MojoExecutor.java:156)
    at org.apache.maven.lifecycle.internal.MojoExecutor.execute (MojoExecutor.java:148)
    at org.apache.maven.lifecycle.internal.LifecycleModuleBuilder.buildProject (LifecycleModuleBuilder.java:117)
    at org.apache.maven.lifecycle.internal.builder.multithreaded.MultiThreadedBuilder$1.call (MultiThreadedBuilder.java:190)
    at org.apache.maven.lifecycle.internal.builder.multithreaded.MultiThreadedBuilder$1.call (MultiThreadedBuilder.java:186)
    at java.util.concurrent.FutureTask.run (FutureTask.java:266)
    at java.util.concurrent.Executors$RunnableAdapter.call (Executors.java:511)
    at java.util.concurrent.FutureTask.run (FutureTask.java:266)
    at java.util.concurrent.ThreadPoolExecutor.runWorker (ThreadPoolExecutor.java:1149)
    at java.util.concurrent.ThreadPoolExecutor$Worker.run (ThreadPoolExecutor.java:624)
    at java.lang.Thread.run (Thread.java:748)
Caused by: java.io.EOFException: SSL peer shut down incorrectly
    at sun.security.ssl.InputRecord.read (InputRecord.java:505)
    at sun.security.ssl.SSLSocketImpl.readRecord (SSLSocketImpl.java:975)
    at sun.security.ssl.SSLSocketImpl.performInitialHandshake (SSLSocketImpl.java:1367)
    at sun.security.ssl.SSLSocketImpl.startHandshake (SSLSocketImpl.java:1395)
    at sun.security.ssl.SSLSocketImpl.startHandshake (SSLSocketImpl.java:1379)
    at org.apache.maven.wagon.providers.http.httpclient.conn.ssl.SSLConnectionSocketFactory.createLayeredSocket (SSLConnectionSocketFactory.java:436)
    at org.apache.maven.wagon.providers.http.httpclient.conn.ssl.SSLConnectionSocketFactory.connectSocket (SSLConnectionSocketFactory.java:384)
    at org.apache.maven.wagon.providers.http.httpclient.impl.conn.DefaultHttpClientConnectionOperator.connect (DefaultHttpClientConnectionOperator.java:142)
    at org.apache.maven.wagon.providers.http.httpclient.impl.conn.PoolingHttpClientConnectionManager.connect (PoolingHttpClientConnectionManager.java:374)
    at org.apache.maven.wagon.providers.http.httpclient.impl.execchain.MainClientExec.establishRoute (MainClientExec.java:393)
    at org.apache.maven.wagon.providers.http.httpclient.impl.execchain.MainClientExec.execute (MainClientExec.java:236)
    at org.apache.maven.wagon.providers.http.httpclient.impl.execchain.ProtocolExec.execute (ProtocolExec.java:186)
    at org.apache.maven.wagon.providers.http.httpclient.impl.execchain.RetryExec.execute (RetryExec.java:89)
    at org.apache.maven.wagon.providers.http.httpclient.impl.execchain.RedirectExec.execute (RedirectExec.java:110)
    at org.apache.maven.wagon.providers.http.httpclient.impl.client.InternalHttpClient.doExecute (InternalHttpClient.java:185)
    at org.apache.maven.wagon.providers.http.httpclient.impl.client.CloseableHttpClient.execute (CloseableHttpClient.java:83)
    at org.apache.maven.wagon.providers.http.wagon.shared.AbstractHttpClientWagon.execute (AbstractHttpClientWagon.java:1002)
    at org.apache.maven.wagon.providers.http.wagon.shared.AbstractHttpClientWagon.fillInputData (AbstractHttpClientWagon.java:1160)
    at org.apache.maven.wagon.providers.http.wagon.shared.AbstractHttpClientWagon.fillInputData (AbstractHttpClientWagon.java:1138)
    at org.apache.maven.wagon.StreamWagon.getInputStream (StreamWagon.java:126)
    at org.apache.maven.wagon.StreamWagon.getIfNewer (StreamWagon.java:88)
    at org.apache.maven.wagon.StreamWagon.get (StreamWagon.java:61)
    at com.googlecode.download.maven.plugin.internal.WGet.doGet (WGet.java:377)
    at com.googlecode.download.maven.plugin.internal.WGet.execute (WGet.java:282)
    at org.apache.maven.plugin.DefaultBuildPluginManager.executeMojo (DefaultBuildPluginManager.java:137)
    at org.apache.maven.lifecycle.internal.MojoExecutor.execute (MojoExecutor.java:210)
    at org.apache.maven.lifecycle.internal.MojoExecutor.execute (MojoExecutor.java:156)
    at org.apache.maven.lifecycle.internal.MojoExecutor.execute (MojoExecutor.java:148)
    at org.apache.maven.lifecycle.internal.LifecycleModuleBuilder.buildProject (LifecycleModuleBuilder.java:117)
    at org.apache.maven.lifecycle.internal.builder.multithreaded.MultiThreadedBuilder$1.call (MultiThreadedBuilder.java:190)
    at org.apache.maven.lifecycle.internal.builder.multithreaded.MultiThreadedBuilder$1.call (MultiThreadedBuilder.java:186)
    at java.util.concurrent.FutureTask.run (FutureTask.java:266)
    at java.util.concurrent.Executors$RunnableAdapter.call (Executors.java:511)
    at java.util.concurrent.FutureTask.run (FutureTask.java:266)
    at java.util.concurrent.ThreadPoolExecutor.runWorker (ThreadPoolExecutor.java:1149)
    at java.util.concurrent.ThreadPoolExecutor$Worker.run (ThreadPoolExecutor.java:624)
    at java.lang.Thread.run (Thread.java:748)

解决

1
2
$ wget https://github.com/jt2594838/mvn-thrift-compiler/raw/master/thrift_0.12.0_0.13.0_mac.exe
$ cp thrift_0.12.0_0.13.0_mac.exe thrift/target/tools/

社区发展

Star 趋势

Apache IoTDB Star History

（图片来源：star-history.t9t.io™ 官网）

个人贡献

　详见：《如何成为 Apache 的 PMC》

欢迎加入我们的技术群，一起交流学习

群名称 群号
人工智能（高级）
人工智能（进阶）
BigData
算法