Apache IoTDB:一款面向物联网的数据库

Apache IoTDB 是什么?

Apache IoTDB™ (Database for Internet of Things) is an IoT native database with high performance for data management and analysis, deployable on the edge and the cloud.

(图片来源:Apache IoTDB™ 官网)

特性

  • 高吞吐量读写
  • 高效的目录结构
  • 丰富的查询语义
  • 硬件成本低
  • 灵活的部署
  • 与开源生态系统的紧密集成

应用场景

  • 高端制造业
  • 本地控制器服务器
  • 云数据管理

部署

Docker 容器版

1
2
3
4
$ docker pull apache/iotdb
$ docker run -d -p 6667:6667 -p 31999:31999 -p 8181:8181 --name yuzhouwan-iotdb apache/iotdb:latest
$ docker exec -it `docker ps | grep iotdb | awk '{print $1}'` bash
$ start-cli.sh
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
IoTDB> login successfully
IoTDB> help
<your-sql> execute your sql statment
show timeseries show how many timeseries are in iotdb
set time_display_type=xxx eg. long, default, ISO8601, yyyy-MM-dd HH:mm:ss.
show time_display_type show time display type
set time_zone=xxx eg. +08:00, Asia/Shanghai.
show time_zone show cli time zone
set fetch_size=xxx set fetch size when querying data from server.
show fetch_size show fetch size
set max_display_num=xxx eg. set max lines for cli to ouput, -1 equals to unlimited.

IoTDB> insert into root.ln.wf01.wt01(timestamp, status) values(100, true);
Msg: The statement is executed successfully.

IoTDB> show timeseries
+------------------------+-----+-------------+--------+--------+-----------+
| timeseries|alias|storage group|dataType|encoding|compression|
+------------------------+-----+-------------+--------+--------+-----------+
|root.ln.wf01.wt01.status| null| root.ln| BOOLEAN| RLE| SNAPPY|
+------------------------+-----+-------------+--------+--------+-----------+
Total line number = 1
It costs 0.008s

IoTDB> select * from root.ln.wf01.wt01;
+-----------------------------+------------------------+
| Time|root.ln.wf01.wt01.status|
+-----------------------------+------------------------+
|1970-01-01T08:00:00.100+08:00| true|
+-----------------------------+------------------------+
Total line number = 1
It costs 0.005s

IoTDB> exit

集群版

安装

1
2
3
4
$ git fetch upstream cluster_new:cluster_new
$ git checkout cluster_new
$ ./mvnw.sh clean package -T 1C -DskipTests
$ find . -name '*.zip' -type f -exec echo {} \;
1
2
3
4
./cluster/target/cluster-0.12.0-SNAPSHOT.zip
./distribution/target/apache-iotdb-0.12.0-SNAPSHOT-bin.zip
./server/target/iotdb-server-0.12.0-SNAPSHOT.zip
./cli/target/iotdb-cli-0.12.0-SNAPSHOT.zip
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
$ mkdir -p ~/apps/iotdb/cluster/node1
$ mkdir -p ~/apps/iotdb/cluster/node2
$ mkdir -p ~/apps/iotdb/cluster/node3
$ cp ./cluster/target/cluster-0.12.0-SNAPSHOT.zip ~/apps/iotdb/cluster/node1/
$ cp ./cluster/target/cluster-0.12.0-SNAPSHOT.zip ~/apps/iotdb/cluster/node2/
$ cp ./cluster/target/cluster-0.12.0-SNAPSHOT.zip ~/apps/iotdb/cluster/node3/
$ cd ~/apps/iotdb/cluster/node1
$ unzip cluster-0.12.0-SNAPSHOT.zip
$ cd ~/apps/iotdb/cluster/node2
$ unzip cluster-0.12.0-SNAPSHOT.zip
$ cd ~/apps/iotdb/cluster/node3
$ unzip cluster-0.12.0-SNAPSHOT.zip
$ cd ~/apps/iotdb/cluster
$ chmod +x node1/sbin/start-node.sh
$ chmod +x node2/sbin/start-node.sh
$ chmod +x node3/sbin/start-node.sh
$ sed -in-place -e 's/31999/31997/g' node1/conf/iotdb-env.sh
$ sed -in-place -e 's/31999/31998/g' node2/conf/iotdb-env.sh
$ node1/sbin/start-node.sh -internal_meta_port 9003 -internal_data_port 40010 -cluster_rpc_port 55560
$ node2/sbin/start-node.sh -internal_meta_port 9005 -internal_data_port 40012 -cluster_rpc_port 55561
$ node3/sbin/start-node.sh -internal_meta_port 9007 -internal_data_port 40014 -cluster_rpc_port 55562

使用

1
2
3
4
5
$ mkdir ~/apps/iotdb/standalone
$ cp ./cli/target/iotdb-cli-0.12.0-SNAPSHOT.zip ~/apps/iotdb/standalone/
$ cd ~/apps/iotdb/standalone
$ chmod +x sbin/start-cli.sh
$ sbin/start-cli.sh
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
---------------------
Starting IoTDB Cli
---------------------
_____ _________ ______ ______
|_ _| | _ _ ||_ _ `.|_ _ \
| | .--.|_/ | | \_| | | `. \ | |_) |
| | / .'`\ \ | | | | | | | __'.
_| |_| \__. | _| |_ _| |_.' /_| |__) |
|_____|'.__.' |_____| |______.'|_______/ version 0.12.0-SNAPSHOT


IoTDB> login successfully
IoTDB> show timeseries
+----------+-----+-------------+--------+--------+-----------+----+----------+
|timeseries|alias|storage group|dataType|encoding|compression|tags|attributes|
+----------+-----+-------------+--------+--------+-----------+----+----------+
+----------+-----+-------------+--------+--------+-----------+----+----------+
Empty set.
It costs 0.276s

IoTDB> set storage group to root.yuzhouwan
Msg: The statement is executed successfully.

IoTDB> create timeseries root.yuzhouwan.user with datatype=INT64,encoding=PLAIN
Msg: The statement is executed successfully.

IoTDB> insert into root.yuzhouwan(timestamp, user) values(1, 1)
Msg: The statement is executed successfully.

IoTDB> show timeseries
+-------------------+-----+--------------+--------+--------+-----------+----+----------+
| timeseries|alias| storage group|dataType|encoding|compression|tags|attributes|
+-------------------+-----+--------------+--------+--------+-----------+----+----------+
|root.yuzhouwan.user| null|root.yuzhouwan| INT64| PLAIN| SNAPPY|null| null|
+-------------------+-----+--------------+--------+--------+-----------+----+----------+
Total line number = 1
It costs 0.015s

IoTDB> select * from root.yuzhouwan
+-----------------------------+-------------------+
| Time|root.yuzhouwan.user|
+-----------------------------+-------------------+
|1970-01-01T08:00:00.001+08:00| 1|
+-----------------------------+-------------------+
Total line number = 1
It costs 0.005s

IoTDB> select * from root.*
+-----------------------------+-------------------+
| Time|root.yuzhouwan.user|
+-----------------------------+-------------------+
|1970-01-01T08:00:00.001+08:00| 1|
+-----------------------------+-------------------+
Total line number = 1
It costs 0.004s

IoTDB> select * from root
+-----------------------------+-------------------+
| Time|root.yuzhouwan.user|
+-----------------------------+-------------------+
|1970-01-01T08:00:00.001+08:00| 1|
+-----------------------------+-------------------+
Total line number = 1
It costs 0.005s

IoTDB> show child paths root
+--------------+
| child paths|
+--------------+
|root.yuzhouwan|
+--------------+
Total line number = 1
It costs 0.003s

IoTDB> show child paths root.*
+-------------------+
| child paths|
+-------------------+
|root.yuzhouwan.user|
+-------------------+
Total line number = 1
It costs 0.002s

停止

1
$ for p in `jps -ml | grep iotdb | awk '{print $1}'`; do echo $p && kill $p; done

待完善列表

  • 模拟集群故障频繁重启的情况,发现 RaftServer 会出现 OOM 问题,可能存在线程泄露问题
  • 可以考虑使用 Google 设计的有界一致性 Hash 算法,替换原生的一致性算法,以提高集群扩缩容时的平衡性

架构

总体架构图

Apache IoTDB Architecture

(图片来源:iotdb.apache.org™)

写入链路

sequenceDiagram

participant Thrift
participant TSIService
participant TSServiceImpl
participant Planner
participant PlanExecutor
participant StorageEngine
participant StorageGroupProcessor
participant TsFileProcessor
participant AbstractMemTable
participant InsertRowPlan
participant WritableMemChunk
participant MeasurementSchema
participant TVList
participant ExclusiveWriteLogNode
participant TsFileFlushPolicy

opt insert into root.yuzhouwan(timestamp, value) values(1,1)
Thrift ->>+ TSIService : getResult
TSIService ->>+ TSServiceImpl : executeStatement
end

opt deviceId: root.yuzhouwan, time: 1, measurements: [value], values: [1.0]
TSServiceImpl ->>+ Planner : parseSQLToPhysicalPlan
Planner ->>- TSServiceImpl : PhysicalPlan

TSServiceImpl ->>+ TSServiceImpl : executeUpdateStatement
TSServiceImpl ->>+ TSServiceImpl : executeNonQueryPlan
TSServiceImpl ->>+ TSServiceImpl : executeNonQuery

TSServiceImpl ->> PlanExecutor : processNonQuery
PlanExecutor ->> StorageEngine : insert
StorageEngine ->> StorageGroupProcessor : insert
StorageGroupProcessor ->> StorageGroupProcessor : insert

StorageGroupProcessor ->>+ StorageGroupProcessor : writeLock


opt MemTable
StorageGroupProcessor ->>+ StorageEngine : getTimePartition

alt enable_partition is true
StorageEngine ->> StorageEngine : time / timePartitionInterval
else
StorageEngine ->> StorageEngine : 0
end

alt !isSequence && time > lastFlushTime && enable_discard_out_of_order_data is true
StorageGroupProcessor -->> TSServiceImpl : return
end

StorageEngine ->>- StorageGroupProcessor : long timePartitionId

StorageGroupProcessor ->> StorageGroupProcessor : insertToTsFileProcessor
StorageGroupProcessor ->> TsFileProcessor : insert
TsFileProcessor ->> AbstractMemTable : insert
AbstractMemTable ->>+ InsertRowPlan : getIndex
InsertRowPlan ->>- AbstractMemTable : long index
AbstractMemTable ->> AbstractMemTable : updatePlanIndexes
AbstractMemTable ->> AbstractMemTable : write
AbstractMemTable ->>+ AbstractMemTable : createIfNotExistAndGet
AbstractMemTable ->>- AbstractMemTable : IWritableMemChunk
AbstractMemTable ->> WritableMemChunk : write(insertTime, objectValue)

WritableMemChunk ->>+ MeasurementSchema : getType
MeasurementSchema ->>- WritableMemChunk : TSDataType

alt BOOLEAN
WritableMemChunk ->> WritableMemChunk : putBoolean
else INT32
WritableMemChunk ->> WritableMemChunk : putInt
else INT64
WritableMemChunk ->> WritableMemChunk : putLong
else FLOAT
WritableMemChunk ->> WritableMemChunk : putFloat
else DOUBLE
WritableMemChunk ->> WritableMemChunk : putDouble
else TEXT
WritableMemChunk ->> WritableMemChunk : putBinary
end

WritableMemChunk ->> TVList : putFloat
end


alt enable_wal is true
opt WAL
TsFileProcessor ->>+ TsFileProcessor : getLogNode
TsFileProcessor ->>- TsFileProcessor : WriteLogNode
TsFileProcessor ->> ExclusiveWriteLogNode : write

ExclusiveWriteLogNode ->> ExclusiveWriteLogNode : putLog

alt >= flush_wal_threshold
ExclusiveWriteLogNode ->> ExclusiveWriteLogNode : sync
end

end
end


opt Flush
StorageGroupProcessor ->>+ TsFileProcessor : shouldFlush
TsFileProcessor ->>- StorageGroupProcessor : boolean

alt memSize of memtable >= memtable_size_threshold or pointNum of memtable >= avg_series_point_number_threshold
StorageGroupProcessor ->> TsFileFlushPolicy : apply
TsFileFlushPolicy ->>+ TsFileProcessor : shouldClose
TsFileProcessor ->>- TsFileFlushPolicy : boolean
alt shouldClose is true
TsFileFlushPolicy ->> StorageGroupProcessor : asyncCloseOneTsFileProcessor
else
TsFileFlushPolicy ->> TsFileProcessor : asyncFlush
end
end

end



StorageGroupProcessor ->>- StorageGroupProcessor : writeUnlock

StorageGroupProcessor -->> TSServiceImpl : may throw exception
end

TSServiceImpl ->>- TSServiceImpl : boolean succ
TSServiceImpl ->>- TSServiceImpl : TSStatus
TSServiceImpl ->>- TSServiceImpl : TSExecuteStatementResp
TSServiceImpl ->>- TSIService : TSExecuteStatementResp
TSIService ->>- Thrift : executeStatement_result
点击这里查看完整大图
写入数据时,时间粒度支持 ms / us / ns 三种,默认为 ms,且存储时间粒度统一为 ms
参数 enable_partition 控制是否 partition 分区,默认为 false
参数 enable_discard_out_of_order_data 控制是否丢弃乱序数据,默认为 false
参数 enable_wal 控制是否开启写 WAL 日志,默认为 true
参数 flush_wal_threshold 控制 MemTable 中缓存了多少条记录后,才会同步到 WAL 日志文件中,默认为 10000
参数 memtable_size_threshold 控制 MemTable 中缓存占用多大内存后,才会 Flush 到 TSFile 数据文件中,默认为 1G
参数 avg_series_point_number_threshold 控制 MemTable 中缓存了多少条记录后,才会 Flush 到 TSFile 数据文件中,默认为 100000

查询链路

sequenceDiagram

participant Thrift
participant TSIService
participant TSServiceImpl
participant Planner
participant PhysicalPlan
participant QueryPlan
participant RawDataQueryPlan
participant RawQueryDataSetWithoutValueFilter
participant TSQueryDataSet
participant TSExecuteStatementResp

opt select * from root.*
Thrift ->>+ TSIService : getResult
TSIService ->>+ TSServiceImpl : requestStatementId
TSServiceImpl ->>- TSIService : long statementId

TSIService ->>+ TSServiceImpl : executeStatement


TSServiceImpl ->>+ Planner : parseSQLToPhysicalPlan
Planner ->>- TSServiceImpl : PhysicalPlan
TSServiceImpl ->>+ PhysicalPlan : isQuery
PhysicalPlan ->>- TSServiceImpl : boolean
alt isQuery
TSServiceImpl ->>+ TSServiceImpl : internalExecuteQueryStatement
TSServiceImpl ->>+ TSServiceImpl : getQueryResp
TSServiceImpl ->>+ TSServiceImpl : getQueryColumnHeaders

TSServiceImpl ->>+ TSServiceImpl : getWideQueryHeaders

opt root.yuzhouwan.value
TSServiceImpl ->>+ QueryPlan : getPaths
QueryPlan ->>- TSServiceImpl : list of PartialPath
end

TSServiceImpl ->>+ TSServiceImpl : getSeriesTypeByPath
TSServiceImpl ->>- TSServiceImpl : TSDataType

TSServiceImpl -->>- TSServiceImpl : list of respColumns and list of columnTypes
end


alt plan instanceof  is true
TSServiceImpl ->>+ RawDataQueryPlan : getDeduplicatedPaths().size()
RawDataQueryPlan ->>- TSServiceImpl : long deduplicatedPathNum
end

TSServiceImpl ->>+ TSServiceImpl : generateQueryId
TSServiceImpl ->>- TSServiceImpl : long queryId

TSServiceImpl ->>+ TSServiceImpl : createQueryDataSet
TSServiceImpl ->>- TSServiceImpl : QueryDataSet

TSServiceImpl ->>+ TSServiceImpl : fillRpcReturnData
TSServiceImpl ->>+ RawQueryDataSetWithoutValueFilter : fillBuffer
RawQueryDataSetWithoutValueFilter ->> TSQueryDataSet : setTime
RawQueryDataSetWithoutValueFilter ->> TSQueryDataSet : setValueList
RawQueryDataSetWithoutValueFilter ->> TSQueryDataSet : setBitmapList
RawQueryDataSetWithoutValueFilter ->>- TSServiceImpl : TSQueryDataSet

TSServiceImpl ->> TSExecuteStatementResp : setQueryDataSet
TSServiceImpl ->> TSExecuteStatementResp : setQueryId

TSServiceImpl ->>- TSServiceImpl : TSQueryDataSet

TSServiceImpl ->>- TSServiceImpl : TSExecuteStatementResp
TSServiceImpl ->>- TSServiceImpl : TSExecuteStatementResp
TSServiceImpl ->>- TSServiceImpl : TSExecuteStatementResp
TSServiceImpl ->>- TSIService : TSExecuteStatementResp
end

TSIService ->>- Thrift : requestStatementId_result
点击这里查看完整大图

目录结构

1
$ tree
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
.
├── data
│   ├── sequence
│   │   └── root.ln
│   │   └── 0
│   │   └── 1606030145164-1-0.tsfile
│   └── unsequence
├── system
│   ├── roles
│   ├── schema
│   │   ├── mlog.txt
│   │   ├── system.properties
│   │   └── tlog.txt
│   ├── storage_groups
│   │   └── root.ln
│   │   ├── 0
│   │   │   └── Version-100
│   │   └── upgrade
│   │   └── Version-100
│   ├── upgrade
│   │   └── upgrade.txt
│   └── users
│   └── root.profile
└── wal
└── root.ln-1606030145164-1-0.tsfile
└── wal1

16 directories, 9 files
1
$ cat system/schema/mlog.txt
1
2
2,root.ln
0,root.ln.wf01.wt01.status,0,2,1,,,
1
$ cat system/schema/system.properties
1
2
3
4
5
6
7
8
9
#System properties:
#Sun Nov 22 15:28:58 CST 2020
partition_interval=9223372036854775807
timestamp_precision=ms
tsfile_storage_fs=LOCAL
enable_partition=false
max_degree_of_index_node=1024
tag_attribute_total_size=700
iotdb_version=UNKNOWN
1
$ cat system/users/root.profile
1
rootc���{��Pykd��HE%
1
$ cat wal/root.ln-1606030145164-1-0.tsfile/wal1
1
8droot.ln.wf01.wt01status�O�f%

压缩策略

  • LevelCompactionTsFileManagement
  • NoCompactionTsFileManagement

性能压测

1
2
3
4
5
6
7
$ git clone --depth 1 --single-branch --branch master https://github.com/thulab/iotdb-benchmark
$ cd iotdb-benchmark
$ mvn clean package -Dmaven.test.skip=true
$ mkdir ../lib
$ find . -name '*jar' -type f -exec echo {} \; -exec cp {} ../lib/ \;
$ mv ../lib .
$ ./benchmark.sh

踩过的坑

Cannot connect to node Node

描述

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
---------------------
Starting IoTDB (Cluster Mode)
---------------------
Maximum memory allocation pool = 4096MB, initial memory allocation pool = 800MB
If you want to change this configuration, please check conf/iotdb-env.sh(Unix or OS X, if you use Windows, check conf/iotdb-env.bat).
2020-12-01 17:12:58,126 [main] INFO org.apache.iotdb.cluster.config.ClusterDescriptor:191 - Start to read config file sbin/../conf/iotdb-cluster.properties
2020-12-01 17:12:58,159 [main] INFO org.apache.iotdb.db.conf.IoTDBDescriptor:148 - Start to read config file file:sbin/../conf/iotdb-engine.properties
2020-12-01 17:12:58,161 [main] INFO org.apache.iotdb.db.conf.IoTDBDescriptor:803 - allocateMemoryForRead = 347078656
2020-12-01 17:12:58,161 [main] INFO org.apache.iotdb.db.conf.IoTDBDescriptor:804 - allocateMemoryForWrite = 2429550592
2020-12-01 17:12:58,161 [main] INFO org.apache.iotdb.db.conf.IoTDBDescriptor:805 - allocateMemoryForSchema = 347078656
2020-12-01 17:12:58,162 [main] INFO org.apache.iotdb.db.conf.IoTDBDescriptor:247 - IoTDB enable memory control: true
2020-12-01 17:12:58,166 [main] INFO org.apache.iotdb.tsfile.common.conf.TSFileDescriptor:108 - Start to read config file sbin/../conf/iotdb-engine.properties
2020-12-01 17:12:58,168 [main] INFO org.apache.iotdb.cluster.ClusterMain:90 - Running mode -s
2020-12-01 17:12:58,250 [main] INFO org.apache.iotdb.db.engine.version.SimpleFileVersionController:115 - Version file updated, previous: /Users/benedictjin/apps/iotdb/cluster/sbin/../data/system/raftLog/0/upgrade/Version-100, current: /Users/benedictjin/apps/iotdb/cluster/sbin/../data/system/raftLog/0/upgrade/Version-200
2020-12-01 17:12:58,252 [main] INFO org.apache.iotdb.cluster.log.manage.serializable.SyncLogDequeSerializer:786 - Recovered log meta: LogManagerMeta{ commitLogTerm=-1, commitLogIndex=-1, lastLogIndex=-1, lastLogTerm=-1, maxHaveAppliedCommitIndex=-1}, availableVersion: [0,9223372036854775807], state: HardState{currentTerm=0, voteFor=null}
2020-12-01 17:12:58,253 [main] INFO org.apache.iotdb.cluster.log.manage.serializable.SyncLogDequeSerializer:482 - Find log type =idx log files [sbin/../data/system/raftLog/0/0-9223372036854775807-101-idx]
2020-12-01 17:12:58,260 [main] INFO org.apache.iotdb.cluster.log.manage.serializable.SyncLogDequeSerializer:482 - Find log type =data log files [sbin/../data/system/raftLog/0/0-9223372036854775807-101-data]
2020-12-01 17:12:58,261 [main] INFO org.apache.iotdb.cluster.log.manage.serializable.SyncLogDequeSerializer:546 - no log index file to recover
2020-12-01 17:12:58,269 [main] INFO org.apache.iotdb.cluster.server.member.MetaGroupMember:1267 - Recovered node identifier 356238364
2020-12-01 17:12:58,270 [main] INFO org.apache.iotdb.cluster.server.member.MetaGroupMember:1294 - The identifier of this node has been set to 356238364
2020-12-01 17:12:58,307 [main] INFO org.apache.iotdb.db.auth.authorizer.BasicAuthorizer$InstanceHolder:87 - Authorizer provider class: org.apache.iotdb.db.auth.authorizer.LocalFileAuthorizer
2020-12-01 17:12:58,313 [main] INFO org.apache.iotdb.db.auth.user.BasicUserManager:79 - Admin initialized
2020-12-01 17:12:58,316 [main] INFO org.apache.iotdb.db.auth.user.BasicUserManager:79 - Admin initialized
2020-12-01 17:12:58,316 [main] INFO org.apache.iotdb.db.auth.authorizer.BasicAuthorizer:67 - Initialization of Authorizer completes
2020-12-01 17:12:58,329 [main] INFO org.apache.iotdb.cluster.server.member.MetaGroupMember:1189 - No partition table file found
2020-12-01 17:12:58,334 [main] INFO org.apache.iotdb.cluster.server.RaftServer:220 - Cluster node Node(ip:127.0.0.1, metaPort:9003, nodeIdentifier:356238364, dataPort:40010, clientPort:55560) begins to set up
2020-12-01 17:12:58,348 [main] INFO org.apache.iotdb.cluster.server.RaftServer:231 - Cluster node Node(ip:127.0.0.1, metaPort:9003, nodeIdentifier:356238364, dataPort:40010, clientPort:55560) is up
2020-12-01 17:12:58,349 [main] INFO org.apache.iotdb.cluster.server.heartbeat.HeartbeatServer:201 - Cluster node's heartbeat Node(ip:127.0.0.1, metaPort:9003, nodeIdentifier:356238364, dataPort:40010, clientPort:55560) begins to set up
2020-12-01 17:12:58,350 [main] INFO org.apache.iotdb.cluster.server.heartbeat.HeartbeatServer:213 - Cluster node's heartbeat Node(ip:127.0.0.1, metaPort:9003, nodeIdentifier:356238364, dataPort:40010, clientPort:55560) is up
2020-12-01 17:12:58,361 [main] WARN org.apache.iotdb.db.service.StartupChecks:35 - iotdb.jmx.port missing from iotdb-env.sh(Unix or OS X, if you use Windows, check conf/iotdb-env.bat)
2020-12-01 17:12:58,361 [main] INFO org.apache.iotdb.db.service.StartupChecks:50 - JDK veriosn is 8.
2020-12-01 17:12:58,361 [main] INFO org.apache.iotdb.db.service.IoTDB:95 - Setting up IoTDB...
2020-12-01 17:12:58,362 [main] INFO org.apache.iotdb.db.service.IoTDB:99 - recover the schema...
2020-12-01 17:12:58,402 [main] INFO org.apache.iotdb.db.service.IoTDB:151 - spend 40ms to recover schema.
2020-12-01 17:12:58,402 [main] INFO org.apache.iotdb.db.service.IoTDB:152 - After initializing, tsFile threshold is 0, memtableSize is 134217728
2020-12-01 17:12:58,407 [main] INFO org.apache.iotdb.db.concurrent.IoTDBThreadPoolFactory:46 - new fixed thread pool: Flush-ServerServiceImpl, thread number: 8
2020-12-01 17:12:58,409 [main] INFO org.apache.iotdb.db.concurrent.IoTDBThreadPoolFactory:80 - new cached thread pool: Flush-SubTask-ServerServiceImpl
2020-12-01 17:12:58,409 [main] INFO org.apache.iotdb.db.engine.flush.pool.FlushSubTaskPoolManager:57 - Flush sub task manager started.
2020-12-01 17:12:58,409 [main] INFO org.apache.iotdb.db.engine.flush.pool.FlushTaskPoolManager:58 - Flush task manager started.
2020-12-01 17:12:58,414 [main] INFO org.apache.iotdb.db.cost.statistic.Measurement:142 - start measurement stats module...
2020-12-01 17:12:58,415 [main] INFO org.apache.iotdb.db.cost.statistic.Measurement:250 - start the consuming task in the measurement stats module...
2020-12-01 17:12:58,422 [main] INFO org.apache.iotdb.db.concurrent.IoTDBThreadPoolFactory:46 - new fixed thread pool: Recovery-Thread-Pool, thread number: 8
2020-12-01 17:12:58,427 [main] INFO org.apache.iotdb.db.concurrent.IoTDBThreadPoolFactory:63 - new single thread pool: Begin-Recovery-Pool
2020-12-01 17:12:58,430 [main] INFO org.apache.iotdb.db.service.thrift.ThriftService:99 - IoTDB: start RPC ServerService...
2020-12-01 17:12:58,441 [RPC-ServerServiceImpl] INFO org.apache.iotdb.db.service.thrift.ThriftServiceThread:125 - The RPC ServerService service thread begin to run...
2020-12-01 17:12:58,544 [main] INFO org.apache.iotdb.db.service.thrift.ThriftService:117 - IoTDB: start RPC ServerService successfully, listening on ip 0.0.0.0 port 6667
2020-12-01 17:12:58,544 [main] INFO org.apache.iotdb.db.service.IoTDB:118 - IoTDB is set up, now may some sgs are not ready, please wait several seconds...
2020-12-01 17:12:58,554 [main] INFO org.apache.iotdb.db.service.UpgradeSevice:112 - finish counting upgrading files, total num:0
2020-12-01 17:12:58,554 [main] INFO org.apache.iotdb.db.service.UpgradeSevice:73 - Waiting for upgrade task pool to shut down
2020-12-01 17:12:58,554 [main] INFO org.apache.iotdb.db.service.UpgradeSevice:78 - Upgrade service stopped
2020-12-01 17:12:58,565 [main] INFO org.apache.iotdb.db.engine.merge.manage.MergeManager:154 - MergeManager started
2020-12-01 17:12:58,566 [main] INFO org.apache.iotdb.db.engine.compaction.CompactionMergeTaskPoolManager:57 - Compaction task manager started.
2020-12-01 17:12:58,566 [main] INFO org.apache.iotdb.db.service.IoTDB:137 - Congratulation, IoTDB is set up successfully. Now, enjoy yourself!
2020-12-01 17:12:58,566 [main] INFO org.apache.iotdb.db.service.IoTDB:91 - IoTDB has started.
2020-12-01 17:12:58,569 [main] INFO org.apache.iotdb.cluster.server.member.RaftMember:282 - Meta started
2020-12-01 17:13:03,578 [pool-12-thread-1] WARN org.apache.iotdb.cluster.server.member.MetaGroupMember:1005 - Start up exception. Cannot connect to node Node(ip:127.0.0.1, metaPort:9005, nodeIdentifier:0, dataPort:40012, clientPort:55561). Try again in next turn.
2020-12-01 17:13:03,578 [pool-12-thread-2] WARN org.apache.iotdb.cluster.server.member.MetaGroupMember:1005 - Start up exception. Cannot connect to node Node(ip:127.0.0.1, metaPort:9007, nodeIdentifier:0, dataPort:40014, clientPort:55562). Try again in next turn.

解决

 检查 conf/iotdb-cluster.properties 配置文件中 seed_nodes 是否都正常启动

org.apache.maven.wagon.TransferFailedException: Transfer failed

描述

1
$ ./mvnw.sh clean install -P mac
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
org.apache.maven.wagon.TransferFailedException: Transfer failed for https://github.com/jt2594838/mvn-thrift-compiler/raw/master/thrift_0.12.0_0.13.0_mac.exe
at org.apache.maven.wagon.providers.http.wagon.shared.AbstractHttpClientWagon.fillInputData (AbstractHttpClientWagon.java:1242)
at org.apache.maven.wagon.providers.http.wagon.shared.AbstractHttpClientWagon.fillInputData (AbstractHttpClientWagon.java:1138)
at org.apache.maven.wagon.StreamWagon.getInputStream (StreamWagon.java:126)
at org.apache.maven.wagon.StreamWagon.getIfNewer (StreamWagon.java:88)
at org.apache.maven.wagon.StreamWagon.get (StreamWagon.java:61)
at com.googlecode.download.maven.plugin.internal.WGet.doGet (WGet.java:377)
at com.googlecode.download.maven.plugin.internal.WGet.execute (WGet.java:282)
at org.apache.maven.plugin.DefaultBuildPluginManager.executeMojo (DefaultBuildPluginManager.java:137)
at org.apache.maven.lifecycle.internal.MojoExecutor.execute (MojoExecutor.java:210)
at org.apache.maven.lifecycle.internal.MojoExecutor.execute (MojoExecutor.java:156)
at org.apache.maven.lifecycle.internal.MojoExecutor.execute (MojoExecutor.java:148)
at org.apache.maven.lifecycle.internal.LifecycleModuleBuilder.buildProject (LifecycleModuleBuilder.java:117)
at org.apache.maven.lifecycle.internal.builder.multithreaded.MultiThreadedBuilder$1.call (MultiThreadedBuilder.java:190)
at org.apache.maven.lifecycle.internal.builder.multithreaded.MultiThreadedBuilder$1.call (MultiThreadedBuilder.java:186)
at java.util.concurrent.FutureTask.run (FutureTask.java:266)
at java.util.concurrent.Executors$RunnableAdapter.call (Executors.java:511)
at java.util.concurrent.FutureTask.run (FutureTask.java:266)
at java.util.concurrent.ThreadPoolExecutor.runWorker (ThreadPoolExecutor.java:1149)
at java.util.concurrent.ThreadPoolExecutor$Worker.run (ThreadPoolExecutor.java:624)
at java.lang.Thread.run (Thread.java:748)
Caused by: javax.net.ssl.SSLHandshakeException: Remote host closed connection during handshake
at sun.security.ssl.SSLSocketImpl.readRecord (SSLSocketImpl.java:994)
at sun.security.ssl.SSLSocketImpl.performInitialHandshake (SSLSocketImpl.java:1367)
at sun.security.ssl.SSLSocketImpl.startHandshake (SSLSocketImpl.java:1395)
at sun.security.ssl.SSLSocketImpl.startHandshake (SSLSocketImpl.java:1379)
at org.apache.maven.wagon.providers.http.httpclient.conn.ssl.SSLConnectionSocketFactory.createLayeredSocket (SSLConnectionSocketFactory.java:436)
at org.apache.maven.wagon.providers.http.httpclient.conn.ssl.SSLConnectionSocketFactory.connectSocket (SSLConnectionSocketFactory.java:384)
at org.apache.maven.wagon.providers.http.httpclient.impl.conn.DefaultHttpClientConnectionOperator.connect (DefaultHttpClientConnectionOperator.java:142)
at org.apache.maven.wagon.providers.http.httpclient.impl.conn.PoolingHttpClientConnectionManager.connect (PoolingHttpClientConnectionManager.java:374)
at org.apache.maven.wagon.providers.http.httpclient.impl.execchain.MainClientExec.establishRoute (MainClientExec.java:393)
at org.apache.maven.wagon.providers.http.httpclient.impl.execchain.MainClientExec.execute (MainClientExec.java:236)
at org.apache.maven.wagon.providers.http.httpclient.impl.execchain.ProtocolExec.execute (ProtocolExec.java:186)
at org.apache.maven.wagon.providers.http.httpclient.impl.execchain.RetryExec.execute (RetryExec.java:89)
at org.apache.maven.wagon.providers.http.httpclient.impl.execchain.RedirectExec.execute (RedirectExec.java:110)
at org.apache.maven.wagon.providers.http.httpclient.impl.client.InternalHttpClient.doExecute (InternalHttpClient.java:185)
at org.apache.maven.wagon.providers.http.httpclient.impl.client.CloseableHttpClient.execute (CloseableHttpClient.java:83)
at org.apache.maven.wagon.providers.http.wagon.shared.AbstractHttpClientWagon.execute (AbstractHttpClientWagon.java:1002)
at org.apache.maven.wagon.providers.http.wagon.shared.AbstractHttpClientWagon.fillInputData (AbstractHttpClientWagon.java:1160)
at org.apache.maven.wagon.providers.http.wagon.shared.AbstractHttpClientWagon.fillInputData (AbstractHttpClientWagon.java:1138)
at org.apache.maven.wagon.StreamWagon.getInputStream (StreamWagon.java:126)
at org.apache.maven.wagon.StreamWagon.getIfNewer (StreamWagon.java:88)
at org.apache.maven.wagon.StreamWagon.get (StreamWagon.java:61)
at com.googlecode.download.maven.plugin.internal.WGet.doGet (WGet.java:377)
at com.googlecode.download.maven.plugin.internal.WGet.execute (WGet.java:282)
at org.apache.maven.plugin.DefaultBuildPluginManager.executeMojo (DefaultBuildPluginManager.java:137)
at org.apache.maven.lifecycle.internal.MojoExecutor.execute (MojoExecutor.java:210)
at org.apache.maven.lifecycle.internal.MojoExecutor.execute (MojoExecutor.java:156)
at org.apache.maven.lifecycle.internal.MojoExecutor.execute (MojoExecutor.java:148)
at org.apache.maven.lifecycle.internal.LifecycleModuleBuilder.buildProject (LifecycleModuleBuilder.java:117)
at org.apache.maven.lifecycle.internal.builder.multithreaded.MultiThreadedBuilder$1.call (MultiThreadedBuilder.java:190)
at org.apache.maven.lifecycle.internal.builder.multithreaded.MultiThreadedBuilder$1.call (MultiThreadedBuilder.java:186)
at java.util.concurrent.FutureTask.run (FutureTask.java:266)
at java.util.concurrent.Executors$RunnableAdapter.call (Executors.java:511)
at java.util.concurrent.FutureTask.run (FutureTask.java:266)
at java.util.concurrent.ThreadPoolExecutor.runWorker (ThreadPoolExecutor.java:1149)
at java.util.concurrent.ThreadPoolExecutor$Worker.run (ThreadPoolExecutor.java:624)
at java.lang.Thread.run (Thread.java:748)
Caused by: java.io.EOFException: SSL peer shut down incorrectly
at sun.security.ssl.InputRecord.read (InputRecord.java:505)
at sun.security.ssl.SSLSocketImpl.readRecord (SSLSocketImpl.java:975)
at sun.security.ssl.SSLSocketImpl.performInitialHandshake (SSLSocketImpl.java:1367)
at sun.security.ssl.SSLSocketImpl.startHandshake (SSLSocketImpl.java:1395)
at sun.security.ssl.SSLSocketImpl.startHandshake (SSLSocketImpl.java:1379)
at org.apache.maven.wagon.providers.http.httpclient.conn.ssl.SSLConnectionSocketFactory.createLayeredSocket (SSLConnectionSocketFactory.java:436)
at org.apache.maven.wagon.providers.http.httpclient.conn.ssl.SSLConnectionSocketFactory.connectSocket (SSLConnectionSocketFactory.java:384)
at org.apache.maven.wagon.providers.http.httpclient.impl.conn.DefaultHttpClientConnectionOperator.connect (DefaultHttpClientConnectionOperator.java:142)
at org.apache.maven.wagon.providers.http.httpclient.impl.conn.PoolingHttpClientConnectionManager.connect (PoolingHttpClientConnectionManager.java:374)
at org.apache.maven.wagon.providers.http.httpclient.impl.execchain.MainClientExec.establishRoute (MainClientExec.java:393)
at org.apache.maven.wagon.providers.http.httpclient.impl.execchain.MainClientExec.execute (MainClientExec.java:236)
at org.apache.maven.wagon.providers.http.httpclient.impl.execchain.ProtocolExec.execute (ProtocolExec.java:186)
at org.apache.maven.wagon.providers.http.httpclient.impl.execchain.RetryExec.execute (RetryExec.java:89)
at org.apache.maven.wagon.providers.http.httpclient.impl.execchain.RedirectExec.execute (RedirectExec.java:110)
at org.apache.maven.wagon.providers.http.httpclient.impl.client.InternalHttpClient.doExecute (InternalHttpClient.java:185)
at org.apache.maven.wagon.providers.http.httpclient.impl.client.CloseableHttpClient.execute (CloseableHttpClient.java:83)
at org.apache.maven.wagon.providers.http.wagon.shared.AbstractHttpClientWagon.execute (AbstractHttpClientWagon.java:1002)
at org.apache.maven.wagon.providers.http.wagon.shared.AbstractHttpClientWagon.fillInputData (AbstractHttpClientWagon.java:1160)
at org.apache.maven.wagon.providers.http.wagon.shared.AbstractHttpClientWagon.fillInputData (AbstractHttpClientWagon.java:1138)
at org.apache.maven.wagon.StreamWagon.getInputStream (StreamWagon.java:126)
at org.apache.maven.wagon.StreamWagon.getIfNewer (StreamWagon.java:88)
at org.apache.maven.wagon.StreamWagon.get (StreamWagon.java:61)
at com.googlecode.download.maven.plugin.internal.WGet.doGet (WGet.java:377)
at com.googlecode.download.maven.plugin.internal.WGet.execute (WGet.java:282)
at org.apache.maven.plugin.DefaultBuildPluginManager.executeMojo (DefaultBuildPluginManager.java:137)
at org.apache.maven.lifecycle.internal.MojoExecutor.execute (MojoExecutor.java:210)
at org.apache.maven.lifecycle.internal.MojoExecutor.execute (MojoExecutor.java:156)
at org.apache.maven.lifecycle.internal.MojoExecutor.execute (MojoExecutor.java:148)
at org.apache.maven.lifecycle.internal.LifecycleModuleBuilder.buildProject (LifecycleModuleBuilder.java:117)
at org.apache.maven.lifecycle.internal.builder.multithreaded.MultiThreadedBuilder$1.call (MultiThreadedBuilder.java:190)
at org.apache.maven.lifecycle.internal.builder.multithreaded.MultiThreadedBuilder$1.call (MultiThreadedBuilder.java:186)
at java.util.concurrent.FutureTask.run (FutureTask.java:266)
at java.util.concurrent.Executors$RunnableAdapter.call (Executors.java:511)
at java.util.concurrent.FutureTask.run (FutureTask.java:266)
at java.util.concurrent.ThreadPoolExecutor.runWorker (ThreadPoolExecutor.java:1149)
at java.util.concurrent.ThreadPoolExecutor$Worker.run (ThreadPoolExecutor.java:624)
at java.lang.Thread.run (Thread.java:748)

解决

1
2
$ wget https://github.com/jt2594838/mvn-thrift-compiler/raw/master/thrift_0.12.0_0.13.0_mac.exe
$ cp thrift_0.12.0_0.13.0_mac.exe thrift/target/tools/

社区发展

Star 趋势

Apache IoTDB Star History

(图片来源:star-history.t9t.io™ 官网)

个人贡献

 详见:《如何成为 Apache 的 PMC

欢迎加入我们的技术群,一起交流学习

群名称 群号
人工智能(高级)
人工智能(进阶)
BigData
算法

欢迎关注我的其它发布渠道