开源时序数据库 InfluxDB

介绍

InfluxDB™ is a time series database designed to handle high write and query loads. It is an integral component of the TICK stack. InfluxDB is meant to be used as a backing store for any use case involving large amounts of timestamped data, including DevOps monitoring, application metrics, IoT sensor data, and real-time analytics.

基本概念

DataBase

 类似于传统数据库中的 DataBase 概念

Measurement

 和 OLAP 中广义上的度量概念一致,部分 OLAP 数据库中又称为 Metric

Tag

 和 OLAP 中广义上的维度概念一致,部分 OLAP 数据库中又称为 TagKV

Field

 数值

Timestamp

 时间戳

Points

 数据点

Series

 数据点组成的序列

Retention Policy

 数据过期策略,即 TTL

特性

优点

  • 无任何额外的依赖(例如 ZooKeeperHDFS 等)
  • 支持类 SQL 查询(InfluxQL)
  • 支持多租户和简单的鉴权功能
  • 开源社区活跃,且常年霸榜 DB-Engines 时序数据库的第一名

缺点

  • 未开源集群版本
  • 无法跨 Measurement 进行 Join 操作
  • 无法存储相同的数据点,会发生覆盖写
  • 不支持冷热数据的分层存储

环境部署

目前,因为 InfluxDB 2.x 还没有稳定版本发布,所以这里我们使用的 InfluxDB 版本是 1.x

源码版

编译

1
2
3
4
5
$ go get github.com/influxdata/influxdb
$ cd $GOPATH/src/github.com/influxdata/influxdb
$ go clean ./...
$ go install -ldflags="-X main.version=1.7.10" ./...
$ ll $GOPATH/bin

启动

1
$ $GOPATH/bin/influxd

容器版

拉取镜像

1
$ docker pull influxdb
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
Using default tag: latest
latest: Pulling from library/influxdb
latest: Pulling from library/influxdb
1c6172af85ee: Downloading
b194b0e3c928: Download complete
1f5ec00f35d5: Download complete
256a3fda0bc5: Download complete
189579438204: Download complete
855d46376ade: Download complete
5d599164b1bb: Download complete
294856b09ff2: Download complete
latest: Pulling from library/influxdb
1c6172af85ee: Pull complete
b194b0e3c928: Pull complete
1f5ec00f35d5: Pull complete
256a3fda0bc5: Pull complete
189579438204: Pull complete
855d46376ade: Pull complete
5d599164b1bb: Pull complete
294856b09ff2: Pull complete
Digest: sha256:68b3ff6f43ffdb6fe7cf067879e2b28788eff1ebf9b104443c8c2bf3e32c43cf
Status: Downloaded newer image for influxdb:latest
docker.io/library/influxdb:latest

启动

1
$ docker run -idt --name influxdb -p 8086:8086 -v ~/influxdb:/var/lib/influxdb influxdb
1
5cf826d1805451e6bec9426e3c274375f76411118eb43c920aace016911e0ebe

校验

1
$ docker ps
1
2
CONTAINER ID        IMAGE               COMMAND                  CREATED             STATUS              PORTS                    NAMES
5cf826d18054 influxdb "/entrypoint.sh infl…" 50 seconds ago Up 49 seconds 0.0.0.0:8086->8086/tcp influxdb

使用

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
# 进入容器
$ docker exec -it influxdb bash

# 连接 InfluxDB 实例
root@5cf826d18054:/# influx
Connected to http://localhost:8086 version 1.8.0
InfluxDB shell version: 1.8.0

# 创建数据库
> CREATE DATABASE yuzhouwan

# 展示数据库
> SHOW DATABASES
name: databases
name
----
_internal
yuzhouwan

# 创建用户
> CREATE USER asdf2014 WITH PASSWORD 'yuzhouwan.com'

# 展示用户
> SHOW USERS
user admin
---- -----
asdf2014 false

# 赋权
> GRANT ALL PRIVILEGES ON yuzhouwan TO asdf2014

Grafana

1
2
3
4
5
6
7
8
9
10
11
# 安装
$ wget https://dl.grafana.com/oss/release/grafana-6.0.0-beta2.x86_64.rpm
$ sudo yum localinstall grafana-6.0.0-beta2.x86_64.rpm

# 启动
$ systemctl daemon-reload
$ systemctl start grafana-server
$ systemctl status grafana-server

# 自启动
$ sudo systemctl enable grafana-server.service

Telegraf

1
2
3
4
$ wget https://dl.influxdata.com/telegraf/releases/telegraf-1.12.3-1.x86_64.rpm
$ sudo yum localinstall telegraf-1.12.3-1.x86_64.rpm
$ telegraf config > telegraf.conf
$ telegraf --config telegraf.conf

基本操作

版本

1
$ curl -sL -I localhost:8086/ping | grep -i version
1
X-Influxdb-Version: v1.7.10

控制台

1
2
# 输入如下命令,即可进入到 InfluxDB 命令交互的控制台
$ influx -host 'localhost' -port '8086'

建表

1
> show databases;
1
2
3
4
name: databases
name
----
_internal
1
2
# CREATE DATABASE <database_name> [WITH [DURATION <duration>] [REPLICATION <n>] [SHARD DURATION <duration>] [NAME <retention-policy-name>]]
> create database "yuzhouwan";
1
> show databases;
1
2
3
4
5
name: databases
name
----
_internal
yuzhouwan
1
> use yuzhouwan;
1
Using database yuzhouwan

写入

1
> insert blog,protocol=https,name=yuzhouwan value=666

查询

明细查询

1
> select * from blog
1
2
3
4
name: blog
time name protocol value
---- ---- -------- -----
1556438552229094000 yuzhouwan https 666

聚合查询

1
> select mean(value) from blog
1
2
3
4
name: blog
time mean
---- ----
0 666

范围查询

1
> select * from blog WHERE time > '2019-04-01T00:00:00Z' OR time < '2019-05-01T00:00:00Z'
1
2
3
4
name: blog
time name protocol value
---- ---- -------- -----
1556438552229094000 yuzhouwan https 666

数据导出

方法 优缺点
通过 HTTP 接口直接查询 简单易用
influx_tools 命令行工具里的 exporter 功能 并不能导出原始数据点,只能操作 shard
influx_inspect 命令行工具里的 export 功能 支持导出原始数据点,直接操作底层 TSM、WAL 文件

query 接口

1
$ curl -G 'http://localhost:8086/query?pretty=true' --data-urlencode "db=yuzhouwan" --data-urlencode 'q=select * from blog'
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
{
"results": [
{
"series": [
{
"columns": [
"time",
"name",
"protocol",
"value"
],
"name": "blog",
"values": [
[
"2019-04-28T08:02:32.229094Z",
"yuzhouwan",
"https",
666
]
]
}
],
"statement_id": 0
}
]
}

influx_inspect 命令

1
$ influx_inspect export -database yuzhouwan -start 2019-01-01T00:00:00+00:00 -end 2019-12-01T00:00:00+00:00  -out yuzhouwan.out
1
writing out tsm file data for yuzhouwan/autogen...complete.
1
$ cat yuzhouwan.out
1
2
3
4
5
6
7
8
# INFLUXDB EXPORT: 2019-01-01T08:00:00+08:00 - 2019-12-01T08:00:00+08:00
# DDL
CREATE DATABASE yuzhouwan WITH NAME autogen
# DML
# CONTEXT-DATABASE:yuzhouwan
# CONTEXT-RETENTION-POLICY:autogen
# writing tsm data
blog,name=yuzhouwan,protocol=https value=666 1556438552229094000

资料

Doc

Github

欢迎加入我们的技术群,一起交流学习

群名称 群号
人工智能(高级)
人工智能(进阶)
BigData
算法

欢迎关注我的其它发布渠道