8. kafka集群部署 - 阿俊博客

8. kafka集群部署#

8.1 准备工作#

名称	IP地址	Hostname	组织结构
zk1	192.168.247.101	zookeeper1
zk2	192.168.247.102	zookeeper2
zk3	192.168.247.103	zookeeper3
kafka1	192.168.247.201	kafka1
kafka2	192.168.247.202	kafka2
kafka3	192.168.247.203	kafka3
kafka4	192.168.247.204	kafka4
orderer0	192.168.247.91	orderer0.test.com
orderer1	192.168.247.92	orderer1.test.com
orderer2	192.168.247.93	orderer2.test.com
peer0	192.168.247.81	peer0.orggo.test.com	OrgGo
peer0	192.168.247.82	peer0.orgcpp.test.com	OrgCpp

为了保证整个集群的正常工作, 需要给集群中的各个节点设置工作目录, 我们要保证各个节点工作目录是相同的

1
# 在以上各个节点的家目录创建工作目录:
2
$ mkdir ~/kafka

8.2. 生成证书文件#

8.2.1 编写配置文件#

1
OrdererOrgs:
2
  - Name: Orderer
3
    Domain: test.com
4
    Specs:
5
      - Hostname: orderer0  # 第1个排序节点: orderer0.test.com
6
      - Hostname: orderer1  # 第2个排序节点: orderer1.test.com
7
      - Hostname: orderer2  # 第3个排序节点: orderer2.test.com
8

9
PeerOrgs:
10
  - Name: OrgGo
11
    Domain: orggo.test.com
12
    Template:
13
      Count: 2  # 当前go组织两个peer节点
14
    Users:
15
      Count: 1
16

17
  - Name: OrgCpp
18
    Domain: orgcpp.test.com
19
    Template:
20
      Count: 2  # 当前cpp组织两个peer节点
21
    Users:
22
      Count: 1

8.2.2 生成证书#

1
$ cryptogen generate --config=crypto-config.yaml
2
$ tree ./ -L 1
3
./
4
├── `crypto-config`   -> 证书文件目录
5
└── crypto-config.yaml

8.3. 生成创始块和通道文件#

8.3.1 编写配置文件#

配置文件名configtx.yaml这个名字是固定的, 不可修改的

1
---
2
################################################################################
3
#
4
#   Section: Organizations
5
#
6
#   - This section defines the different organizational identities which will
7
#   be referenced later in the configuration.
8
#
9
################################################################################
10
Organizations:
11
    - &OrdererOrg
12
        Name: OrdererOrg
13
        ID: OrdererMSP
14
        MSPDir: crypto-config/ordererOrganizations/test.com/msp
15

16
    - &go_org
17
        Name: OrgGoMSP
18
        ID: OrgGoMSP
19
        MSPDir: crypto-config/peerOrganizations/orggo.test.com/msp
20
        AnchorPeers:
21
            - Host: peer0.orggo.test.com
22
              Port: 7051
23

24
    - &cpp_org
25
        Name: OrgCppMSP
26
        ID: OrgCppMSP
27
        MSPDir: crypto-config/peerOrganizations/orgcpp.test.com/msp
28
        AnchorPeers:
29
            - Host: peer0.orgcpp.test.com
30
              Port: 7051
31

32
################################################################################
33
#
34
#   SECTION: Capabilities
35
#
36
################################################################################
37
Capabilities:
38
    Global: &ChannelCapabilities
39
        V1_1: true
40
    Orderer: &OrdererCapabilities
41
        V1_1: true
42
    Application: &ApplicationCapabilities
43
        V1_2: true
44

45
################################################################################
46
#
47
#   SECTION: Application
48
#
49
################################################################################
50
Application: &ApplicationDefaults
51
    Organizations:
52

53
################################################################################
54
#
55
#   SECTION: Orderer
56
#
57
################################################################################
58
Orderer: &OrdererDefaults
59
    # Available types are "solo" and "kafka"
60
    OrdererType: kafka
61
    Addresses:
62
        # 排序节点服务器地址
63
        - orderer0.test.com:7050
64
        - orderer1.test.com:7050
65
        - orderer2.test.com:7050
66

67
    BatchTimeout: 2s
68
    BatchSize:
69
        MaxMessageCount: 10
70
        AbsoluteMaxBytes: 99 MB
71
        PreferredMaxBytes: 512 KB
72
    Kafka:
73
        Brokers:
74
            # kafka服务器地址
75
            - 192.168.247.201:9092
76
            - 192.168.247.202:9092
77
            - 192.168.247.203:9092
78
            - 192.168.247.204:9092
79
    Organizations:
80

81
################################################################################
82
#
83
#   Profile
84
#
85
################################################################################
86
Profiles:
87
    OrgsOrdererGenesis:
88
        Capabilities:
89
            <<: *ChannelCapabilities
90
        Orderer:
91
            <<: *OrdererDefaults
92
            Organizations:
93
                - *OrdererOrg
94
            Capabilities:
95
                <<: *OrdererCapabilities
96
        Consortiums:
97
            SampleConsortium:
98
                Organizations:
99
                    - *go_org
100
                    - *cpp_org
101
    OrgsChannel:
102
        Consortium: SampleConsortium
103
        Application:
104
            <<: *ApplicationDefaults
105
            Organizations:
106
                - *go_org
107
                - *cpp_org
108
            Capabilities:
109
                <<: *ApplicationCapabilities

8.3.2 生成通道和创始块文件#

生成创始块文件

1
# 我们先创建一个目录 channel-artifacts 存储生成的文件, 目的是为了和后边的配置文件模板的配置项保持一致
2
$ mkdir channel-artifacts
3
# 生成通道文件
4
$ configtxgen -profile OrgsOrdererGenesis -outputBlock ./channel-artifacts/genesis.block

生成通道文件

1
# 生成创始块文件
2
$ configtxgen -profile OrgsChannel -outputCreateChannelTx ./channel-artifacts/channel.tx -channelID testchannel

8.4. Zookeeper设置#

8.4.1 基本概念#

Zookeeper一种在分布式系统中被广泛用来作为分布式状态管理、分布式协调管理、分布式配置管理和分布式锁服务的集群。

zookeeper 的运作流程
在配置之前, 让我们先了解一下 Zookeeper 的基本运转流程:
- 选举Leader
  - 选举Leader过程中算法有很多，但要达到的选举标准是一致的
  - Leader要具有最高的执行ID，类似root权限。
  - 集群中大多数的机器得到响应并跟随选出的Leader。
- 数据同步
Zookeeper的集群数量

Zookeeper 集群的数量可以是 3, 5, 7, 它值需要是一个奇数以避免脑裂问题（split-brain）的情况。同时选择大于1的值是为了避免单点故障，如果集群的数量超过7个Zookeeper服务将会无法承受。

8.4.2 zookeeper配置文件模板#

配置文件模板

下面我们来看一个示例配置文件, 研究下zookeeper如何配置:

1
version: '2'
2
services:
3
  zookeeper1: # 服务器名, 自己起
4
    container_name: zookeeper1 # 容器名, 自己起
5
    hostname: zookeeper1  # 访问的主机名, 自己起, 需要和IP有对应关系
6
    image: hyperledger/fabric-zookeeper:latest
7
    restart: always  # 指定为always
8
    environment:
9
      # ID在集合中必须是唯一的并且应该有一个值，在1和255之间。
10
      - ZOO_MY_ID=1
11
      # server.x=hostname:prot1:port2
12
      - ZOO_SERVERS=server.1=zookeeper1:2888:3888 server.2=zookeeper2:2888:3888 server.3=zookeeper3:2888:3888
13
    ports:
14
      - 2181:2181
15
      - 2888:2888
16
      - 3888:3888
17
    extra_hosts:
18
      - zookeeper1:192.168.24.201
19
      - zookeeper2:192.168.24.202
20
      - zookeeper3:192.168.24.203
21
      - kafka1:192.168.24.204
22
      - kafka2:192.168.24.205
23
      - kafka3:192.168.24.206
24
      - kafka4:192.168.24.207

相关配置项解释:
1. docker 的restart策略
  - no – 容器退出时不要自动重启，这个是默认值。
  - on-failure[] – 只在容器以非0状态码退出时重启，例如：on-failure:10
  - always – 不管退出状态码是什么始终重启容器
  - unless-stopped – 不管退出状态码是什么始终重启容器，不过当daemon启动时，如果容器之前已经为停止状态，不要尝试启动它。
2. 环境变量
  - ZOO_MY_ID
    
    zookeeper集群中的当前zookeeper服务器节点的ID, 在集群中这个只是唯一的, 范围: 1-255
  - ZOO_SERVERS
    - 组成zookeeper集群的服务器列表
    - 列表中每个服务器的值都附带两个端口号
      
      第一个: 追随者用来连接 Leader 使用的
      
      第二个: 用户选举 Leader
3. zookeeper服务器中三个重要端口:
  - 访问zookeeper的端口: 2181
  - zookeeper集群中追随者连接 Leader 的端口: 2888
  - zookeeper集群中选举 Leader 的端口: 3888
4. extra_hosts
  - 设置服务器名和其指向的IP地址的对应关系
  - zookeeper1:192.168.24.201
    - 看到名字zookeeper1就会将其解析为IP地址: 192.168.24.201

8.4.3 各个zookeeper节点的配置#

zookeeper1 配置#

1
version: '2'
2

3
services:
4

5
  zookeeper1:
6
    container_name: zookeeper1
7
    hostname: zookeeper1
8
    image: hyperledger/fabric-zookeeper:latest
9
    restart: always
10
    environment:
11
      # ID在集合中必须是唯一的并且应该有一个值，在1和255之间。
12
      - ZOO_MY_ID=1
13
      # server.x=[hostname]:nnnnn[:nnnnn]
14
      - ZOO_SERVERS=server.1=zookeeper1:2888:3888 server.2=zookeeper2:2888:3888 server.3=zookeeper3:2888:3888
15
    ports:
16
      - 2181:2181
17
      - 2888:2888
18
      - 3888:3888
19
    extra_hosts:
20
      - zookeeper1:192.168.247.101
21
      - zookeeper2:192.168.247.102
22
      - zookeeper3:192.168.247.103
23
      - kafka1:192.168.247.201
24
      - kafka2:192.168.247.202
25
      - kafka3:192.168.247.203
26
      - kafka4:192.168.247.204

zookeeper2 配置#

1
version: '2'
2

3
services:
4

5
  zookeeper2:
6
    container_name: zookeeper2
7
    hostname: zookeeper2
8
    image: hyperledger/fabric-zookeeper:latest
9
    restart: always
10
    environment:
11
      # ID在集合中必须是唯一的并且应该有一个值，在1和255之间。
12
      - ZOO_MY_ID=2
13
      # server.x=[hostname]:nnnnn[:nnnnn]
14
      - ZOO_SERVERS=server.1=zookeeper1:2888:3888 server.2=zookeeper2:2888:3888 server.3=zookeeper3:2888:3888
15
    ports:
16
      - 2181:2181
17
      - 2888:2888
18
      - 3888:3888
19
    extra_hosts:
20
      - zookeeper1:192.168.247.101
21
      - zookeeper2:192.168.247.102
22
      - zookeeper3:192.168.247.103
23
      - kafka1:192.168.247.201
24
      - kafka2:192.168.247.202
25
      - kafka3:192.168.247.203
26
      - kafka4:192.168.247.204

zookeeper3 配置#

1
version: '2'
2

3
services:
4

5
  zookeeper3:
6
    container_name: zookeeper3
7
    hostname: zookeeper3
8
    image: hyperledger/fabric-zookeeper:latest
9
    restart: always
10
    environment:
11
      # ID在集合中必须是唯一的并且应该有一个值，在1和255之间。
12
      - ZOO_MY_ID=3
13
      # server.x=[hostname]:nnnnn[:nnnnn]
14
      - ZOO_SERVERS=server.1=zookeeper1:2888:3888 server.2=zookeeper2:2888:3888 server.3=zookeeper3:2888:3888
15
    ports:
16
      - 2181:2181
17
      - 2888:2888
18
      - 3888:3888
19
    extra_hosts:
20
      - zookeeper1:192.168.247.101
21
      - zookeeper2:192.168.247.102
22
      - zookeeper3:192.168.247.103
23
      - kafka1:192.168.247.201
24
      - kafka2:192.168.247.202
25
      - kafka3:192.168.247.203
26
      - kafka4:192.168.247.204

8.5. Kafka设置#

8.5.1 基本概念#

Katka是一个分布式消息系统，由LinkedIn使用scala编写，用作LinkedIn的活动流（Activitystream)和运营数据处理管道（Pipeline）的基础。具有高水平扩展和高吞吐量。

在Fabric网络中，数据是由Peer节点提交到Orderer排序服务，而Orderer相对于Kafka来说相当于上游模块，且Orderer还兼具提供了对数据进行排序及生成符合配置规范及要求的区块。而使用上游模块的数据计算、统计、分析，这个时候就可以使用类似于Kafka这样的分布式消息系统来协助业务流程。

有人说Kafka是一种共识模式，也就是说平等信任，所有的HyperLedger Fabric网络加盟方都是可信方，因为消息总是均匀地分布在各处。但具体生产使用的时候是依赖于背书来做到确权，相对而言，Kafka应该只能是一种启动Fabric网络的模式或类型。

Zookeeper一种在分布式系统中被广泛用来作为分布式状态管理、分布式协调管理、分布式配置管理和分布式锁服务的集群。Kafka增加和减少服务器都会在Zookeeper节点上触发相应的事件，Kafka系统会捕获这些事件，进行新一轮的负载均衡，客户端也会捕获这些事件来进行新一轮的处理。

Orderer排序服务是Fablic网络事务流中的最重要的环节，也是所有请求的点，它并不会立刻对请求给予回馈，一是因为生成区块的条件所限，二是因为依托下游集群的消息处理需要等待结果。

8.5.2 kafka配置文件模板#

kafka配置文件模板

1
version: '2'
2

3
services:
4
  kafka1:
5
    container_name: kafka1
6
    hostname: kafka1
7
    image: hyperledger/fabric-kafka:latest
8
    restart: always
9
    environment:
10
      # broker.id
11
      - KAFKA_BROKER_ID=1
12
      - KAFKA_MIN_INSYNC_REPLICAS=2
13
      - KAFKA_DEFAULT_REPLICATION_FACTOR=3
14
      - KAFKA_ZOOKEEPER_CONNECT=zookeeper1:2181,zookeeper2:2181,zookeeper3:2181
15
      # 99 * 1024 * 1024 B
16
      - KAFKA_MESSAGE_MAX_BYTES=103809024
17
      - KAFKA_REPLICA_FETCH_MAX_BYTES=103809024 # 99 * 1024 * 1024 B
18
      - KAFKA_UNCLEAN_LEADER_ELECTION_ENABLE=false
19
      - KAFKA_LOG_RETENTION_MS=-1
20
      - KAFKA_HEAP_OPTS=-Xmx256M -Xms128M
21
    ports:
22
      - 9092:9092
23
    extra_hosts:
24
      - "zookeeper1:192.168.24.201"
25
      - zookeeper2:192.168.24.202
26
      - zookeeper3:192.168.24.203
27
      - kafka1:192.168.24.204
28
      - kafka2:192.168.24.205
29
      - kafka3:192.168.24.206
30
      - kafka4:192.168.24.207

配置项解释
1. Kafka 默认端口为: 9092
2. 环境变量:
- KAFKA_BROKER_ID
  - 是一个唯一的非负整数, 可以作为代理Broker的名字
- KAFKA_MIN_INSYNC_REPLICAS
  - 最小同步备份
  - 该值要小于环境变量 KAFKA_DEFAULT_REPLICATION_FACTOR的值
- KAFKA_DEFAULT_REPLICATION_FACTOR
  - 默认同步备份, 该值要小于kafka集群数量
- KAFKA_ZOOKEEPER_CONNECT
  - 指向zookeeper节点的集合
- KAFKA_MESSAGE_MAX_BYTES
  - 消息的最大字节数
  - 和配置文件configtx.yaml中的Orderer.BatchSize.AbsoluteMaxBytes对应
  - 由于消息都有头信息, 所以这个值要比计算出的值稍大, 多加1M就足够了
- KAFKA_REPLICA_FETCH_MAX_BYTES=103809024
  - 副本最大字节数, 试图为每个channel获取的消息的字节数
  - AbsoluteMaxBytes <KAFKA_REPLICA_FETCH_MAX_BYTES <= KAFKA_MESSAGE_MAX_BYTES
- KAFKA_UNCLEAN_LEADER_ELECTION_ENABLE=false
  - 非一致性的 Leader 选举
    - 开启: true
    - 关闭: false
- KAFKA_LOG_RETENTION_MS=-1
  - 对压缩日志保留的最长时间
  - 这个选项在Kafka中已经默认关闭
- KAFKA_HEAP_OPTS
  - 设置堆内存大小, kafka默认为 1G
    - -Xmx256M -> 允许分配的堆内存
    - -Xms128M -> 初始分配的堆内存

8.5.3 各个kafka节点的配置#

kafka1 配置#

1
version: '2'
2

3
services:
4

5
  kafka1:
6
    container_name: kafka1
7
    hostname: kafka1
8
    image: hyperledger/fabric-kafka:latest
9
    restart: always
10
    environment:
11
      # broker.id
12
      - KAFKA_BROKER_ID=1
13
      - KAFKA_MIN_INSYNC_REPLICAS=2
14
      - KAFKA_DEFAULT_REPLICATION_FACTOR=3
15
      - KAFKA_ZOOKEEPER_CONNECT=zookeeper1:2181,zookeeper2:2181,zookeeper3:2181
16
      # 100 * 1024 * 1024 B
17
      - KAFKA_MESSAGE_MAX_BYTES=104857600
18
      - KAFKA_REPLICA_FETCH_MAX_BYTES=104857600
19
      - KAFKA_UNCLEAN_LEADER_ELECTION_ENABLE=false
20
      - KAFKA_LOG_RETENTION_MS=-1
21
      - KAFKA_HEAP_OPTS=-Xmx512M -Xms256M
22
    ports:
23
      - 9092:9092
24
    extra_hosts:
25
      - zookeeper1:192.168.247.101
26
      - zookeeper2:192.168.247.102
27
      - zookeeper3:192.168.247.103
28
      - kafka1:192.168.247.201
29
      - kafka2:192.168.247.202
30
      - kafka3:192.168.247.203
31
      - kafka4:192.168.247.204

kafka2 配置#

1
version: '2'
2

3
services:
4

5
  kafka2:
6
    container_name: kafka2
7
    hostname: kafka2
8
    image: hyperledger/fabric-kafka:latest
9
    restart: always
10
    environment:
11
      # broker.id
12
      - KAFKA_BROKER_ID=2
13
      - KAFKA_MIN_INSYNC_REPLICAS=2
14
      - KAFKA_DEFAULT_REPLICATION_FACTOR=3
15
      - KAFKA_ZOOKEEPER_CONNECT=zookeeper1:2181,zookeeper2:2181,zookeeper3:2181
16
      # 100 * 1024 * 1024 B
17
      - KAFKA_MESSAGE_MAX_BYTES=104857600
18
      - KAFKA_REPLICA_FETCH_MAX_BYTES=104857600
19
      - KAFKA_UNCLEAN_LEADER_ELECTION_ENABLE=false
20
      - KAFKA_LOG_RETENTION_MS=-1
21
      - KAFKA_HEAP_OPTS=-Xmx512M -Xms256M
22
    ports:
23
      - 9092:9092
24
    extra_hosts:
25
      - zookeeper1:192.168.247.101
26
      - zookeeper2:192.168.247.102
27
      - zookeeper3:192.168.247.103
28
      - kafka1:192.168.247.201
29
      - kafka2:192.168.247.202
30
      - kafka3:192.168.247.203
31
      - kafka4:192.168.247.204

kafka3 配置#

1
version: '2'
2

3
services:
4

5
  kafka3:
6
    container_name: kafka3
7
    hostname: kafka3
8
    image: hyperledger/fabric-kafka:latest
9
    restart: always
10
    environment:
11
      # broker.id
12
      - KAFKA_BROKER_ID=3
13
      - KAFKA_MIN_INSYNC_REPLICAS=2
14
      - KAFKA_DEFAULT_REPLICATION_FACTOR=3
15
      - KAFKA_ZOOKEEPER_CONNECT=zookeeper1:2181,zookeeper2:2181,zookeeper3:2181
16
      # 100 * 1024 * 1024 B
17
      - KAFKA_MESSAGE_MAX_BYTES=104857600
18
      - KAFKA_REPLICA_FETCH_MAX_BYTES=104857600
19
      - KAFKA_UNCLEAN_LEADER_ELECTION_ENABLE=false
20
      - KAFKA_LOG_RETENTION_MS=-1
21
      - KAFKA_HEAP_OPTS=-Xmx512M -Xms256M
22
    ports:
23
      - 9092:9092
24
    extra_hosts:
25
      - zookeeper1:192.168.247.101
26
      - zookeeper2:192.168.247.102
27
      - zookeeper3:192.168.247.103
28
      - kafka1:192.168.247.201
29
      - kafka2:192.168.247.202
30
      - kafka3:192.168.247.203
31
      - kafka4:192.168.247.204

kafka4 配置#

1
version: '2'
2
services:
3

4
  kafka4:
5
    container_name: kafka4
6
    hostname: kafka4
7
    image: hyperledger/fabric-kafka:latest
8
    restart: always
9
    environment:
10
      # broker.id
11
      - KAFKA_BROKER_ID=4
12
      - KAFKA_MIN_INSYNC_REPLICAS=2
13
      - KAFKA_DEFAULT_REPLICATION_FACTOR=3
14
      - KAFKA_ZOOKEEPER_CONNECT=zookeeper1:2181,zookeeper2:2181,zookeeper3:2181
15
      # 100 * 1024 * 1024 B
16
      - KAFKA_MESSAGE_MAX_BYTES=104857600
17
      - KAFKA_REPLICA_FETCH_MAX_BYTES=104857600
18
      - KAFKA_UNCLEAN_LEADER_ELECTION_ENABLE=false
19
      - KAFKA_LOG_RETENTION_MS=-1
20
      - KAFKA_HEAP_OPTS=-Xmx512M -Xms256M
21
    ports:
22
      - 9092:9092
23
    extra_hosts:
24
      - zookeeper1:192.168.247.101
25
      - zookeeper2:192.168.247.102
26
      - zookeeper3:192.168.247.103
27
      - kafka1:192.168.247.201
28
      - kafka2:192.168.247.202
29
      - kafka3:192.168.247.203
30
      - kafka4:192.168.247.204

8.6. orderer节点设置#

8.6.1 orderer节点配置文件模板#

orderer节点配置文件模板

1
version: '2'
2

3
services:
4

5
  orderer0.example.com:
6
    container_name: orderer0.example.com
7
    image: hyperledger/fabric-orderer:latest
8
    environment:
9
      - CORE_VM_DOCKER_HOSTCONFIG_NETWORKMODE=aberic_default
10
      - ORDERER_GENERAL_LOGLEVEL=debug
11
      - ORDERER_GENERAL_LISTENADDRESS=0.0.0.0
12
      - ORDERER_GENERAL_LISTENPORT=7050
13
      - ORDERER_GENERAL_GENESISMETHOD=file
14
      - ORDERER_GENERAL_GENESISFILE=/var/hyperledger/orderer/orderer.genesis.block
15
      - ORDERER_GENERAL_LOCALMSPID=OrdererMSP
16
      - ORDERER_GENERAL_LOCALMSPDIR=/var/hyperledger/orderer/msp
17
      # enabled TLS
18
      - ORDERER_GENERAL_TLS_ENABLED=false
19
      - ORDERER_GENERAL_TLS_PRIVATEKEY=/var/hyperledger/orderer/tls/server.key
20
      - ORDERER_GENERAL_TLS_CERTIFICATE=/var/hyperledger/orderer/tls/server.crt
21
      - ORDERER_GENERAL_TLS_ROOTCAS=[/var/hyperledger/orderer/tls/ca.crt]
22

23
      - ORDERER_KAFKA_RETRY_LONGINTERVAL=10s
24
      - ORDERER_KAFKA_RETRY_LONGTOTAL=100s
25
      - ORDERER_KAFKA_RETRY_SHORTINTERVAL=1s
26
      - ORDERER_KAFKA_RETRY_SHORTTOTAL=30s
27
      - ORDERER_KAFKA_VERBOSE=true
28
      - ORDERER_KAFKA_BROKERS=[192.168.24.204:9092,192.168.24.205:9092,192.168.24.206:9092,192.168.24.207:9092]
29
    working_dir: /opt/gopath/src/github.com/hyperledger/fabric
30
    command: orderer
31
    volumes:
32
      - ./channel-artifacts/genesis.block:/var/hyperledger/orderer/orderer.genesis.block
33
      - ./crypto-config/ordererOrganizations/example.com/orderers/orderer0.example.com/msp:/var/hyperledger/orderer/msp
34
      - ./crypto-config/ordererOrganizations/example.com/orderers/orderer0.example.com/tls/:/var/hyperledger/orderer/tls
35
    networks:
36
      default:
37
        aliases:
38
          - aberic
39
    ports:
40
      - 7050:7050
41
    extra_hosts:
42
      - kafka1:192.168.24.204
43
      - kafka2:192.168.24.205
44
      - kafka3:192.168.24.206
45
      - kafka4:192.168.24.207

细节解释
1. 环境变量
  - ORDERER_KAFKA_RETRY_LONGINTERVAL
    - 每隔多长时间进行一次重试, 单位:秒
  - ORDERER_KAFKA_RETRY_LONGTOTAL
    - 总共重试的时长, 单位: 秒
  - ORDERER_KAFKA_RETRY_SHORTINTERVAL
    - 每隔多长时间进行一次重试, 单位:秒
  - ORDERER_KAFKA_RETRY_SHORTTOTAL
    - 总共重试的时长, 单位: 秒
  - ORDERER_KAFKA_VERBOSE
    - 启用日志与kafka进行交互, 启用: true, 不启用: false
  - ORDERER_KAFKA_BROKERS
    - 指向kafka节点的集合
2. 关于重试的时长
  - 先使用ORDERER_KAFKA_RETRY_SHORTINTERVAL进行重连, 重连的总时长为ORDERER_KAFKA_RETRY_SHORTTOTAL
  - 如果上述步骤没有重连成功, 使用ORDERER_KAFKA_RETRY_LONGINTERVAL进行重连, 重连的总时长为ORDERER_KAFKA_RETRY_LONGTOTAL

8.6.3 orderer各节点的配置#

orderer0配置#

1
version: '2'
2

3
services:
4

5
  orderer0.test.com:
6
    container_name: orderer0.test.com
7
    image: hyperledger/fabric-orderer:latest
8
    environment:
9
      - CORE_VM_DOCKER_HOSTCONFIG_NETWORKMODE=kafka_default
10
      - ORDERER_GENERAL_LOGLEVEL=debug
11
      - ORDERER_GENERAL_LISTENADDRESS=0.0.0.0
12
      - ORDERER_GENERAL_LISTENPORT=7050
13
      - ORDERER_GENERAL_GENESISMETHOD=file
14
      - ORDERER_GENERAL_GENESISFILE=/var/hyperledger/orderer/orderer.genesis.block
15
      - ORDERER_GENERAL_LOCALMSPID=OrdererMSP
16
      - ORDERER_GENERAL_LOCALMSPDIR=/var/hyperledger/orderer/msp
17
      # enabled TLS
18
      - ORDERER_GENERAL_TLS_ENABLED=false
19
      - ORDERER_GENERAL_TLS_PRIVATEKEY=/var/hyperledger/orderer/tls/server.key
20
      - ORDERER_GENERAL_TLS_CERTIFICATE=/var/hyperledger/orderer/tls/server.crt
21
      - ORDERER_GENERAL_TLS_ROOTCAS=[/var/hyperledger/orderer/tls/ca.crt]
22

23
      - ORDERER_KAFKA_RETRY_LONGINTERVAL=10s
24
      - ORDERER_KAFKA_RETRY_LONGTOTAL=100s
25
      - ORDERER_KAFKA_RETRY_SHORTINTERVAL=1s
26
      - ORDERER_KAFKA_RETRY_SHORTTOTAL=30s
27
      - ORDERER_KAFKA_VERBOSE=true
28
      - ORDERER_KAFKA_BROKERS=[192.168.247.201:9092,192.168.247.202:9092,192.168.247.203:9092,192.168.247.204:9092]
29
    working_dir: /opt/gopath/src/github.com/hyperledger/fabric
30
    command: orderer
31
    volumes:
32
      - ./channel-artifacts/genesis.block:/var/hyperledger/orderer/orderer.genesis.block
33
      - ./crypto-config/ordererOrganizations/test.com/orderers/orderer0.test.com/msp:/var/hyperledger/orderer/msp
34
      - ./crypto-config/ordererOrganizations/test.com/orderers/orderer0.test.com/tls/:/var/hyperledger/orderer/tls
35
    networks:
36
    default:
37
      aliases:
38
        - kafka
39
    ports:
40
      - 7050:7050
41
    extra_hosts:
42
      - kafka1:192.168.247.201
43
      - kafka2:192.168.247.202
44
      - kafka3:192.168.247.203
45
      - kafka4:192.168.247.204

orderer1配置#

1
version: '2'
2

3
services:
4

5
  orderer1.test.com:
6
    container_name: orderer1.test.com
7
    image: hyperledger/fabric-orderer:latest
8
    environment:
9
      - CORE_VM_DOCKER_HOSTCONFIG_NETWORKMODE=kafka_default
10
      - ORDERER_GENERAL_LOGLEVEL=debug
11
      - ORDERER_GENERAL_LISTENADDRESS=0.0.0.0
12
      - ORDERER_GENERAL_LISTENPORT=7050
13
      - ORDERER_GENERAL_GENESISMETHOD=file
14
      - ORDERER_GENERAL_GENESISFILE=/var/hyperledger/orderer/orderer.genesis.block
15
      - ORDERER_GENERAL_LOCALMSPID=OrdererMSP
16
      - ORDERER_GENERAL_LOCALMSPDIR=/var/hyperledger/orderer/msp
17
      # enabled TLS
18
      - ORDERER_GENERAL_TLS_ENABLED=false
19
      - ORDERER_GENERAL_TLS_PRIVATEKEY=/var/hyperledger/orderer/tls/server.key
20
      - ORDERER_GENERAL_TLS_CERTIFICATE=/var/hyperledger/orderer/tls/server.crt
21
      - ORDERER_GENERAL_TLS_ROOTCAS=[/var/hyperledger/orderer/tls/ca.crt]
22

23
      - ORDERER_KAFKA_RETRY_LONGINTERVAL=10s
24
      - ORDERER_KAFKA_RETRY_LONGTOTAL=100s
25
      - ORDERER_KAFKA_RETRY_SHORTINTERVAL=1s
26
      - ORDERER_KAFKA_RETRY_SHORTTOTAL=30s
27
      - ORDERER_KAFKA_VERBOSE=true
28
      - ORDERER_KAFKA_BROKERS=[192.168.247.201:9092,192.168.247.202:9092,192.168.247.203:9092,192.168.247.204:9092]
29
    working_dir: /opt/gopath/src/github.com/hyperledger/fabric
30
    command: orderer
31
    volumes:
32
      - ./channel-artifacts/genesis.block:/var/hyperledger/orderer/orderer.genesis.block
33
      - ./crypto-config/ordererOrganizations/test.com/orderers/orderer1.test.com/msp:/var/hyperledger/orderer/msp
34
      - ./crypto-config/ordererOrganizations/test.com/orderers/orderer1.test.com/tls/:/var/hyperledger/orderer/tls
35
    networks:
36
    default:
37
      aliases:
38
        - kafka
39
    ports:
40
      - 7050:7050
41
    extra_hosts:
42
      - kafka1:192.168.247.201
43
      - kafka2:192.168.247.202
44
      - kafka3:192.168.247.203
45
      - kafka4:192.168.247.204

orderer2配置#

1
version: '2'
2

3
services:
4

5
  orderer2.test.com:
6
    container_name: orderer2.test.com
7
    image: hyperledger/fabric-orderer:latest
8
    environment:
9
      - CORE_VM_DOCKER_HOSTCONFIG_NETWORKMODE=kafka_default
10
      - ORDERER_GENERAL_LOGLEVEL=debug
11
      - ORDERER_GENERAL_LISTENADDRESS=0.0.0.0
12
      - ORDERER_GENERAL_LISTENPORT=7050
13
      - ORDERER_GENERAL_GENESISMETHOD=file
14
      - ORDERER_GENERAL_GENESISFILE=/var/hyperledger/orderer/orderer.genesis.block
15
      - ORDERER_GENERAL_LOCALMSPID=OrdererMSP
16
      - ORDERER_GENERAL_LOCALMSPDIR=/var/hyperledger/orderer/msp
17
      # enabled TLS
18
      - ORDERER_GENERAL_TLS_ENABLED=false
19
      - ORDERER_GENERAL_TLS_PRIVATEKEY=/var/hyperledger/orderer/tls/server.key
20
      - ORDERER_GENERAL_TLS_CERTIFICATE=/var/hyperledger/orderer/tls/server.crt
21
      - ORDERER_GENERAL_TLS_ROOTCAS=[/var/hyperledger/orderer/tls/ca.crt]
22

23
      - ORDERER_KAFKA_RETRY_LONGINTERVAL=10s
24
      - ORDERER_KAFKA_RETRY_LONGTOTAL=100s
25
      - ORDERER_KAFKA_RETRY_SHORTINTERVAL=1s
26
      - ORDERER_KAFKA_RETRY_SHORTTOTAL=30s
27
      - ORDERER_KAFKA_VERBOSE=true
28
      - ORDERER_KAFKA_BROKERS=[192.168.247.201:9092,192.168.247.202:9092,192.168.247.203:9092,192.168.247.204:9092]
29
    working_dir: /opt/gopath/src/github.com/hyperledger/fabric
30
    command: orderer
31
    volumes:
32
      - ./channel-artifacts/genesis.block:/var/hyperledger/orderer/orderer.genesis.block
33
      - ./crypto-config/ordererOrganizations/test.com/orderers/orderer2.test.com/msp:/var/hyperledger/orderer/msp
34
      - ./crypto-config/ordererOrganizations/test.com/orderers/orderer2.test.com/tls/:/var/hyperledger/orderer/tls
35
    networks:
36
    default:
37
      aliases:
38
        - kafka
39
    ports:
40
      - 7050:7050
41
    extra_hosts:
42
      - kafka1:192.168.247.201
43
      - kafka2:192.168.247.202
44
      - kafka3:192.168.247.203
45
      - kafka4:192.168.247.204

8.7. 启动集群#

Kafka集群的启动顺序是这样的: 先启动Zookeeper集群, 随后启动Kafka集群, 最后启动Orderer排序服务器集群。由于peer节点只能和集群中orderer节点进行通信, 所以不管是使用solo集群还是kafka集群对peer都是没有影响的, 所以当我们的kafka集群顺利启动之后, 就可以启动对应的Peer节点了。

8.7.1 启动Zookeeper集群#

zookeeper1:192.168.247.101

1
$ cd ~/kafka
2
# 将写好的 zookeeper1.yaml 配置文件放到该目录下, 通过 docker-compose 启动容器
3
# 该命令可以不加 -d 参数, 这样就能看到当前 zookeeper 服务器启动的情况了
4
$ docker-compose -f zookeeper1.yaml up

zookeeper2:192.168.247.102

1
$ cd ~/kafka
2
# 将写好的 zookeeper2.yaml 配置文件放到该目录下, 通过 docker-compose 启动容器
3
# 该命令可以不加 -d 参数, 这样就能看到当前 zookeeper 服务器启动的情况了
4
$ docker-compose -f zookeeper2.yaml up

zookeeper3:192.168.247.103

1
$ cd ~/kafka
2
# 将写好的 zookeeper3.yaml 配置文件放到该目录下, 通过 docker-compose 启动容器
3
# 该命令可以不加 -d 参数, 这样就能看到当前 zookeeper 服务器启动的情况了
4
$ docker-compose -f zookeeper3.yaml up

8.7.2 启动Kafka集群#

kafka1:192.168.247.201

1
$ cd ~/kafka
2
# 将写好的 kafka1.yaml 配置文件放到该目录下, 通过 docker-compose 启动容器
3
# 该命令可以不加 -d 参数, 这样就能看到当前 kafka 服务器启动的情况了
4
$ docker-compose -f kafka1.yaml up

kafka2:192.168.247.202

1
$ cd ~/kafka
2
# 将写好的 kafka2.yaml 配置文件放到该目录下, 通过 docker-compose 启动容器
3
$ docker-compose -f kafka2.yaml up -d

kafka3:192.168.247.203

1
$ cd ~/kafka
2
# 将写好的 kafka3.yaml 配置文件放到该目录下, 通过 docker-compose 启动容器
3
$ docker-compose -f kafka3.yaml up -d

kafka4:192.168.247.204

1
$ cd ~/kafka
2
# 将写好的 kafka4.yaml 配置文件放到该目录下, 通过 docker-compose 启动容器
3
$ docker-compose -f kafka4.yaml up

8.7.3 启动Orderer集群#

orderer0:192.168.247.91

1
$ cd ~/kafka
2
# 假设生成证书和通道创始块文件操作是在当前 orderer0 上完成的, 那么应该在当前 kafka 工作目录下
3
$ tree ./ -L 1
4
./
5
├── channel-artifacts
6
├── configtx.yaml
7
├── crypto-config
8
└── crypto-config.yaml
9
# 将写好的 orderer0.yaml 配置文件放到该目录下, 通过 docker-compose 启动容器
10
$ docker-compose -f orderer0.yaml up -d

orderer1:192.168.247.92

1
# 将生成的 证书文件目录 和 通道创始块 文件目录拷贝到当前主机的 ~/kafka目录中
2
$ cd ~/kafka
3
# 创建子目录 crypto-config
4
$ mkdir crypto-config
5
# 远程拷贝
6
$ scp -f itcast@192.168.247.91:/home/itcast/kafka/crypto-config/ordererOrganizations ./crypto-config
7
# # 将写好的 orderer1.yaml 配置文件放到该目录下, 通过 docker-compose 启动容器
8
$ docker-compose -f orderer1.yaml up -d

orderer2:192.168.247.93

1
# 将生成的 证书文件目录 和 通道创始块 文件目录拷贝到当前主机的 ~/kafka目录中
2
$ cd ~/kafka
3
# 创建子目录 crypto-config
4
$ mkdir crypto-config
5
# 远程拷贝
6
$ scp -f itcast@192.168.247.91:/home/itcast/kafka/crypto-config/ordererOrganizations ./crypto-config
7
# # 将写好的 orderer3.yaml 配置文件放到该目录下, 通过 docker-compose 启动容器
8
$ docker-compose -f orderer3.yaml up -d

8.7.4 启动Peer集群#

关于 Peer 节点的部署和操作和 Solo 多机多节点部署的方式是完全一样的, 在此不再阐述, 请查阅第七章。