【消息队列】RocketMQ配置⽂件broker
前⾔
broker主要涉及broker 配置⽂件、broker 启动流程、broker延迟消息、broker消息存储。
⼀、 broker 配置⽂件
1.1 broker⽬录结构
├── abort
├── checkpoint
├── commitlog
│└── 00000000000000000000
├── config
│├── consumerFilter.json
│├── consumerOfft.json
│├── delayOfft.json
│├── subscriptionGroup.json
│├── topics.json
├── consumequeue
如何写个人简介│├── SCHEDULE_TOPIC_XXXX
││├── 0
│││└── 00000000000000000000
││├── 1
│││└── 00000000000000000000
││└── 2
││└── 00000000000000000000
│└── TopicTest
│├── 0
││└── 00000000000000000000
│├── 1
││└── 00000000000000000000
│├── 2
││└── 00000000000000000000
│├── 3
││└── 00000000000000000000
│├── 4
││└── 00000000000000000000
│├── 5
││└── 00000000000000000000
│├── 6
││└── 00000000000000000000
│└── 7
│└── 00000000000000000000
├── index
│└── 20200502170505253
└── lock
commitlog记录broker真正存储的消息。
config记录mq的各类配置信息。
consumequeue记录topic的各个队列的消费情况。
index记录mq的索引信息,指向commitlog。
1.2 topic.json
记录该broker上各topic信息,包括queue信息、读写权限。
{
"dataVersion":{
"counter":8,
"timestamp":1588575388560
},
"topicConfigTable":{
"TopicTest":{
"order":fal,
"perm":6,
罪刑法定"readQueueNums":8,
"topicFilterType":"SINGLE_TAG",
"topicName":"TopicTest",
"topicSysFlag":0,
"writeQueueNums":8
},
"SELF_TEST_TOPIC":{泥石流的危害
"order":fal,
"perm":6,
"readQueueNums":1,
"topicFilterType":"SINGLE_TAG",
"topicName":"SELF_TEST_TOPIC",
"topicSysFlag":0,
"writeQueueNums":1
},
"%RETRY%consumer_group_test":{
"order":fal,
"perm":6,
"readQueueNums":1,
"topicFilterType":"SINGLE_TAG",
"topicName":"%RETRY%consumer_group_test", "topicSysFlag":0,
"writeQueueNums":1
},
"DefaultCluster":{
"order":fal,
"perm":7,
"readQueueNums":16,
"topicFilterType":"SINGLE_TAG",
"topicName":"DefaultCluster",
"topicSysFlag":0,
"writeQueueNums":16
},
"%RETRY%order_consumer":{
"order":fal,
"perm":6,
"readQueueNums":1,
"topicFilterType":"SINGLE_TAG",
"topicName":"%RETRY%order_consumer",
"topicSysFlag":0,
"writeQueueNums":1
},
"RMQ_SYS_TRANS_HALF_TOPIC":{
"order":fal,
"perm":6,
"readQueueNums":1,
"topicFilterType":"SINGLE_TAG",
"topicName":"RMQ_SYS_TRANS_HALF_TOPIC", "topicSysFlag":0,
"writeQueueNums":1
},
"broker-a":{
"order":fal,
"perm":7,
"readQueueNums":1,
"topicName":"broker-a",
"topicSysFlag":0,
"writeQueueNums":1
},
"TBW102":{
"order":fal,
"perm":7,
"readQueueNums":8,
"topicFilterType":"SINGLE_TAG",
"topicName":"TBW102",
"topicSysFlag":0,
"writeQueueNums":8
},
"BenchmarkTest":{
"order":fal,
"perm":6,
"readQueueNums":1024,
"topicFilterType":"SINGLE_TAG",
"topicName":"BenchmarkTest",
"topicSysFlag":0,
"writeQueueNums":1024
},
"OFFSET_MOVED_EVENT":{
"order":fal,
"perm":6,
"readQueueNums":1,
"topicFilterType":"SINGLE_TAG",
"topicName":"OFFSET_MOVED_EVENT", "topicSysFlag":0,
"writeQueueNums":1
}
}
}
辣白菜的正宗做法
1.3 subscriptionGroup.json
记录该broker上各类订阅关系。
{
"dataVersion":{
"counter":1,
"timestamp":1588575546295
},
"subscriptionGroupTable":{
"SELF_TEST_C_GROUP":{
"brokerId":0,
"consumeBroadcastEnable":true,
"consumeEnable":true,
"consumeFromMinEnable":true,
"groupName":"SELF_TEST_C_GROUP", "notifyConsumerIdsChangedEnable":true, "retryMaxTimes":16,
"retryQueueNums":1,
"whichBrokerWhenConsumeSlowly":1
},
"consumer_group_test":{
"brokerId":0,
"consumeBroadcastEnable":true,
"consumeEnable":true,
"consumeFromMinEnable":true,
"groupName":"consumer_group_test",
"notifyConsumerIdsChangedEnable":true, "retryMaxTimes":16,
"retryQueueNums":1,
"whichBrokerWhenConsumeSlowly":1
分组函数},
"CID_ONSAPI_OWNER":{
"brokerId":0,
"consumeBroadcastEnable":true,
"consumeEnable":true,
"consumeFromMinEnable":true,
"groupName":"CID_ONSAPI_OWNER",
"notifyConsumerIdsChangedEnable":true, "retryMaxTimes":16,
"retryQueueNums":1,
"whichBrokerWhenConsumeSlowly":1
},
"CID_ONSAPI_PERMISSION":{
"brokerId":0,
"consumeBroadcastEnable":true,
"consumeEnable":true,
"consumeFromMinEnable":true,
"groupName":"CID_ONSAPI_PERMISSION", "notifyConsumerIdsChangedEnable":true, "retryMaxTimes":16,
"retryQueueNums":1,
"whichBrokerWhenConsumeSlowly":1
},
"order_consumer":{
"brokerId":0,
"consumeBroadcastEnable":true,
"consumeEnable":true,
"consumeFromMinEnable":true,
"groupName":"order_consumer",
"notifyConsumerIdsChangedEnable":true, "retryMaxTimes":16,
"retryQueueNums":1,
"whichBrokerWhenConsumeSlowly":1
},
"TOOLS_CONSUMER":{
"brokerId":0,
"consumeBroadcastEnable":true,
"consumeEnable":true,
"consumeFromMinEnable":true,
"groupName":"TOOLS_CONSUMER",
"notifyConsumerIdsChangedEnable":true, "retryMaxTimes":16,
"retryQueueNums":1,
"whichBrokerWhenConsumeSlowly":1
},
苦涩年华"CID_ONS-HTTP-PROXY":{
"brokerId":0,
"consumeBroadcastEnable":true,
"consumeEnable":true,
"consumeFromMinEnable":true,
"groupName":"CID_ONS-HTTP-PROXY",
"notifyConsumerIdsChangedEnable":true, "retryMaxTimes":16,
"retryQueueNums":1,
"whichBrokerWhenConsumeSlowly":1
},
"FILTERSRV_CONSUMER":{
"brokerId":0,
"consumeBroadcastEnable":true,
"consumeEnable":true,
"consumeFromMinEnable":true,
"groupName":"FILTERSRV_CONSUMER", "notifyConsumerIdsChangedEnable":true,
"retryQueueNums":1,
"whichBrokerWhenConsumeSlowly":1
},
"CID_ONSAPI_PULL":{
古诗春雪"brokerId":0,
"consumeBroadcastEnable":true,
库内"consumeEnable":true,
"consumeFromMinEnable":true,
"groupName":"CID_ONSAPI_PULL",
"notifyConsumerIdsChangedEnable":true,
"retryMaxTimes":16,
"retryQueueNums":1,
"whichBrokerWhenConsumeSlowly":1
}
}
}
1.4 delayOfft.json
记录该broker延迟队列的消费位移情况,1:1000中1表⽰延迟粒度,1000表⽰位移。
{
"offtTable":{1:1000,2:1000,3:1000
}
}
1.5 consumerOfft.json
记录该broker上针对每个topic的每个consumer group的针对每个queue的消费位移。
{
"offtTable":{
"TopicTest@plea_rename_unique_group_name_4":{0:125,1:125,2:125,3:125,4:125,5:125,6:125,7:125 },
"TopicTest@order_consumer":{0:165,1:165,2:111,3:145,4:145,5:145,6:145,7:145
},
"TopicTest@consumer_group_test":{0:540,1:540,2:520,3:520,4:520,5:520,6:520,7:520
},
"%RETRY%consumer_group_test@consumer_group_test":{0:0
},
"%RETRY%plea_rename_unique_group_name_4@plea_rename_unique_group_name_4":{0:0
},
"%RETRY%order_consumer@order_consumer":{0:0
}
}
}