pinot伪分布式搭建
本文中的伪分布式指的是pinot架构中每个组件controller、server、broker分别启动组成集群
1、进入linux 的pinot目录的bin文件夹下
2、pinot启动controller，使用自己的zookeeper集群非pinot中的zookeeper
nohup ./start-controller.sh -clusterName PinotCluster -controllerHost 192.168.0.110 -controllerPort 9000 -dataDir /tmp/PinotController1 -zkAddress 192.168.0.110:2181,192.168.0.111:2181,192.168.0.112:2181 &
controller界面查询
3、pinot启动broker
nohup ./start-broker.sh -brokerHost 192.168.0.110 -brokerPort 8000 -zkAddress 192.168.0.110:2181,192.168.0.111:2181,192.168.0.112:2181 &
4、pinot启动server
nohup ./start-server.sh -clusterName PinotCluster -serverHost 192.168.0.110 -serverPort 7000 -serverAdminPort 7500 -dataDir /tmp/1581494836753/PinotServerData0 -segmentDir /tmp/1581494836753/PinotServerSegment0 -zkAddress 192.168.0.110:2181,192.168.0.111:2181,192.168.0.112:2181 &
5、pinot创建表（linux命令行形式）
./pinot-admin.sh AddTable -tableConfigFile /wb/pinot_compile/apache-pinot-incubating-0.3.0-SNAPSHOT-bin/quickStartData1581476113668/meetupRsvp_realtime_table_config.json -schemaFile /wb/pinot_compile/apache-pinot-incubating-0.3.0-SNAPSHOT-bin/quickStartData1581476113668/meetupRsvp_schema.json -controllerHost 192.168.0.110 -controllerPort 9000 -exec
表的配置meetupRsvp_realtime_table_config.json格式
{
"tableName": "meetupRsvp",
"tableType": "REALTIME",
"segmentsConfig": {
"timeColumnName": "mtime",
"timeType": "MILLISECONDS",
"segmentPushType": "APPEND",
"segmentAssignmentStrategy": "BalanceNumSegmentAssignmentStrategy",
"schemaName": "meetupRsvp",
"replication": "1"
},
"tenants": {},
"tableIndexConfig": {
"loadMode": "MMAP",
"streamConfigs": {
"streamType": "kafka",
"stream.kafka.consumer.type": "highLevel",
"stream.kafka.topic.name": "meetupRSVPEvents",
"stream.kafka.decoder.class.name": "org.apache.pinot.plugin.stream.kafka.KafkaJSONMessageDecoder",
"stream.kafka.hlc.zk.connect.string": "localhost:2191/kafka",
"stream.kafka.consumer.factory.class.name": "org.apache.pinot.plugin.stream.kafka20.KafkaConsumerFactory",
"stream.kafka.zk.broker.url": "localhost:2191/kafka",
"stream.kafka.hlc.bootstrap.server": "localhost:19092"
}
},
"metadata": {
"customConfigs": {}
}
}
schema的配置 meetupRsvp_schema.json
{
"metricFieldSpecs": [
{
"dataType": "INT",
"name": "rsvp_count"
}
],
"dimensionFieldSpecs": [
{
"dataType": "STRING",
"name": "venue_name"
},
{
"dataType": "STRING",
"name": "event_name"
},
{
"dataType": "STRING",
"name": "event_id"
},
{
"dataType": "LONG",
"name": "event_time"
},
{
"dataType": "STRING",
"name": "group_city"
},
{
"dataType": "STRING",
"name": "group_country"
},
{
"dataType": "LONG",
"name": "group_id"
},
{
"dataType": "STRING",
"name": "group_name"
}
],
"timeFieldSpec": {
"incomingGranularitySpec": {
"timeType": "MILLISECONDS",
"dataType": "LONG",
"name": "mtime"
}
},
"schemaName": "meetupRsvp"
}
结果可以创建成功并查询（使用linux命令行进行查询）
./pinot-admin.sh PostQuery -brokerPort 8000 -query "select * from meetupRsvp order by mtime limit 10"