cmak(kafka Manager) 编译教程

何睿范
2023-12-01

【kafka Manager 编译教程】【编译环境:Centos7】

KafkaManager下载

【2.0以下版本存在 JQuery漏洞,3.0.0.1+依赖zk3.5+,所以选用3.0.0.0,适配现网zk3.4.10】

【官方地址】
 https://github.com/yahoo/CMAK

 【版本】【3.0.0.2开始已提供编译版本】
  CMAK-3.0.0.0.zip

Sbt下载

【编译环境按照kafka manager的要求下载】

【官方地址】
 https://www.scala-sbt.org/download.html

 【版本】
 sbt 1.3.8 (.zip)

Jdk11下载

【3.0+ 依赖 jdk11,编译环境也必须配套Jdk11】

【官方地址】
 https://www.oracle.com/java/technologies/javase-jdk11-downloads.html

 【版本】
 jdk-11.0.7_linux-x64_bin.tar.gz

配置Jdk11

【3.0+ 依赖 jdk11】

【解压】
 # tar zxvf jdk-11.0.7_linux-x64_bin.tar.gz -C /home/java11

【环境变量】
# vim ~/.bashrc

#jdk#
export JAVA_HOME=/home/java11
export PATH=$JAVA_HOME/bin:$PATH

【检查】
# source ~/.bashrc
# java -version

检查sbt版本

【检查kafka Manager依赖的sbt版本】

【解压压缩包】
# unzip CMAK-3.0.0.0.zip
# vim CMAK-3.0.0.0/sbt

[可见该版本依赖1.3.8版本的sbt]
declare -r sbt_release_version="1.3.8"

安装sbt

【解压】
# unzip -q -d /usr/share/ sbt-1.3.8.zip
# cd /usr/share/sbt/

【启动脚本】
# vim /usr/share/sbt/sbt
#!/bin/bash
BT_OPTS="-Xms2048M -Xmx4096M -Xss1M -XX:+CMSClassUnloadingEnabled -XX:MaxPermSize=256M"
java $SBT_OPTS -jar /usr/share/sbt/bin/sbt-launch.jar "$@"
# chmod u+x /usr/share/sbt/sbt

【环境变量】
# vim ~/.bashrc
#SBT#
export SBT_HOME=/usr/share/sbt
export PATH=$PATH:$SBT_HOME/bin

# source ~/.bashrc

修改 sbt-launch.jar

【解包修改】
# cd /usr/share/sbt/bin/
# mkdir tmpd;cd tmpd
# cp ../sbt-launch.jar .
# mv ../sbt-launch.jar ../sbt-launch.jar.bak
# unzip -q sbt-launch.jar
# mkdir conf
# vim conf/repo.properties
[repositories]
  local
  aliyun: http://maven.aliyun.com/nexus/content/groups/public/
  central: http://repo1.maven.org/maven2/

【打包jar】
# jar -cfM ./sbt-launch.jar .
# chmod a+x sbt-launch.jar
# mv sbt-launch.jar .. && cd ../
# rm -rf tmpd

下载sbt依赖

# sbt version

copying runtime jar...
[info] [launcher] getting org.scala-sbt sbt 1.3.8  (this may take some time)...
waitting ......

【失败则修改源后重试,下载完后重新执行命令测试】
【如果下载卡在那个包,已经没有下行网速了,则ctrl+c退出后重新下载】
【修改源】
# vim ~/.sbt/repositories
[repositories]
local
aliyun: http://maven.aliyun.com/nexus/content/groups/public/
typesafe: http://repo.typesafe.com/typesafe/ivy-releases/, [organization]/[module]/(scala_[scalaVersion]/)(sbt_[sbtVersion]/)[revision]/[type]s/[artifact](-[classifier]).[ext], bootOnly
sonatype-oss-releases
maven-central
sonatype-oss-snapshots

编译KafkaManager

# cd /home/CMAK-3.0.0.0
# ./sbt clean dist

编译完成后会在:/home/kafkamanager/CMAK-3.0.0.0/target/universal 生成安装包 cmak-3.0.0.0.zip

安装KfkaManager

【解压】
# unzip -q -d /home/alpha cmak-3.0.0.0.zip
# mv cmak-3.0.0.0 cmak

目录结构

【目录结构】

【项目】
cmak/
├── bin
├── conf
├── java11
├── lib
├── README.md
├── share
├── start.sh
└── stop.sh

【日志】
/data01/alpha/cmak/logs/
└── application.log

部署目录

/home/alpha/cmak

服务端口

Port:9000

配置服务

【登陆鉴权】

$ vim conf/application.conf
【1】
kafka-manager.zkhosts="10.66.40.97:3181,10.66.40.98:3181,10.66.40.99:3181"
【2】
basicAuthentication.enabled=true
【3】
akka {
  loggers = ["akka.event.slf4j.Slf4jLogger"]
  loglevel = "OFF"
}
【4】
basicAuthentication.password="Jet#2dg%S@A"

日志配置

【默认关闭所有日志,异常时日志会很多】

【=================== logger.xml ==========================】
$ vim conf/logger.xml

<configuration>

  <conversionRule conversionWord="coloredLevel" converterClass="play.api.Logger$ColoredLevel" />

  <appender name="FILE" class="ch.qos.logback.core.rolling.RollingFileAppender">
    <file>/data01/alpha/cmak/logs/application.log</file>
    <encoder>
       <pattern>%date - [%level] - from %logger in %thread %n%message%n%xException%n</pattern>
    </encoder>

    <rollingPolicy class="ch.qos.logback.core.rolling.TimeBasedRollingPolicy">
      <fileNamePattern>/data01/alpha/cmak/logs/application.%d{yyyy-MM-dd}.log</fileNamePattern>
      <maxHistory>3</maxHistory>
      <totalSizeCap>100MB</totalSizeCap>
    </rollingPolicy>
  </appender>

  <logger name="play" level="OFF" />
  <logger name="application" level="OFF" />

  <!-- Off these ones as they are annoying, and anyway we manage configuration ourself -->
  <logger name="com.avaje.ebean.config.PropertyMapLoader" level="OFF" />
  <logger name="com.avaje.ebeaninternal.server.core.XmlConfigLoader" level="OFF" />
  <logger name="com.avaje.ebeaninternal.server.lib.BackgroundThread" level="OFF" />
  <logger name="com.gargoylesoftware.htmlunit.javascript" level="OFF" />
  <logger name="org.apache.zookeeper.ClientCnxn" level="OFF" />
  <logger name="org.apache.zookeeper" level="OFF"/>
  <logger name="org.apache.curator.ConnectionState" level="OFF"/>
  <logger name="kafka.manager.KafkaManager" level="OFF"/>

  <logger name="akka" level="OFF" />
  <logger name="kafka" level="OFF" />

  <root level="OFF">
    <appender-ref ref="FILE" />
  </root>

</configuration>

【=================== logback.xml ==========================】
$ vim conf/logback.xml
<!--
  ~ Copyright (C) 2009-2015 Typesafe Inc. <http://www.typesafe.com>
  -->
<!-- The default logback configuration that Play uses if no other configuration is provided -->
<configuration>

    <appender name="FILE" class="ch.qos.logback.core.rolling.RollingFileAppender">
        <file>/data01/alpha/cmak/logs/application.log</file>
        <encoder>
           <pattern>%date - [%level] - from %logger in %thread %n%message%n%xException%n</pattern>
        </encoder>

        <rollingPolicy class="ch.qos.logback.core.rolling.TimeBasedRollingPolicy">
            <fileNamePattern>/data01/alpha/cmak/logs/application.%d{yyyy-MM-dd}.log</fileNamePattern>
            <maxHistory>3</maxHistory>
            <totalSizeCap>100MB</totalSizeCap>
        </rollingPolicy>
    </appender>

    <appender name="STDOUT" class="ch.qos.logback.core.ConsoleAppender">
        <encoder>
            <pattern>%date - [%level] %logger{15} - %message%n%xException{10}</pattern>
        </encoder>
    </appender>

    <appender name="ASYNCFILE" class="ch.qos.logback.classic.AsyncAppender">
        <appender-ref ref="FILE" />
    </appender>

    <appender name="ASYNCSTDOUT" class="ch.qos.logback.classic.AsyncAppender">
        <appender-ref ref="STDOUT" />
    </appender>

    <logger name="play" level="OFF" />
    <logger name="application" level="OFF" />
    <logger name="kafka.manager" level="OFF" />

    <!-- Off these ones as they are annoying, and anyway we manage configuration ourself -->
    <logger name="com.avaje.ebean.config.PropertyMapLoader" level="OFF" />
    <logger name="com.avaje.ebeaninternal.server.core.XmlConfigLoader" level="OFF" />
    <logger name="com.avaje.ebeaninternal.server.lib.BackgroundThread" level="OFF" />
    <logger name="com.gargoylesoftware.htmlunit.javascript" level="OFF" />
    <logger name="org.apache.zookeeper" level="OFF"/>
    <logger name="org.apache.curator.ConnectionState" level="OFF"/>

    <root level="OFF">
        <appender-ref ref="ASYNCFILE" />
        <appender-ref ref="ASYNCSTDOUT" />
    </root>

</configuration>

存储目录

【创建日志目录】

$ mkdir -p /data01/alpha/cmak/logs

服务管理

【启动脚本】

$ vim cmak/start.sh
#!/bin/bash
if [ -f RUNNING_PID ];then echo -e "Delete RUNNING_PID !\n";rm RUNNING_PID;fi
/home/alpha/cmak/bin/cmak -Dconfig.file=/home/alpha/cmak/conf/application.conf -java-home /home/alpha/cmak/java11 -Dhttp.port=9000 2>&1 >/dev/null &

【停止脚本】

$ cat cmak/stop.sh
#!/bin/bash
jps|grep -w 'ProdServerStart'|awk '{print $1}'|xargs kill -9
if [ -f RUNNING_PID ];then echo -e "Delete RUNNING_PID !\n";rm RUNNING_PID;fi

启动服务

【启动】
$ cd ~/cmak
$ ./start.sh

【检查】
$ jps
1234 ProdServerStart

登陆页面

【web控制台】
http://10.66.40.97:9000

【用户名/密码】
admin / Jet#2dg%S@A

新建集群

【WEB创建集群】

配置项参数
Cluster Namelnyd-1
Cluster Zookeeper Hosts10.66.40.97:3181,10.66.40.98:3181,10.66.40.99:3181
Kafka Version0.10.1.0
勾选Enable JMX Polling
勾选Poll consumer information
勾选Filter out inactive consumers
勾选Enable Active OffsetCache
勾选Display Broker and Topic Size

【注意】勾选“Enable JMX Polling” 后,kafka的启动脚本 “kafka-server-start.sh” 需要增加 “export JMX_PORT=9099”

# vim kafka/bin/kafka-server-start.sh
if [ "x$KAFKA_HEAP_OPTS" = "x" ]; then
    export KAFKA_HEAP_OPTS="-Xmx4G -Xms4G"
    export JMX_PORT=9099
fi

krb 认证

【配置】
# vim conf/application.conf

【kerberos】
-Djava.security.auth.login.config=/etc/ksecurity/my-jaas.conf
 类似资料: