kafka 8和内存——没有足够的内存供Java运行时环境继续运行

Question

我正在使用具有512兆内存的Digi海洋实例，我得到以下kafka错误。我不是一个熟练的java开发人员。我如何调整kafka以利用少量的ram。这是一个开发服务器。我不想为更大的机器支付每小时更多的费用。

#
# There is insufficient memory for the Java Runtime Environment to continue.
# Native memory allocation (malloc) failed to allocate 986513408 bytes for committing reserved memory.
# An error report file with more information is saved as:
# //hs_err_pid6500.log
OpenJDK 64-Bit Server VM warning: INFO: os::commit_memory(0x00000000bad30000, 986513408, 0) failed; error='Cannot allocate memory' (errno=12)

Answer 1

添加环境变量JAVA_OPTS。

Linux，编辑~/. bashrc文件，并更好地获取它。

export JAVA_OPTS="$JAVA_OPTS -Xms128M -Xmx128M"

在寡妇中，添加系统变量

JAVA_OPTS "-Xms128M -Xmx128M"

然后重新启动。

如果它不起作用，那么应该更改许多文件，例如bin/kafka-server-start.sh、config/server.properties。减少这些文件中的数字。

Answer 2

区域：热点/气相色谱

概要

Crashes due to failure to allocate large pages.

On Linux, failures when allocating large pages can lead to crashes. When running JDK 7u51 or later versions, the issue can be recognized in two ways:

    Before the crash happens, one or more lines similar to the following example will have been printed to the log:

    os::commit_memory(0x00000006b1600000, 352321536, 2097152, 0) failed;
    error='Cannot allocate memory' (errno=12); Cannot allocate large pages, 
    falling back to regular pages

    If a file named hs_err is generated, it will contain a line similar to the following example:

    Large page allocation failures have occurred 3 times

The problem can be avoided by running with large page support turned off, for example, by passing the "-XX:-UseLargePages" option to the java binary.

Answer 3

您可以通过编辑 kafka-server-start.sh、zookeeper-server-start.sh等来调整 JVM 堆大小：

export KAFKA_HEAP_OPTS="-Xmx1G -Xms1G"

< code>-Xms参数指定最小堆大小。要让您的服务器至少能够启动，请尝试将其更改为使用较少的内存。假设您只有512M，您还应该更改最大堆大小(< code>-Xmx ):

export KAFKA_HEAP_OPTS="-Xmx256M -Xms128M"

我不确定默认配置中 kafka 的最低内存要求是什么 - 也许您需要调整 kafka 中的消息大小才能使其运行。