OutOfMemoryError错误:请求的数组大小超过了apache camel中的虚拟机限制

Question

在我的应用程序中，正在使用 apache 驼峰处理 txt 文件，应用程序使用 exchange 对象将整个文件读入字符串

我的应用程序支持16GB堆空间。处理前的文件大小为685MB

from("file:<filepath>/processed/INPUT?filter=#fileFilter&delay=120000&exclusiveReadLockStrategy=#fileReadLockStrategy&delete=true&autoCreate=true")
                    .threads(10, 10)
                    .process(fileProcessor).log("Came out of file processor")
                    .recipientList(header(DESTIONATION_PATH));

这是一个从输入文件夹中读取文件的简单路由

Error has occurred: 
org.apache.camel.CamelExecutionException: Exception occurred during execution on the exchange: Exchange[ID-x01btadmdts3a-vsi-uat-dbs-com-1586519346908-0-4]
    at org.apache.camel.util.ObjectHelper.wrapCamelExecutionException(ObjectHelper.java:1842) ~[camel-core-2.24.1.jar!/:2.24.1]
    at org.apache.camel.impl.DefaultExchange.setException(DefaultExchange.java:385) ~[camel-core-2.24.1.jar!/:2.24.1]
    at org.apache.camel.util.AsyncProcessorConverterHelper$ProcessorToAsyncProcessorBridge.process(AsyncProcessorConverterHelper.java:64) ~[camel-core-2.24.1.jar!/:2.24.1]
    at org.apache.camel.processor.RedeliveryErrorHandler.process(RedeliveryErrorHandler.java:548) [camel-core-2.24.1.jar!/:2.24.1]
    at org.apache.camel.processor.CamelInternalProcessor.process(CamelInternalProcessor.java:201) [camel-core-2.24.1.jar!/:2.24.1]
    at org.apache.camel.processor.MulticastProcessor.doProcessSequential(MulticastProcessor.java:715) [camel-core-2.24.1.jar!/:2.24.1]
    at org.apache.camel.processor.MulticastProcessor.doProcessSequential(MulticastProcessor.java:638) [camel-core-2.24.1.jar!/:2.24.1]
    at org.apache.camel.processor.MulticastProcessor.process(MulticastProcessor.java:248) [camel-core-2.24.1.jar!/:2.24.1]
    at org.apache.camel.processor.RecipientList.sendToRecipientList(RecipientList.java:173) [camel-core-2.24.1.jar!/:2.24.1]
    at org.apache.camel.processor.RecipientList.process(RecipientList.java:133) [camel-core-2.24.1.jar!/:2.24.1]
    at org.apache.camel.processor.Pipeline.process(Pipeline.java:138) [camel-core-2.24.1.jar!/:2.24.1]
    at org.apache.camel.processor.Pipeline.process(Pipeline.java:101) [camel-core-2.24.1.jar!/:2.24.1]
    at org.apache.camel.management.InstrumentationProcessor.process(InstrumentationProcessor.java:76) [camel-core-2.24.1.jar!/:2.24.1]
    at org.apache.camel.processor.RedeliveryErrorHandler.process(RedeliveryErrorHandler.java:548) [camel-core-2.24.1.jar!/:2.24.1]
    at org.apache.camel.processor.CamelInternalProcessor.process(CamelInternalProcessor.java:201) [camel-core-2.24.1.jar!/:2.24.1]
    at org.apache.camel.processor.Pipeline.process(Pipeline.java:138) [camel-core-2.24.1.jar!/:2.24.1]
    at org.apache.camel.processor.Pipeline.process(Pipeline.java:101) [camel-core-2.24.1.jar!/:2.24.1]
    at org.apache.camel.processor.Pipeline.process(Pipeline.java:138) [camel-core-2.24.1.jar!/:2.24.1]
    at org.apache.camel.processor.Pipeline.access$100(Pipeline.java:43) [camel-core-2.24.1.jar!/:2.24.1]
    at org.apache.camel.processor.Pipeline$1.done(Pipeline.java:157) [camel-core-2.24.1.jar!/:2.24.1]
    at org.apache.camel.processor.ThreadsProcessor$ProcessCall.run(ThreadsProcessor.java:87) [camel-core-2.24.1.jar!/:2.24.1]
    at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511) [na:1.8.0_171]
    at java.util.concurrent.FutureTask.run(FutureTask.java:266) [na:1.8.0_171]
    at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) [na:1.8.0_171]
    at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) [na:1.8.0_171]
    at java.lang.Thread.run(Thread.java:748) [na:1.8.0_171]
    Suppressed: org.apache.camel.CamelExecutionException: Exception occurred during execution on the exchange: Exchange[ID-x01btadmdts3a-vsi-uat-dbs-com-1586519346908-0-4]
        ... 26 common frames omitted
    ***Caused by: java.lang.OutOfMemoryError: Requested array size exceeds VM limit***
        at java.lang.StringCoding$StringEncoder.encode(StringCoding.java:300)
        at java.lang.StringCoding.encode(StringCoding.java:344)
        at java.lang.String.getBytes(String.java:918)
        at org.apache.camel.converter.IOConverter.toInputStream(IOConverter.java:198)
        at org.apache.camel.converter.IOConverterOptimised.convertTo(IOConverterOptimised.java:55)
        at org.apache.camel.impl.converter.OptimisedTypeConverter.convertTo(OptimisedTypeConverter.java:63)
        at org.apache.camel.impl.converter.BaseTypeConverterRegistry.doConvertTo(BaseTypeConverterRegistry.java:299)
        at org.apache.camel.impl.converter.BaseTypeConverterRegistry.mandatoryConvertTo(BaseTypeConverterRegistry.java:188)
        at org.apache.camel.impl.MessageSupport.getMandatoryBody(MessageSupport.java:115)
        at org.apache.camel.component.file.FileOperations.storeFile(FileOperations.java:333)
        at org.apache.camel.component.file.GenericFileProducer.writeFile(GenericFileProducer.java:305)
        at org.apache.camel.component.file.GenericFileProducer.processExchange(GenericFileProducer.java:169)
        at org.apache.camel.component.file.GenericFileProducer.process(GenericFileProducer.java:80)
        at org.apache.camel.util.AsyncProcessorConverterHelper$ProcessorToAsyncProcessorBridge.process(AsyncProcessorConverterHelper.java:61)
        at org.apache.camel.processor.RedeliveryErrorHandler.process(RedeliveryErrorHandler.java:548)
        at org.apache.camel.processor.CamelInternalProcessor.process(CamelInternalProcessor.java:201)
        at org.apache.camel.processor.MulticastProcessor.doProcessSequential(MulticastProcessor.java:715)
        at org.apache.camel.processor.MulticastProcessor.doProcessSequential(MulticastProcessor.java:638)
        at org.apache.camel.processor.MulticastProcessor.process(MulticastProcessor.java:248)
        at org.apache.camel.processor.RecipientList.sendToRecipientList(RecipientList.java:173)
        at org.apache.camel.processor.RecipientList.process(RecipientList.java:133)
        at org.apache.camel.processor.Pipeline.process(Pipeline.java:138)
        at org.apache.camel.processor.Pipeline.process(Pipeline.java:101)
        at org.apache.camel.management.InstrumentationProcessor.process(InstrumentationProcessor.java:76)
        at org.apache.camel.processor.RedeliveryErrorHandler.process(RedeliveryErrorHandler.java:548)
        at org.apache.camel.processor.CamelInternalProcessor.process(CamelInternalProcessor.java:201)
        at org.apache.camel.processor.Pipeline.process(Pipeline.java:138)
        at org.apache.camel.processor.Pipeline.process(Pipeline.java:101)
        at org.apache.camel.processor.Pipeline.process(Pipeline.java:138)
        at org.apache.camel.processor.Pipeline.access$100(Pipeline.java:43)
        at org.apache.camel.processor.Pipeline$1.done(Pipeline.java:157)
        at org.apache.camel.processor.ThreadsProcessor$ProcessCall.run(ThreadsProcessor.java:87)
Caused by: java.lang.OutOfMemoryError: Requested array size exceeds VM limit
    at java.lang.StringCoding$StringEncoder.encode(StringCoding.java:300) ~[na:1.8.0_171]
    at java.lang.StringCoding.encode(StringCoding.java:344) ~[na:1.8.0_171]
    at java.lang.String.getBytes(String.java:918) ~[na:1.8.0_171]
    at org.apache.camel.converter.IOConverter.toInputStream(IOConverter.java:198) ~[camel-core-2.24.1.jar!/:2.24.1]
    at org.apache.camel.converter.IOConverterOptimised.convertTo(IOConverterOptimised.java:55) ~[camel-core-2.24.1.jar!/:2.24.1]
    at org.apache.camel.impl.converter.OptimisedTypeConverter.convertTo(OptimisedTypeConverter.java:63) ~[camel-core-2.24.1.jar!/:2.24.1]
    at org.apache.camel.impl.converter.BaseTypeConverterRegistry.doConvertTo(BaseTypeConverterRegistry.java:299) ~[camel-core-2.24.1.jar!/:2.24.1]
    at org.apache.camel.impl.converter.BaseTypeConverterRegistry.mandatoryConvertTo(BaseTypeConverterRegistry.java:188) ~[camel-core-2.24.1.jar!/:2.24.1]
    at org.apache.camel.impl.MessageSupport.getMandatoryBody(MessageSupport.java:115) ~[camel-core-2.24.1.jar!/:2.24.1]
    at org.apache.camel.component.file.FileOperations.storeFile(FileOperations.java:333) ~[camel-core-2.24.1.jar!/:2.24.1]
    at org.apache.camel.component.file.GenericFileProducer.writeFile(GenericFileProducer.java:305) ~[camel-core-2.24.1.jar!/:2.24.1]
    at org.apache.camel.component.file.GenericFileProducer.processExchange(GenericFileProducer.java:169) ~[camel-core-2.24.1.jar!/:2.24.1]
    at org.apache.camel.component.file.GenericFileProducer.process(GenericFileProducer.java:80) ~[camel-core-2.24.1.jar!/:2.24.1]
    at org.apache.camel.util.AsyncProcessorConverterHelper$ProcessorToAsyncProcessorBridge.process(AsyncProcessorConverterHelper.java:61) ~[camel-core-2.24.1.jar!/:2.24.1]
    at org.apache.camel.processor.RedeliveryErrorHandler.process(RedeliveryErrorHandler.java:548) [camel-core-2.24.1.jar!/:2.24.1]
    at org.apache.camel.processor.CamelInternalProcessor.process(CamelInternalProcessor.java:201) [camel-core-2.24.1.jar!/:2.24.1]
    at org.apache.camel.processor.MulticastProcessor.doProcessSequential(MulticastProcessor.java:715) [camel-core-2.24.1.jar!/:2.24.1]
    at org.apache.camel.processor.MulticastProcessor.doProcessSequential(MulticastProcessor.java:638) [camel-core-2.24.1.jar!/:2.24.1]
    at org.apache.camel.processor.MulticastProcessor.process(MulticastProcessor.java:248) [camel-core-2.24.1.jar!/:2.24.1]
    at org.apache.camel.processor.RecipientList.sendToRecipientList(RecipientList.java:173) [camel-core-2.24.1.jar!/:2.24.1]
    at org.apache.camel.processor.RecipientList.process(RecipientList.java:133) [camel-core-2.24.1.jar!/:2.24.1]
    at org.apache.camel.processor.Pipeline.process(Pipeline.java:138) [camel-core-2.24.1.jar!/:2.24.1]
    at org.apache.camel.processor.Pipeline.process(Pipeline.java:101) [camel-core-2.24.1.jar!/:2.24.1]
    at org.apache.camel.management.InstrumentationProcessor.process(InstrumentationProcessor.java:76) [camel-core-2.24.1.jar!/:2.24.1]
    at org.apache.camel.processor.RedeliveryErrorHandler.process(RedeliveryErrorHandler.java:548) [camel-core-2.24.1.jar!/:2.24.1]
    at org.apache.camel.processor.CamelInternalProcessor.process(CamelInternalProcessor.java:201) [camel-core-2.24.1.jar!/:2.24.1]
    at org.apache.camel.processor.Pipeline.process(Pipeline.java:138) [camel-core-2.24.1.jar!/:2.24.1]
    at org.apache.camel.processor.Pipeline.process(Pipeline.java:101) [camel-core-2.24.1.jar!/:2.24.1]
    at org.apache.camel.processor.Pipeline.process(Pipeline.java:138) [camel-core-2.24.1.jar!/:2.24.1]
    at org.apache.camel.processor.Pipeline.access$100(Pipeline.java:43) [camel-core-2.24.1.jar!/:2.24.1]
    at org.apache.camel.processor.Pipeline$1.done(Pipeline.java:157) [camel-core-2.24.1.jar!/:2.24.1]
    at org.apache.camel.processor.ThreadsProcessor$ProcessCall.run(ThreadsProcessor.java:87) [camel-core-2.24.1.jar!/:2.24.1]

Answer 1

抱歉，在这里回答晚了。我能够解决上述问题。当我仔细查看日志时，它在标准java类“StringCoding.java”中失败了。

我的代码中的错误在两行以下:

String content = exchange.getIn().getBody(String.class); 
//Add deleimters in each line content string & write back the whole string in file.
//derive new content string from original content
exchange.getOut().setBody(content);// Camel stores content in String & sends it via route using character array. 

分析：

Java将字符串作为字符数组存储在后台。按照逻辑，我们在每一行中添加了一些分隔符。我发现(原始文件中的字符数和添加的分隔符数)是

解决方案：

File contentFile = exchange.getIn().getBody(File.class);
//Process the contentFile as file line by line & write the records back in file.
exchange.getOut().setBody(contentFile);// Camel stores content in File & sends it via route using InputStream. 

我遵守了以上准则

感谢您的阅读。快乐编码…！！！