15 五月 2017 17:17:09,814 WARN [PollableSourceRunner-TaildirSource-r-op_hiveserver] (org.apache.flume.channel.kafka.KafkaChannel$KafkaTransaction.doCommit:560) - Sending events to Kafka failed java.util.concurrent.ExecutionException: org.apache.kafka.common.errors.RecordTooLargeException: The message is 1730741 bytes when serialized which is larger than the maximum request size you have configured with the max.request.size configuration. at org.apache.kafka.clients.producer.KafkaProducer$FutureFailure.<init>(KafkaProducer.java:686) at org.apache.kafka.clients.producer.KafkaProducer.send(KafkaProducer.java:449) at org.apache.flume.channel.kafka.KafkaChannel$KafkaTransaction.doCommit(KafkaChannel.java:546) at org.apache.flume.channel.BasicTransactionSemantics.commit(BasicTransactionSemantics.java:151) at org.apache.flume.channel.ChannelProcessor.processEventBatch(ChannelProcessor.java:194) at org.apache.flume.source.taildir.TaildirSource.tailFileProcess(TaildirSource.java:263) at org.apache.flume.source.taildir.TaildirSource.process(TaildirSource.java:226) at org.apache.flume.source.PollableSourceRunner$PollingRunner.run(PollableSourceRunner.java:133) at java.lang.Thread.run(Thread.java:745) Caused by: org.apache.kafka.common.errors.RecordTooLargeException: The message is 1730741 bytes when serialized which is larger than the maximum request size you have configured with the max.request.size configuration.
15 五月 2017 17:20:00,799 WARN [PollableSourceRunner-TaildirSource-r-op_hiveserver] (org.apache.flume.channel.kafka.KafkaChannel$KafkaTransaction.doCommit:560) - Sending events to Kafka failed java.util.concurrent.ExecutionException: org.apache.kafka.common.errors.RecordTooLargeException: The request included a message larger than the max message size the server will accept. at org.apache.kafka.clients.producer.internals.FutureRecordMetadata.valueOrError(FutureRecordMetadata.java:56) at org.apache.kafka.clients.producer.internals.FutureRecordMetadata.get(FutureRecordMetadata.java:43) at org.apache.kafka.clients.producer.internals.FutureRecordMetadata.get(FutureRecordMetadata.java:25) at org.apache.flume.channel.kafka.KafkaChannel$KafkaTransaction.doCommit(KafkaChannel.java:552) at org.apache.flume.channel.BasicTransactionSemantics.commit(BasicTransactionSemantics.java:151) at org.apache.flume.channel.ChannelProcessor.processEventBatch(ChannelProcessor.java:194) at org.apache.flume.source.taildir.TaildirSource.tailFileProcess(TaildirSource.java:263) at org.apache.flume.source.taildir.TaildirSource.process(TaildirSource.java:226) at org.apache.flume.source.PollableSourceRunner$PollingRunner.run(PollableSourceRunner.java:133) at java.lang.Thread.run(Thread.java:745) Caused by: org.apache.kafka.common.errors.RecordTooLargeException: The request included a message larger than the max message size the server will accept.
wechat