消费kafka数据出错Found a message larger than the maximum fetch size

具体错误如下:
ERROR [2017-01-05 02:59:26,534] com.elasticsearch.river.kafka.KafkaRiver: Unexpected Error Occurred
! kafka.common.MessageSizeTooLargeException: Found a message larger than the maximum fetch size of this consumer on topic mobileTopic partition 0 at fetch offset 2563424. Increase the fetch size, or decrease the maximum message size the broker will allow.
! at kafka.consumer.ConsumerIterator.makeNext(ConsumerIterator.scala:91) ~[kafka-consumer-for-hdfs.jar:na]
! at kafka.consumer.ConsumerIterator.makeNext(ConsumerIterator.scala:33) ~[kafka-consumer-for-hdfs.jar:na]
! at kafka.utils.IteratorTemplate.maybeComputeNext(IteratorTemplate.scala:66) ~[kafka-consumer-for-hdfs.jar:na]
! at kafka.utils.IteratorTemplate.hasNext(IteratorTemplate.scala:58) ~[kafka-consumer-for-hdfs.jar:na]
! at com.elasticsearch.river.kafka.KafkaRiver$KafkaRiverFetcher.run(KafkaRiver.java:203) ~[original-kafka-consumer-for-hdfs.jar:1.1.5]
! at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) [na:1.7.0_51]
! at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) [na:1.7.0_51]
! at java.lang.Thread.run(Thread.java:744) [na:1.7.0_51]
已邀请:

采菊篱下 - 无分享不快乐 助力技术开源分享 运维一枚

赞同来自:

这个就是你kafka中有一条消息大于你能获取的大小,要不增加客户端获取消息大小,增大:
fetch.message.max.bytes
的值,要不就是在kafka broker那控制只允许多大的数据进入。

要回复问题请先登录注册