How to fix garbled characters received in Kafka Audit Log Listener

Varun_Singh — Wed, 10 Jul 2019 10:30:32 GMT

We have setup Kafka as audit log listener and we are seeing the logs. Following is the way we have configured

Wrote extension point with target: org.nuxeo.runtime.stream.kafka.service point: kafkaConfig

<?xml version="1.0"?>
<component name="com.softcell.kafka.contrib.kafkacontrib">

  <extension target="org.nuxeo.runtime.stream.kafka.service" point="kafkaConfig">
    <kafkaConfig name="default" topicPrefix="nuxeo-">
      <producer>
        <property name="bootstrap.servers">localhost:9092</property>
      </producer>
      <consumer>
        <property name="bootstrap.servers">localhost:9092</property>
        <property name="request.timeout.ms">65000</property>
        <property name="max.poll.interval.ms">60000</property>
        <property name="session.timeout.ms">20000</property>
        <property name="heartbeat.interval.ms">1000</property>
        <property name="max.poll.records">50</property>
      </consumer>
    </kafkaConfig>
  </extension>
</component>

Added following config to this contributions nuxeo.defaults:

#kafkacontrib.target=
kafka.enabled=true
nuxeo.template.includes=common
nuxeo.stream.audit.enabled=true
nuxeo.stream.audit.log.config=audit
nuxeo.stream.audit.batch.size=25
nuxeo.stream.audit.batch.threshold.ms=500

After doing this, I start listening to topic nuxeo-audit using consoleConsumer using this command: bin/kafka-console-consumer.sh --bootstrap-server localhost:9092 --topic nuxeo-audit

I can see the logs getting generated. However there are garbled characters in them. Pasting a sample below:

I can see in the source-code that BytesDeserializer is being used for value but I am not sure why am I seeing garbled characters. Any help in this would be deeply appreciated. Thanks

Re: How to fix garbled characters received in Kafka Audit Log Listener

LaraGranite_ — Wed, 10 Jul 2019 19:16:43 GMT

I think you need to add this to nuxeo.conf

Re: How to fix garbled characters received in Kafka Audit Log Listener

Varun_Singh — Fri, 12 Jul 2019 05:08:09 GMT

Thanks for your inputs LaraGranite. I added the codec config like you mentioned but didn't see any changes. I can see that there are a few more codecs available. I will try them and will post the findings here.

topic How to fix garbled characters received in Kafka Audit Log Listener in Nuxeo Forum

How to fix garbled characters received in Kafka Audit Log Listener

Re: How to fix garbled characters received in Kafka Audit Log Listener

Re: How to fix garbled characters received in Kafka Audit Log Listener