Qn: If i've diff kerberos Realms, and the broker sits on Linux and producer sits on windows, how do enable the connectivity using Kerberos? I have valid keytab. and here is the krb5
Please see marked answer to this question in this link.
Connect to Kafka on Unix from Windows with Kerberos
below question is continuation for 3rd scenario explained by @Samson. answering some of Samson's suggestions, 1 default realm is added in krb5. 2.there is one way trust. the broker domain trusts my domain.
[libdefaults]
renew_lifetime = 7d
forwardable = false
default_realm = SomeUrl.COM
ticket_lifetime = 24h
dns_lookup_realm = false
dns_lookup_kdc = false
udp_preference_limit = 1
[domain_realm]
.machine.test.group = SomeUrl.COM
machine.test.group = SomeUrl.COM
[realms]
SomeUrl.COM = {
admin_server = SomeUrl.COM
kdc = SomeUrl.COM
}
SomeUrl.com is where the broker is placed.
here is the log from producer for this scenario.
producer connection to broker log: (I've edited the actual broker name and IP address)
7|2017-06-14 09:03:49.181|rdkafka#producer-1|BROKER| [thrd:app]:
sasl_plaintext: //some.machine.test.group:9092/bootstrap: Added new broker with NodeId -1 7|2017-06-14 09:03:49.180|rdkafka#producer-1|BRKMAIN| [thrd::0/internal]: :0/int ernal: Enter main broker thread 7|2017-06-14 09:03:49.227|rdkafka#producer-1|STATE| [thrd::0/internal]: :0/inter nal: Broker changed state INIT -> UP 7|2017-06-14 09:03:49.229|rdkafka#producer-1|BRKMAIN| [thrd:sasl_plaintext://some.machine.test.group:9092/bootstrap]: sasl_plaintext://some.machine. test.group:9092/bootstrap: Enter main broker thread 7|2017-06-14 09:03:49.230|rdkafka#producer-1|CONNECT| [thrd:sasl_plaintext://some.machine.test.group:9092/bootstrap]: sasl_plaintext://some.machine. test.group:9092/bootstrap: broker in state INIT connecting rdkafka#producer-1 producing on KAFKA_MM_L0. q to exit.
When I try to send msgs (
rdkafka#producer-1 producing on KAFKA_MM_L0. q to exit. 7|2017-06-14 09:04:33.625|rdkafka#producer-1|CONNECT| [thrd:sasl_plaintext://some.machine.test.group:9092/bootstrap]: sasl_plaintext://some.machine. test.group:9092/bootstrap: Connecting to ipv4#1.1.1.1:9092 (sasl_plaintext) with socket 184 7|2017-06-14 09:04:33.627|rdkafka#producer-1|STATE| [thrd:sasl_plaintext://some.machine.test.group:9092/bootstrap]: sasl_plaintext://some.machine.te st.group:9092/bootstrap: Broker changed state INIT -> CONNECT 7|2017-06-14 09:04:33.637|rdkafka#producer-1|CONNECT| [thrd:sasl_plaintext://some.machine.test.group:9092/bootstrap]: sasl_plaintext://some.machine. test.group:9092/bootstrap: Connected to ipv4#1.1.1.1:9092 7|2017-06-14 09:04:33.637|rdkafka#producer-1|CONNECTED| [thrd:sasl_plaintext://some.machine.test.group:9092/bootstrap]: sasl_plaintext://some.machin e.test.group:9092/bootstrap: Connected (#1) 7|2017-06-14 09:04:33.638|rdkafka#producer-1|APIVERSION| [thrd:sasl_plaintext://some.machine.test.group:9092/bootstrap]: sasl_plaintext://some.machi ne.test.group:9092/bootstrap: Using (configuration fallback) 0.9.0 protocol features 7|2017-06-14 09:04:33.640|rdkafka#producer-1|FEATURE| [thrd:sasl_plaintext://some.machine.test.group:9092/bootstrap]: sasl_plaintext://some.machine. test.group:9092/bootstrap: Updated enabled protocol features to BrokerBalancedCo nsumer,ThrottleTime,Sasl,BrokerGroupCoordinator,LZ4 7|2017-06-14 09:04:33.643|rdkafka#producer-1|AUTH| [thrd:sasl_plaintext://some.machine.test.group:9092/bootstrap]: sasl_plaintext://some.machine.tes t.group:9092/bootstrap: Auth in state CONNECT (handshake not supported) 7|2017-06-14 09:04:33.645|rdkafka#producer-1|STATE| [thrd:sasl_plaintext://some.machine.test.group:9092/bootstrap]: sasl_plaintext://some.machine.te st.group:9092/bootstrap: Broker changed state CONNECT -> AUTH 7|2017-06-14 09:04:33.646|rdkafka#producer-1|SASL| [thrd:sasl_plaintext://some.machine.test.group:9092/bootstrap]: sasl_plaintext://some.machine.test.group:9092/bootstrap: Initializing SASL client: service name kafka, hostname some.machine.test.group, mechanisms GSSAPI 7|2017-06-14 09:04:33.665|rdkafka#producer-1|SASL| [thrd:sasl_plaintext://some.machine.test.group:9092/bootstrap]: sasl_plaintext://some.machine.test.group:9092/bootstrap: Acquired Kerberos credentials handle (expiry in 2147483455.928712703s) 7|2017-06-14 09:04:33.676|rdkafka#producer-1|BROKERFAIL| [thrd:sasl_plaintext://some.machine.test.group:9092/bootstrap]: sasl_plaintext://some.machi ne.test.group:9092/bootstrap: failed: err: Local: Authentication failure: (errno: Invalid argument)
