Kafka integration in unity3d throwing Win32Excepti

Question

I am trying to run a code sample of Kafka in unity environment and for this reason, I created a consumer client (Code given below).

using System.Collections;
using System.Collections.Generic;
using UnityEngine;
using Confluent.Kafka;
using Confluent.Kafka.Serialization;
using System.Text;

public class KafkaConsumer : MonoBehaviour 
{
    // Use this for initialization
    void Start () 
    {
        /*
         * The consumer application will then pick the messages from the same topic and write them to console output. 
         * The process to create the consumer application is also very simple.
         */
        var config = new Dictionary<string, object>
        {
            { "group.id","JavaInUseGroup" },
            { "bootstrap.servers", "localhost:9092" },
            { "enable.auto.commit", "false" }
        };

        using (var consumer = new Consumer<Null, string>(config, null, new StringDeserializer(Encoding.UTF8)))
        {
            consumer.Subscribe(new string[] { "javainuse-topic" });

            consumer.OnMessage += (_, msg) =>
            {
                //Console.WriteLine($"Topic: {msg.Topic} Partition: {msg.Partition} Offset :{msg.Offset} {msg.Value}");
                Debug.Log($"Topic: {msg.Topic} Partition: {msg.Partition} Offset :{msg.Offset} {msg.Value}");

                consumer.CommitAsync(msg);
            };

            while (true)
            {
                consumer.Poll(100);
            }
        }
    }
}

In order to execute the above code sample I have also added confluent.Kafka dll into my project asset folder. but whenever I run my unity game it throws an error:Win32Exception: The specified module could not be found.

Rethrow as InvalidOperationException: Error while loading librdkafka.dll or its dependencies from Assets/librdkafka.dll. Check the directory exists, if not check your deployment process. You can also load the library and its dependencies by yourself before any call to Confluent.Kafka Confluent.Kafka.Impl.LibRdKafka.Initialize (System.String userSpecifiedPath) (at <700d5bbe3b974ce5aed001c82b789f6a>:0) Confluent.Kafka.Consumer..ctor (System.Collections.Generic.IEnumerable1[T] config) (at <700d5bbe3b974ce5aed001c82b789f6a>:0) Confluent.Kafka.Consumer2[TKey,TValue]..ctor (System.Collections.Generic.IEnumerable1[T] config, Confluent.Kafka.Serialization.IDeserializer1[T] keyDeserializer, Confluent.Kafka.Serialization.IDeserializer`1[T] valueDeserializer) (at <700d5bbe3b974ce5aed001c82b789f6a>:0) KafkaConsumer.Start () (at Assets/KafkaConsumer.cs:26)

As the error states that there is dependancy problem so i have also copied these dll in assets/librdkafka/x64 folder

1. librdkafka

2. librdkafkacpp

3. msvcr120

4. zlib

Now the problem is, my project get stuck whenever i try to play it.

Remember: I have downloaded all these dll through nuget in vs 2017. Then i bring these dll into unity.

Answer 1

For future user, here is the process to add Kafka in your Unity3d Project: Actually there is a specific order or folder hierarchy to add dll in your project. (I didn't find any authoritative reference about this if someone found then please share)

1. First paste Confluen.Kafka dll in whatever your folder

2. Then Make librdkafka folder(Make sure it is created next to confluent.kafka dll) and paste your platform related dll in x64 or x86 Folder

Now, you can run my code sample(mentioned in question).

IMP NOTE: After building the player, you have to manually copy the dll files in your Player/Managed/librdkafka folder. You have to create librdkafka folder in managed folder, then paste your dlls.(Again i don't know why it require but if someone found authoritative refrence then share)