When/how does a topic “marked for deletion” get fi

Question

I have issued the command to delete a topic:

./bin/kafka-topics.sh --zookeeper localhost:2181 --delete --topic  vip_ips_alerts

It seemed to give a happy response:

[2014-05-31 20:58:10,112] INFO zookeeper state changed (SyncConnected) (org.I0Itec.zkclient.ZkClient)
Topic "vip_ips_alerts" queued for deletion.

But now 10 minutes later the topic still appears in the --list command:

./bin/kafka-topics.sh --zookeeper localhost:2181 --list
vip_ips_alerts - marked for deletion

So what does that mean? When will the topic be really deleted? How do I expedite this process?

Answer 1

The deletion happened for me almost immediately (v 0.9). I believe that it should be the same for you.

Once marked, the deletion will be triggered on the kafka node which is the topic partition leader. The thing to remember is that the the topic partition leader must be configured correctly (in terms of listeners), otherwise it will not receive the instruction to delete the logs.

I had my "marked for deletion" topics stuck at that state until I corrected my server properties and restarted the respective node.

Answer 2

tl;dr Set delete.topic.enable = true in config/server.properties of Kafka brokers and...be patient.

It happens with the latest development version of Kafka 0.8.3-SNAPSHOT:

➜  kafka_2.11-0.8.3-SNAPSHOT git:(trunk) ✗ ./bin/kafka-topics.sh --zookeeper localhost:2181 --create --topic my-topic --partitions 2 --replication-factor 1
Created topic "my-topic".

➜  kafka_2.11-0.8.3-SNAPSHOT git:(trunk) ✗ ./bin/kafka-topics.sh --zookeeper localhost:2181 --describe --topic my-topic
Topic:my-topic  PartitionCount:2    ReplicationFactor:1 Configs:
    Topic: my-topic Partition: 0    Leader: 0   Replicas: 0 Isr: 0
    Topic: my-topic Partition: 1    Leader: 0   Replicas: 0 Isr: 0

➜  kafka_2.11-0.8.3-SNAPSHOT git:(trunk) ✗ ./bin/kafka-topics.sh --zookeeper localhost:2181 --delete --topic my-topic
Topic my-topic is marked for deletion.
Note: This will have no impact if delete.topic.enable is not set to true.

➜  kafka_2.11-0.8.3-SNAPSHOT git:(trunk) ✗ ./bin/kafka-topics.sh --zookeeper localhost:2181 --list
➜  kafka_2.11-0.8.3-SNAPSHOT git:(trunk) ✗

The point is to have delete.topic.enable=true in config/server.properties that you use to start a Kafka broker.

➜  kafka_2.11-0.8.3-SNAPSHOT git:(trunk) ✗ grep delete.topic.enable config/server.properties
delete.topic.enable=true

You can also ensure the setting be true in a broker's log:

➜  kafka_2.11-0.8.3-SNAPSHOT git:(trunk) ✗ ./bin/kafka-server-start.sh config/server.properties
[2015-07-24 22:33:26,184] INFO KafkaConfig values:
        ...
        delete.topic.enable = true

Answer 3

In my case where i am using Kafka 8.2.2, I had to delete entries from the following manually -

1. Delete topic folder from Kafka broker machine.

2. Login to zookeeper and -

   hbase zkcli
   rmr /brokers/topics/{topic_name}
   rmr /admin/delete_topics/{topic_name}
   

Answer 4

For kafka version 0.10.0.0 it is enough to enable the topic deletion by setting:

delete.topic.enable

The topic is deleted within few moments. No restart needed.

kafka-topics --delete --zookeeper your-zk:2181 --topic yourTopicName

You can confirm that it is gone with the command:

kafka-topics --describe --zookeeper your-zk:2181 --topic yourTopicName

Answer 5

We had this issue when deleting topics. The topics had been created when delete.topic.enable=true had not been set. We set that in config, restarted kafka to apply the new config. Deleted the topics and saw the "marked for deletion". We then restarted kafka again. After 40 minutes though all topics had been deleted (9 topics with total partitions in the thousands). The topics with higher numbers of partitions seemed to take longer, which initially made it look like nothing was happening.

Answer 6

For kafka version 0.10.0.0 it is enough to enable the topic deletion by setting:

delete.topic.enable

The topic is deleted within few moments

kafka-topics --delete --zookeeper your-zk:2181 --topic yourTopicName

You can confirm that it is gone with the following command:

kafka-topics --describe --zookeeper your-zk:2181 --topic yourTopicName