Kubernetes service architecture

Question

Within the same kubernetes cluster,

(1) Can I have multiple StatefulSets attached to one headless service or should each StatefulSet have it's own headless service? What are the pros and cons of doing this?

(2) Can I mix standard and headless services in the same cluster? Specifically, I would like to use LoadBalancer service to load balance headless services. Can I define a service of type LoadBalancer and have headless services (ClusterIP = None) attached to it? If yes, how can I achieve this?

Here is my intended architecture:

Load Balancer Service
  - Headless Service (Database-service)
    -  MySql
    - BlazeGraph
  - Headless Service (Web / Tomcat)
    - Web Service (RESTful / GraphQL)

Any advice and insight is appreciated. Thanks!

Answer 1

Headless Service you should use in any case where you want to automatically discover all pods under the service as opposed to regular Service where you get ClusterIP instead. As an illustration from above mentioned example here is difference between DNS entries for Service (with ClusterIP) and Headless Service (without ClusterIP):

• Standard service you will get the clusterIP value:

  kubectl exec zookeeper-0 -- nslookup zookeeper
  Server:        10.0.0.10
  Address:    10.0.0.10#53
  
  Name:    zookeeper.default.svc.cluster.local
  Address: 10.0.0.213
  

• Headless service you will get IP of each pod

  kubectl exec zookeeper-0 -- nslookup zookeeper
  Server:        10.0.0.10
  Address:    10.0.0.10#53
  
  Name:    zookeeper.default.svc.cluster.local
  Address: 172.17.0.6
  Name:    zookeeper.default.svc.cluster.local
  Address: 172.17.0.7
  Name:    zookeeper.default.svc.cluster.local
  Address: 172.17.0.8
  

Now, If you connect two statefulset with single headless service, it will return the address of each pod in both the statefulset. There will be no way to differentiate the pods from two applications if you create two statefulset and one headless service for that. See the following article to understand why headless services are used

Headless service allow developer to reduce coupling from kubernetes system by allowing them to do discovery their own way. For such services, clusterIP is not allocated, kube-proxy doesn't handle these services and there is no load balancing and proxying done by platform for them. So, If you define clusterIP: None in your service there will be no load-balancing will be done from kubernetes end.

Hope this helps.

EDIT:

I did a little experiment to answer your queries, created two statefulsets of mysql database named mysql and mysql2, with 1 replica for each statefulset. They have their own PV, PVC but bound by only single headless service.

[root@ip-10-0-1-235 centos]# kubectl get pods -l app=mysql -o wide
NAME       READY     STATUS    RESTARTS   AGE       IP              NODE
mysql-0    1/1       Running   0          4m        192.168.13.21   ip-10-0-1-235.ec2.internal
mysql2-0   1/1       Running   0          3m        192.168.13.22   ip-10-0-1-235.ec2.internal

Now you can see the single headless service attached to both the pods

[root@ip-10-0-1-235 centos]# kubectl describe svc mysql
Name:              mysql
Namespace:         default
Labels:            <none>
Annotations:       <none>
Selector:          app=mysql
Type:              ClusterIP
IP:                None
Port:              <unset>  3306/TCP
TargetPort:        3306/TCP
Endpoints:         192.168.13.21:3306,192.168.13.22:3306
Session Affinity:  None
Events:            <none>

Now when you lookup the service from some other pod, it returns IP address of both the pods:

[root@rtp-worker-0 /]# nslookup mysql
Server:     10.96.0.10
Address:    10.96.0.10#53

Name:   mysql.default.svc.cluster.local
Address: 192.168.13.21
Name:   mysql.default.svc.cluster.local
Address: 192.168.13.22

Now, it is impossible to identify which address(pod) is of which statefulset. Now I tried to identify the statefulset using its metadata name, but couldn't

[root@rtp-worker-0 /]# nslookup mysql2.mysql.default.svc.cluster.local
Server:     10.96.0.10
Address:    10.96.0.10#53

** server can't find mysql2.mysql.default.svc.cluster.local: NXDOMAIN

Hope it clarifies.

Answer 2

@Prafull Ladha, here is my setup. My service and the statefulsets attached to it have different labels.app database-service: app=database mysqlset: app=mysql

khteh@khteh-T580:~ 2007 $ k get pods -l app=mysql -o wide
NAME      READY   STATUS    RESTARTS   AGE   IP         NODE         NOMINATED NODE
mysql-0   1/1     Running   1          18h   10.1.1.4   khteh-t580   <none>

khteh@khteh-T580:~ 2008 $ k get pods -l app=blazegraph -o wide
NAME           READY   STATUS    RESTARTS   AGE   IP           NODE         NOMINATED NODE
blazegraph-0   1/1     Running   1          18h   10.1.1.254   khteh-t580   <none>

khteh@khteh-T580:~ 2009 $ k describe service database-service
Name:              database-service
Namespace:         default
Labels:            app=database
Annotations:       kubectl.kubernetes.io/last-applied-configuration:
                 {"apiVersion":"v1","kind":"Service","metadata":{"annotations":{},"labels":{"app":"database"},"name":"database-service","namespace":"defaul...
Selector:          app=database,tier=database
Type:              ClusterIP
IP:                None
Port:              mysql  3306/TCP
TargetPort:        3306/TCP
Endpoints:         <none>
Port:              blazegraph  9999/TCP
TargetPort:        9999/TCP
Endpoints:         <none>
Session Affinity:  None
Events:            <none>

Notice the service Endpoints is . I am not sure this is the right setup.