Enable SCTP in kubernetes

Til
Page content

Check if SCTP is supported by creating SCTP service

https://blog.aweimeow.tw/enable-sctp-in-kubernetes-cluster/

cychong@mini1:~/work/sctp$ cat service.yaml
apiVersion: v1
kind: Service
metadata:
  name: sctp
spec:
  selector:
    app: sctp
  ports:
  - protocol: SCTP
    port: 9999
    targetPort: 30001

cychong@mini1:~/work/sctp$ kubectl create -f service.yaml
The Service "sctp" is invalid: spec.ports[0].protocol: Unsupported value: "SCTP": supported values: "TCP", "UDP"

Enable SCTP in running kubernetes cluster

https://stackoverflow.com/questions/55909512/how-to-configure-already-running-cluster-in-kubernetes

Basically you must pass this flag to kube-apiserver. How you can do that depends on how you set up the cluster. If you used kubeadm or kubespray then you should edit file /etc/kubernetes/manifests/kube-apiserver.yaml and add this flag somewhere under “command” field (somewhere between other flags). After that kube-apiserver pod should be restarted automatically. If not - you can kill it by hand.

Add --feature-gates=SCTPSupport=True to /etc/kubernetes/manifest/kube-apiserver.yaml

    - --tls-private-key-file=/etc/kubernetes/pki/apiserver.key
    - --feature-gates=SCTPSupport=True
    image: k8s.gcr.io/kube-apiserver:v1.16.1

Restart kube-apiserver. Just kill it and wait restarted

cychong@mini1:/etc/kubernetes$ ps -ef |grep kube-api
root     21846 21824  9 00:02 ?        00:00:24 kube-apiserver --advertise-address=192.168.1.100 --allow-privileged=true --authorization-mode=Node,RBAC --client-ca-file=/etc/kubernetes/pki/ca.crt --enable-admission-plugins=NodeRestriction --enable-bootstrap-token-auth=true --etcd-cafile=/etc/kubernetes/pki/etcd/ca.crt --etcd-certfile=/etc/kubernetes/pki/apiserver-etcd-client.crt --etcd-keyfile=/etc/kubernetes/pki/apiserver-etcd-client.key --etcd-servers=https://127.0.0.1:2379 --insecure-port=0 --kubelet-client-certificate=/etc/kubernetes/pki/apiserver-kubelet-client.crt --kubelet-client-key=/etc/kubernetes/pki/apiserver-kubelet-client.key --kubelet-preferred-address-types=InternalIP,ExternalIP,Hostname --proxy-client-cert-file=/etc/kubernetes/pki/front-proxy-client.crt --proxy-client-key-file=/etc/kubernetes/pki/front-proxy-client.key --requestheader-allowed-names=front-proxy-client --requestheader-client-ca-file=/etc/kubernetes/pki/front-proxy-ca.crt --requestheader-extra-headers-prefix=X-Remote-Extra- --requestheader-group-headers=X-Remote-Group --requestheader-username-headers=X-Remote-User --secure-port=6443 --service-account-key-file=/etc/kubernetes/pki/sa.pub --service-cluster-ip-range=10.96.0.0/12 --tls-cert-file=/etc/kubernetes/pki/apiserver.crt --tls-private-key-file=/etc/kubernetes/pki/apiserver.key --feature-gates=SCTPSupport=True
cychong  29605 13731  0 00:06 pts/1    00:00:00 grep --color=auto kube-api
cychong@mini1:/etc/kubernetes$ sudo kill -SIGHUP 21846


cychong@mini1:/etc/kubernetes$ ps -ef |grep kube-api
root     30272 30246 79 00:07 ?        00:00:06 kube-apiserver --advertise-address=192.168.1.100 --allow-privileged=true --authorization-mode=Node,RBAC --client-ca-file=/etc/kubernetes/pki/ca.crt --enable-admission-plugins=NodeRestriction --enable-bootstrap-token-auth=true --etcd-cafile=/etc/kubernetes/pki/etcd/ca.crt --etcd-certfile=/etc/kubernetes/pki/apiserver-etcd-client.crt --etcd-keyfile=/etc/kubernetes/pki/apiserver-etcd-client.key --etcd-servers=https://127.0.0.1:2379 --insecure-port=0 --kubelet-client-certificate=/etc/kubernetes/pki/apiserver-kubelet-client.crt --kubelet-client-key=/etc/kubernetes/pki/apiserver-kubelet-client.key --kubelet-preferred-address-types=InternalIP,ExternalIP,Hostname --proxy-client-cert-file=/etc/kubernetes/pki/front-proxy-client.crt --proxy-client-key-file=/etc/kubernetes/pki/front-proxy-client.key --requestheader-allowed-names=front-proxy-client --requestheader-client-ca-file=/etc/kubernetes/pki/front-proxy-ca.crt --requestheader-extra-headers-prefix=X-Remote-Extra- --requestheader-group-headers=X-Remote-Group --requestheader-username-headers=X-Remote-User --secure-port=6443 --service-account-key-file=/etc/kubernetes/pki/sa.pub --service-cluster-ip-range=10.96.0.0/12 --tls-cert-file=/etc/kubernetes/pki/apiserver.crt --tls-private-key-file=/etc/kubernetes/pki/apiserver.key --feature-gates=SCTPSupport=True
cychong  30644 13731  0 00:07 pts/1    00:00:00 grep --color=auto kube-api

Check if SCTP service is supported

cychong@mini1:~/work/sctp$ kubectl create -f service.yaml
service/sctp created

Check iptables

cychong@mini1:~/work/sctp$ sudo iptables -L -n |grep 9999
REJECT     sctp --  0.0.0.0/0            10.98.74.252         /* default/sctp: has no endpoints */ sctp dpt:9999 reject-with icmp-port-unreachable

Check SCTP by creating sample SCTP-server

First delete manullay created service

cychong@mini1:~/work/sctp$ kubectl delete svc sctp
service "sctp" deleted

Install helm chart from https://github.com/aweimeow/sctp-server. This helm chart will deploy a pod which has python based SCTP server.

cychong@mini1:~/work/sctp$ git clone https://github.com/aweimeow/sctp-server
Cloning into 'sctp-server'...
remote: Enumerating objects: 17, done.
remote: Total 17 (delta 0), reused 0 (delta 0), pack-reused 17
Unpacking objects: 100% (17/17), done.

cychong@mini1:~/work/sctp$ helm install -n sctp sctp-server
NAME:   sctp
LAST DEPLOYED: Tue Oct  8 00:12:01 2019
NAMESPACE: default
STATUS: DEPLOYED

RESOURCES:
==> v1/Deployment
NAME  READY  UP-TO-DATE  AVAILABLE  AGE
sctp  0/1    0           0          3s

==> v1/Namespace
NAME  STATUS  AGE
sctp  Active  4s

==> v1/Pod(related)
NAME                   READY  STATUS   RESTARTS  AGE
sctp-7c94d9b5c9-wsjnd  0/1    Pending  0         1s

==> v1/Service
NAME  TYPE      CLUSTER-IP      EXTERNAL-IP  PORT(S)          AGE
sctp  NodePort  10.108.218.164  <none>       9999:30001/SCTP  4s


NOTES:
1. Get the application URL by running these commands:
  export NODE_PORT=$(kubectl get --namespace default -o jsonpath="{.spec.ports[0].nodePort}" services sctp)
  export NODE_IP=$(kubectl get nodes --namespace default -o jsonpath="{.items[0].status.addresses[0].address}")
  echo http://$NODE_IP:$NODE_PORT

Check the pod and service

cychong@mini1:~/work/sctp$ kubectl get svc
NAME         TYPE        CLUSTER-IP       EXTERNAL-IP     PORT(S)           AGE
kubernetes   ClusterIP   10.96.0.1        <none>          443/TCP           29d
my-ghost     NodePort    10.105.125.54    192.168.1.100   2368:32326/TCP    11d
sctp         NodePort    10.108.218.164   <none>          9999:30001/SCTP   2m42s
cychong@mini1:~/work/sctp$ kubectl get pods -o wide
NAME                        READY   STATUS    RESTARTS   AGE     IP             NODE    NOMINATED NODE   READINESS GATES
my-ghost-5f6578fd76-lb7xc   1/1     Running   41         11d     10.244.51.82   mini1   <none>           <none>
sctp-7c94d9b5c9-wsjnd       1/1     Running   0          2m55s   10.244.51.90   mini1   <none>           <none>

From the host, check if SCTP server is reachable from the host

cychong@mini1:~/work/sctp$ ncat --sctp mini1 30001
Howdy! What's your name?
Memphis
Ncat: Connection reset by peer.
cychong@mini1:~/work/sctp$

TODO - client in Pod and server in outside cluster

Reference