layout: true name: title_layout class: title_slide --- layout: true name: section_layout class: section_slide --- layout: true name: agenda_layout class: content_slide agenda_slide --- layout: true name: thank_you class: thank_you --- layout: true name: content_layout class: content_slide --- template: title_layout # Kuryr-Kubernetes ## Neutron networking for K8s Michał Dulko - Software Engineer - Red Hat
OpenStack Day Poland, 12.06.2018
--- template: agenda_layout # AGENDA -- 1. Networking concepts in K8s and CNI -- 2. Kuryr-Kubernetes * Basics * Under the hood * Limitations and future -- Slides available at: **https://dulek.github.io/openstack-days-kuryr** --- template: section_layout Kubernetes and Container Network Interface (CNI) --- # Kubernetes and CNI ## Kubernetes networking concepts -- **Pod** - smallest schedulable K8s entity consisting of one or more containers guaranteed to be placed on a single host. **Those containers are sharing the networking settings and IP.** -- **Service** - an entitity making K8s Pods providing the same service accessible through a single IP or hostname. There are multiple types of Services: -- * `ClusterIP` - exposed only on cluster-internal IP. -- * `LoadBalancer` - exposed outside using cloud provider's load balancer. -- * `NodePort` - exposed on a certain port of the K8s node. -- * `ExternalName` - returns CNAME record. --- # Kubernetes and CNI ## Kubernetes networking concepts **Pod** - smallest schedulable K8s entity consisting of one or more containers guaranteed to be placed on a single host. **Those containers are sharing the networking settings and IP.** **Service** - an entitity making K8s Pods providing the same service accessible through a single IP or hostname. There are multiple types of Services: * `ClusterIP` - exposed only on cluster-internal IP. * `LoadBalancer` - exposed outside using cloud provider's load balancer. * ~~`NodePort` - exposed on a certain port of the K8s node.~~ * ~~`ExternalName` - returns CNAME record.~~ -- **Ingress (Router)** - entity defining rules allowing inbound connections to reach services on K8s cluster, normally implemented as L7 load balancer. Single Ingress can give **multiple** services externally reachable URLs, LB, terminate SSL. Requires *Ingress controller*, e.g. pod that watches for Ingress resources and adjusts its nginx rules accordingly. -- **Network Policies** - Kubernetes equivalent of Security Groups. --- # Kubernetes and CNI ## Service of type `LoadBalancer` vs Ingress .diag[ .left-column[ .center[`LoadBalancer` Service] ] ] -- .diag[ .right-column[ .center[Ingress] ] ] --- # Kubernetes and CNI ## Container Network Interface * Project governed by CNCF (*Cloud Native Computing Foundation*). -- * API is based on net.d configuration, executables and environment variables: -- * CNI plugins are configured in net.d directory (default `/etc/cni/net.d`) by config files. CNI starts looking from first file lexicographically, e.g. `10-kuryr.conf`. -- * CNI plugins are just binaries that should be run with certain environment variables set. -- - `CNI_COMMAND` - either `ADD`, `DEL` or `VERSION`. -- - `CNI_CONTAINERID` - unique container ID, key when storing state. -- - `CNI_NETNS` - network namespace that interface should be placed in. -- - `CNI_IFNAME` - requested name of the interface in the `CNI_NETNS`. -- * When run with `CNI_COMMAND`=`ADD`, CNI plugin is responsible for putting the interface named `CNI_IFNAME` into `CNI_NETNS` network namespace. This interface should be connected to the desired network. Returned are the interface details like its IP, gateway, DNS and routes. -- * Opposite applies for `CNI_COMMAND`=`DEL`, just the interface doesn't need to be deleted as it will go automatically with deleted netns. --- # Kubernetes and CNI ## CNI ADD example .left-column[ * `CNI_COMMAND="ADD"` * `CNI_CONTAINERID="
"` * `CNI_NETNS="CONTAINER NET NS"` * `CNI_IFNAME="eth0"` ] .right-column[.center[]] --- # Kubernetes and CNI ## CNI ADD example .left-column[ * `CNI_COMMAND="ADD"` * `CNI_CONTAINERID="
"` * `CNI_NETNS="CONTAINER NET NS"` * `CNI_IFNAME="eth0"` ] .right-column[.center[]] --- # Kubernetes and CNI ## CNI ADD example .left-column[ * `CNI_COMMAND="ADD"` * `CNI_CONTAINERID="
"` * `CNI_NETNS="CONTAINER NET NS"` * `CNI_IFNAME="eth0"` ] .right-column[.center[]] --- # Kubernetes and CNI ## CNI ADD example .left-column[ * `CNI_COMMAND="ADD"` * `CNI_CONTAINERID="
"` * `CNI_NETNS="CONTAINER NET NS"` * `CNI_IFNAME="eth0"` ] .right-column[.center[]] --- # Kubernetes and CNI ## CNI ADD example .left-column[ * `CNI_COMMAND="ADD"` * `CNI_CONTAINERID="
"` * `CNI_NETNS="CONTAINER NET NS"` * `CNI_IFNAME="eth0"` ] .right-column[.center[]] --- # Kubernetes and CNI ## CNI ADD example .left-column[ * `CNI_COMMAND="ADD"` * `CNI_CONTAINERID="
"` * `CNI_NETNS="CONTAINER NET NS"` * `CNI_IFNAME="eth0"` ] .right-column[.center[]] --- template: section_layout Kuryr-Kubernetes --- # Kuryr-Kubernetes ## Logos or mascots #.left-column[] -- #.right-column[.center[.half[]]] --- # Kuryr-Kubernetes ## Primer * Kuryr-Kubernetes is part of OpenStack Kuryr project. ??? Kuryr - Docker networking plugin, kuryr-libnetwork - Docker libnetwork plugin, fuxi - Docker volume plugin, fuxi-kubernetes - K8s volume plugin. -- * Repo: https://git.openstack.org/openstack/kuryr-kubernetes * Written in Python, *kuryr-kubernetes* on Python Package Index. -- * Developers from Red Hat, Huawei, Intel, Samsung… -- * Out of experimental phase in **OpenStack Queens release** with version **0.4.3**. * Follows **cycle-with-intermediary** release model. -- * Designed to work as either as: * **standalone** service along with other OpenStack services; * **in pods** on the K8s cluster it's supposed to provide networking for. -- * Kuryr-Kubernetes services require connection to: * OpenStack (Keystone and Neutron, optionally Octavia) * Kubernetes API --- # Kuryr-Kubernetes ## Motivations -- * Harness the power of OpenStack Neutron for your Kubernetes cluster! -- .center[.seventy[]] --- # Kuryr-Kubernetes ## Motivations * ~~Harness the power of OpenStack Neutron for your Kubernetes cluster!~~ -- * Use some of Neutron functionalities for Pods networking. -- * Provide **interconnectivity** between OpenStack VMs and Kubernetes Pods and Services. * Pod->VM, VM->Pod * VM->Services -- * Avoiding having multiple network overlays in **Kubernetes on OpenStack** deployments. --- # Kuryr-Kubernetes ## Multiple-Overlays .center[.seventy[]] --- # Kuryr-Kubernetes ## Kuryr components -- * kuryr-controller * Responsible for OpenStack API operations. -- * kuryr-cni * Executed by CNI * When deployed without kuryr-daemon, wires the pods -- * kuryr-daemon (optional in Queens, recommended in Rocky+) * Watches for new pods and wires them when requested by kuryr-cni. --- # Kuryr-Kubernetes ## Kuryr components .eighty[.center[]] --- # Kuryr-Kubernetes ## Baremetal containers networking .sixty[.center[]] --- # Kuryr-Kubernetes ## Nested containers networking .sixty[.center[]] --- # Kuryr-Kubernetes ## Port pools * To mitigate Neutron performance issues, we're doing port pooling. -- * Pooling happens in kuryr-controller. -- * Pool key consists of: * host * project_id * security_groups * network_id -- * Pool gets populated with ports when we need the first on or on demand. -- * We want to make this faster by moving port choice from the pool into the kuryr-daemon. --- # Kuryr-Kubernetes ## Services support .half[.center[]] --- # Kuryr-Kubernetes ## Containerized deployment ``` apiVersion: apps/v1beta1 kind: Deployment spec: replicas: 1 template: spec: serviceAccountName: kuryr-controller automountServiceAccountToken: true hostNetwork: true containers: - image: kuryr/controller:latest readinessProbe: httpGet: path: /ready livenessProbe: httpGet: path: /alive ``` --- # Kuryr-Kubernetes ## Current limitations (as of Queens release - 0.4.3) -- * Performance issues - Neutron -- * Only a single kuryr-controller in environment -- * No support for Ingress and Routers -- * No support for Network Policies --- # Kuryr-Kubernetes ## Features in development -- * Ingress and Router support (Rocky) * It'll be L7 LB - Kuryr will translate the rules. -- * Active/Passive HA (Rocky) * Only on containerized deployment. -- * Moving VIF management to kuryr-daemon (Rocky) -- * Network-per-namespace (Rocky) -- * Neutron performance improvements (Rocky) -- * Multi-network support (Rocky or Stein) -- * Network Policies support (Rocky or Stein) -- * DPDK support (Rocky or Stein) -- * Active/Active HA (Stein) --- # Kuryr-Kubernetes ## How to use, how to contribute * Documentation: https://docs.openstack.org/kuryr-kubernetes/latest -- * IRC channel: [#openstack-kuryr@Freenode](irc://chat.freenode.net/openstack-kuryr) -- * Bugs: https://bugs.launchpad.net/kuryr-kubernetes -- * How to contribute: https://wiki.openstack.org/wiki/How_To_Contribute --- template: section_layout  .medium-text[ Slides are available on http://creativecommons.org/licenses/by/4.0/ ] --- template: thank_you # Q&A ## Thank you! **Slides can be found at:** - https://dulek.github.io/openstack-days-kuryr - https://github.com/dulek/openstack-days-kuryr **Michał Dulko**  [mdulko@redhat.com](mailto:mdulko@redhat.com)  [dulek (freenode)](irc://chat.freenode.net/dulek,isnick)