Add an example for static-ip and deployment
This commit is contained in:
bprashanth
parent
3e87a2de40
commit
c243d930b0
15 changed files with 544 additions and 3 deletions
|
|
@ -1,3 +0,0 @@
|
|||
# Static IPs through the GCE Ingress controller
|
||||
|
||||
Placeholder
|
||||
128
examples/static-ip/gce/README.md
Normal file
128
examples/static-ip/gce/README.md
Normal file
|
|
@ -0,0 +1,128 @@
|
|||
# Static IPs
|
||||
|
||||
This example demonstrates how to assign a [static-ip](https://cloud.google.com/compute/docs/configure-instance-ip-addresses#reserve_new_static) to an Ingress on GCE.
|
||||
|
||||
## Prerequisites
|
||||
|
||||
You need a [TLS cert](/examples/PREREQUISITES.md#tls-certificates) and a [test HTTP service](/examples/PREREQUISITES.md#test-http-service) for this example.
|
||||
You will also need to make sure you Ingress targets exactly one Ingress
|
||||
controller by specifying the [ingress.class annotation](/examples/PREREQUISITES.md#ingress-class).
|
||||
|
||||
## Acquiring a static IP
|
||||
|
||||
In GCE, static IP belongs to a given project until the owner decides to release
|
||||
it. If you create a static IP and assign it to an Ingress, deleting the Ingress
|
||||
or tearing down the GKE cluster *will not* delete the static IP. You can check
|
||||
the static IPs you have as follows
|
||||
|
||||
```console
|
||||
$ gcloud compute addresses list --global
|
||||
NAME REGION ADDRESS STATUS
|
||||
test-ip 35.186.221.137 RESERVED
|
||||
|
||||
$ gcloud compute addresses list
|
||||
NAME REGION ADDRESS STATUS
|
||||
test-ip 35.186.221.137 RESERVED
|
||||
test-ip us-central1 35.184.21.228 RESERVED
|
||||
```
|
||||
|
||||
Note the difference between a regional and a global static ip. Only global
|
||||
static-ips will work with Ingress. If you don't already have an IP, you can
|
||||
create it
|
||||
|
||||
```console
|
||||
$ gcloud compute addresses create test-ip --global
|
||||
Created [https://www.googleapis.com/compute/v1/projects/kubernetesdev/global/addresses/test-ip].
|
||||
---
|
||||
address: 35.186.221.137
|
||||
creationTimestamp: '2017-01-31T10:32:29.889-08:00'
|
||||
description: ''
|
||||
id: '9221457935391876818'
|
||||
kind: compute#address
|
||||
name: test-ip
|
||||
selfLink: https://www.googleapis.com/compute/v1/projects/kubernetesdev/global/addresses/test-ip
|
||||
status: RESERVED
|
||||
```
|
||||
|
||||
## Assigning a static IP to an Ingress
|
||||
|
||||
You can now add the static IP from the previous step to an Ingress,
|
||||
by specifying the `kubernetes.io/global-static-ip-name` annotation,
|
||||
the example yaml in this directory already has it set to `test-ip`
|
||||
|
||||
```console
|
||||
$ kubectl create -f gce-static-ip-ingress.yaml
|
||||
ingress "static-ip" created
|
||||
|
||||
$ gcloud compute addresses list test-ip
|
||||
NAME REGION ADDRESS STATUS
|
||||
test-ip 35.186.221.137 IN_USE
|
||||
test-ip us-central1 35.184.21.228 RESERVED
|
||||
|
||||
$ kubectl get ing
|
||||
NAME HOSTS ADDRESS PORTS AGE
|
||||
static-ip * 35.186.221.137 80, 443 1m
|
||||
|
||||
$ curl 35.186.221.137 -Lk
|
||||
CLIENT VALUES:
|
||||
client_address=10.180.1.1
|
||||
command=GET
|
||||
real path=/
|
||||
query=nil
|
||||
request_version=1.1
|
||||
request_uri=http://35.186.221.137:8080/
|
||||
...
|
||||
```
|
||||
|
||||
## Retaining the static IP
|
||||
|
||||
You can test retention by deleting the Ingress
|
||||
|
||||
```console
|
||||
$ kubectl delete -f gce-static-ip-ingress.yaml
|
||||
ingress "static-ip" deleted
|
||||
|
||||
$ kubectl get ing
|
||||
No resources found.
|
||||
|
||||
$ gcloud compute addresses list test-ip --global
|
||||
NAME REGION ADDRESS STATUS
|
||||
test-ip 35.186.221.137 RESERVED
|
||||
```
|
||||
|
||||
## Promote ephemeral to static IP
|
||||
|
||||
If you simply create a HTTP Ingress resource, it gets an ephemeral IP
|
||||
|
||||
```console
|
||||
$ kubectl create -f gce-http-ingress.yaml
|
||||
ingress "http-ingress" created
|
||||
|
||||
$ kubectl get ing
|
||||
NAME HOSTS ADDRESS PORTS AGE
|
||||
http-ingress * 35.186.195.33 80 1h
|
||||
|
||||
$ gcloud compute forwarding-rules list
|
||||
NAME REGION IP_ADDRESS IP_PROTOCOL TARGET
|
||||
k8s-fw-default-http-ingress--32658fa96c080068 35.186.195.33 TCP k8s-tp-default-http-ingress--32658fa96c080068
|
||||
```
|
||||
|
||||
Note that because this is an ephemeral IP, it won't show up in the output of
|
||||
`gcloud compute addresses list`.
|
||||
|
||||
If you either directly create an Ingress with a TLS section, or modify a HTTP
|
||||
Ingress to have a TLS section, it gets a static IP.
|
||||
|
||||
```console
|
||||
$ kubectl patch ing http-ingress -p '{"spec":{"tls":[{"secretName":"tls-secret"}]}}'
|
||||
"http-ingress" patched
|
||||
|
||||
$ kubectl get ing
|
||||
NAME HOSTS ADDRESS PORTS AGE
|
||||
http-ingress * 35.186.195.33 80, 443 1h
|
||||
|
||||
$ gcloud compute addresses list
|
||||
NAME REGION ADDRESS STATUS
|
||||
k8s-fw-default-http-ingress--32658fa96c080068 35.186.195.33 IN_USE
|
||||
```
|
||||
|
||||
12
examples/static-ip/gce/gce-http-ingress.yaml
Normal file
12
examples/static-ip/gce/gce-http-ingress.yaml
Normal file
|
|
@ -0,0 +1,12 @@
|
|||
apiVersion: extensions/v1beta1
|
||||
kind: Ingress
|
||||
metadata:
|
||||
name: http-ingress
|
||||
annotations:
|
||||
kubernetes.io/ingress.class: "gce"
|
||||
spec:
|
||||
backend:
|
||||
# This assumes http-svc exists and routes to healthy endpoints.
|
||||
serviceName: http-svc
|
||||
servicePort: 80
|
||||
|
||||
19
examples/static-ip/gce/gce-static-ip-ingress.yaml
Normal file
19
examples/static-ip/gce/gce-static-ip-ingress.yaml
Normal file
|
|
@ -0,0 +1,19 @@
|
|||
apiVersion: extensions/v1beta1
|
||||
kind: Ingress
|
||||
metadata:
|
||||
name: static-ip
|
||||
# Assumes a global static ip with the same name exists.
|
||||
# You can acquire a static IP by running
|
||||
# gcloud compute addresses create test-ip --global
|
||||
annotations:
|
||||
kubernetes.io/ingress.global-static-ip-name: "test-ip"
|
||||
kubernetes.io/ingress.class: "gce"
|
||||
spec:
|
||||
tls:
|
||||
# This assumes tls-secret exists.
|
||||
- secretName: tls-secret
|
||||
backend:
|
||||
# This assumes http-svc exists and routes to healthy endpoints.
|
||||
serviceName: http-svc
|
||||
servicePort: 80
|
||||
|
||||
118
examples/static-ip/nginx/README.md
Normal file
118
examples/static-ip/nginx/README.md
Normal file
|
|
@ -0,0 +1,118 @@
|
|||
# Static IPs
|
||||
|
||||
|
||||
This example demonstrates how to assign a static-ip to an Ingress on through
|
||||
the Nginx controller.
|
||||
|
||||
## Prerequisites
|
||||
|
||||
You need a [TLS cert](/examples/PREREQUISITES.md#tls-certificates) and a [test HTTP service](/examples/PREREQUISITES.md#test-http-service) for this example.
|
||||
You will also need to make sure you Ingress targets exactly one Ingress
|
||||
controller by specifying the [ingress.class annotation](/examples/PREREQUISITES.md#ingress-class).
|
||||
|
||||
## Acquiring an IP
|
||||
|
||||
Since instances of the nginx controller actually run on nodes in your cluster,
|
||||
by default nginx Ingresses will only get static IPs if your cloudprovider
|
||||
supports static IP assignments to nodes. On GKE/GCE for example, even though
|
||||
nodes get static IPs, the IPs are not retained across upgrade.
|
||||
|
||||
To acquire a static IP for the nginx ingress controller, simply put it
|
||||
behind a Service of `Type=LoadBalancer`.
|
||||
|
||||
First, create a loadbalancer Service and wait for it to acquire an IP
|
||||
|
||||
```console
|
||||
$ kubectl create -f static-ip-svc.yaml
|
||||
service "nginx-ingress-lb" created
|
||||
|
||||
$ kubectl get svc nginx-ingress-lb
|
||||
NAME CLUSTER-IP EXTERNAL-IP PORT(S) AGE
|
||||
nginx-ingress-lb 10.0.138.113 104.154.109.191 80:31457/TCP,443:32240/TCP 15m
|
||||
```
|
||||
|
||||
then, update the ingress controller so it adopts the static IP of the Service
|
||||
by passing the `--publish-service` flag (the example yaml used in the next step
|
||||
already has it set to "nginx-ingress-lb").
|
||||
|
||||
```console
|
||||
$ kubectl create -f nginx-ingress-controller.yaml
|
||||
deployment "nginx-ingress-controller" created
|
||||
```
|
||||
|
||||
## Assigning the IP to an Ingress
|
||||
|
||||
From here on every Ingress created with the `ingress.class` annotation set to
|
||||
`nginx` will get the IP allocated in the previous step
|
||||
|
||||
```console
|
||||
$ kubectl create -f nginx-ingress.yaml
|
||||
ingress "nginx-ingress" created
|
||||
|
||||
$ kubectl get ing nginx-ingress
|
||||
NAME HOSTS ADDRESS PORTS AGE
|
||||
nginx-ingress * 104.154.109.191 80, 443 13m
|
||||
|
||||
$ curl 104.154.109.191 -kL
|
||||
CLIENT VALUES:
|
||||
client_address=10.180.1.25
|
||||
command=GET
|
||||
real path=/
|
||||
query=nil
|
||||
request_version=1.1
|
||||
request_uri=http://104.154.109.191:8080/
|
||||
...
|
||||
```
|
||||
|
||||
## Retaining the IP
|
||||
|
||||
You can test retention by deleting the Ingress
|
||||
|
||||
```console
|
||||
$ kubectl delete ing nginx-ingress
|
||||
ingress "nginx-ingress" deleted
|
||||
|
||||
$ kubectl create -f nginx-ingress.yaml
|
||||
ingress "nginx-ingress" created
|
||||
|
||||
$ kubectl get ing nginx-ingress
|
||||
NAME HOSTS ADDRESS PORTS AGE
|
||||
nginx-ingress * 104.154.109.191 80, 443 13m
|
||||
```
|
||||
|
||||
Note that unlike the GCE Ingress, the same loadbalancer IP is shared amongst all
|
||||
Ingresses, because all requests are proxied through the same set of nginx
|
||||
controllers.
|
||||
|
||||
## Promote ephemeral to static IP
|
||||
|
||||
To promote the allocated IP to static, you can update the Service manifest
|
||||
|
||||
```console
|
||||
$ kubectl patch svc nginx-ingress-lb -p '{"spec": {"loadBalancerIP": "104.154.109.191"}}'
|
||||
"nginx-ingress-lb" patched
|
||||
```
|
||||
|
||||
and promote the IP to static (promotion works differently for cloudproviders,
|
||||
provided example is for GKE/GCE)
|
||||
`
|
||||
```console
|
||||
$ gcloud compute addresses create nginx-ingress-lb --addresses 104.154.109.191 --region us-central1
|
||||
Created [https://www.googleapis.com/compute/v1/projects/kubernetesdev/regions/us-central1/addresses/nginx-ingress-lb].
|
||||
---
|
||||
address: 104.154.109.191
|
||||
creationTimestamp: '2017-01-31T16:34:50.089-08:00'
|
||||
description: ''
|
||||
id: '5208037144487826373'
|
||||
kind: compute#address
|
||||
name: nginx-ingress-lb
|
||||
region: us-central1
|
||||
selfLink: https://www.googleapis.com/compute/v1/projects/kubernetesdev/regions/us-central1/addresses/nginx-ingress-lb
|
||||
status: IN_USE
|
||||
users:
|
||||
- us-central1/forwardingRules/a09f6913ae80e11e6a8c542010af0000
|
||||
```
|
||||
|
||||
Now even if the Service is deleted, the IP will persist, so you can recreate the
|
||||
Service with `spec.loadBalancerIP` set to `104.154.109.191`.
|
||||
|
||||
52
examples/static-ip/nginx/nginx-ingress-controller.yaml
Normal file
52
examples/static-ip/nginx/nginx-ingress-controller.yaml
Normal file
|
|
@ -0,0 +1,52 @@
|
|||
apiVersion: extensions/v1beta1
|
||||
kind: Deployment
|
||||
metadata:
|
||||
name: nginx-ingress-controller
|
||||
labels:
|
||||
k8s-app: nginx-ingress-controller
|
||||
spec:
|
||||
replicas: 1
|
||||
template:
|
||||
metadata:
|
||||
labels:
|
||||
k8s-app: nginx-ingress-controller
|
||||
spec:
|
||||
# hostNetwork makes it possible to use ipv6 and to preserve the source IP correctly regardless of docker configuration
|
||||
# however, it is not a hard dependency of the nginx-ingress-controller itself and it may cause issues if port 10254 already is taken on the host
|
||||
# that said, since hostPort is broken on CNI (https://github.com/kubernetes/kubernetes/issues/31307) we have to use hostNetwork where CNI is used
|
||||
# like with kubeadm
|
||||
# hostNetwork: true
|
||||
terminationGracePeriodSeconds: 60
|
||||
containers:
|
||||
- image: gcr.io/google_containers/nginx-ingress-controller:0.9.0-beta.1
|
||||
name: nginx-ingress-controller
|
||||
readinessProbe:
|
||||
httpGet:
|
||||
path: /healthz
|
||||
port: 10254
|
||||
scheme: HTTP
|
||||
livenessProbe:
|
||||
httpGet:
|
||||
path: /healthz
|
||||
port: 10254
|
||||
scheme: HTTP
|
||||
initialDelaySeconds: 10
|
||||
timeoutSeconds: 1
|
||||
ports:
|
||||
- containerPort: 80
|
||||
hostPort: 80
|
||||
- containerPort: 443
|
||||
hostPort: 443
|
||||
env:
|
||||
- name: POD_NAME
|
||||
valueFrom:
|
||||
fieldRef:
|
||||
fieldPath: metadata.name
|
||||
- name: POD_NAMESPACE
|
||||
valueFrom:
|
||||
fieldRef:
|
||||
fieldPath: metadata.namespace
|
||||
args:
|
||||
- /nginx-ingress-controller
|
||||
- --default-backend-service=$(POD_NAMESPACE)/default-http-backend
|
||||
- --publish-service=$(POD_NAMESPACE)/nginx-ingress-lb
|
||||
17
examples/static-ip/nginx/nginx-ingress.yaml
Normal file
17
examples/static-ip/nginx/nginx-ingress.yaml
Normal file
|
|
@ -0,0 +1,17 @@
|
|||
apiVersion: extensions/v1beta1
|
||||
kind: Ingress
|
||||
metadata:
|
||||
name: nginx-ingress
|
||||
annotations:
|
||||
kubernetes.io/ingress.class: "nginx"
|
||||
spec:
|
||||
tls:
|
||||
# This assumes tls-secret exists.
|
||||
- secretName: tls-secret
|
||||
rules:
|
||||
- http:
|
||||
paths:
|
||||
- backend:
|
||||
# This assumes http-svc exists and routes to healthy endpoints.
|
||||
serviceName: http-svc
|
||||
servicePort: 80
|
||||
23
examples/static-ip/nginx/static-ip-svc.yaml
Normal file
23
examples/static-ip/nginx/static-ip-svc.yaml
Normal file
|
|
@ -0,0 +1,23 @@
|
|||
# This is the backend service
|
||||
apiVersion: v1
|
||||
kind: Service
|
||||
metadata:
|
||||
name: nginx-ingress-lb
|
||||
annotations:
|
||||
service.beta.kubernetes.io/external-traffic: OnlyLocal
|
||||
labels:
|
||||
app: nginx-ingress-lb
|
||||
spec:
|
||||
type: LoadBalancer
|
||||
loadBalancerIP: 104.154.109.191
|
||||
ports:
|
||||
- port: 80
|
||||
name: http
|
||||
targetPort: 80
|
||||
- port: 443
|
||||
name: https
|
||||
targetPort: 443
|
||||
selector:
|
||||
# Selects nginx-ingress-controller pods
|
||||
k8s-app: nginx-ingress-controller
|
||||
|
||||
Loading…
Add table
Add a link
Reference in a new issue