Merge branch 'master' into xff
This commit is contained in:
Dario Nieuwenhuis
GitHub
commit
b5bcb93a4b
1532 changed files with 65966 additions and 34963 deletions
|
|
@ -2,17 +2,18 @@
|
|||
|
||||
## Contents
|
||||
|
||||
- [Mandatory command](#mandatory-command)
|
||||
- [Custom Provider](#custom-provider)
|
||||
- [Docker for Mac](#docker-for-mac)
|
||||
- [minikube](#minikube)
|
||||
- [AWS](#aws)
|
||||
- [GCE - GKE](#gce---gke)
|
||||
- [Azure](#azure)
|
||||
- [Baremetal](#baremetal)
|
||||
- [Generic Deployment](#generic-deployment)
|
||||
- [Mandatory command](#mandatory-command)
|
||||
- [Provider Specific Steps](#provider-specific-steps)
|
||||
- [Docker for Mac](#docker-for-mac)
|
||||
- [minikube](#minikube)
|
||||
- [AWS](#aws)
|
||||
- [GCE - GKE](#gce---gke)
|
||||
- [Azure](#azure)
|
||||
- [Baremetal](#baremetal)
|
||||
- [Verify installation](#verify-installation)
|
||||
- [Detect installed version](#detect-installed-version)
|
||||
- [Using Helm](#using-helm)
|
||||
- [Verify installation](#verify-installation)
|
||||
- [Detect installed version](#detect-installed-version)
|
||||
|
||||
## Generic Deployment
|
||||
|
||||
|
|
@ -24,16 +25,14 @@ The following resources are required for a generic deployment.
|
|||
kubectl apply -f https://raw.githubusercontent.com/kubernetes/ingress-nginx/master/deploy/mandatory.yaml
|
||||
```
|
||||
|
||||
## Custom Service Provider Deployment
|
||||
### Provider Specific Steps
|
||||
|
||||
There are cloud provider specific yaml files.
|
||||
|
||||
### Docker for Mac
|
||||
#### Docker for Mac
|
||||
|
||||
Kubernetes is available for Docker for Mac's Edge channel. Switch to the [Edge
|
||||
channel][edge] and [enable Kubernetes][enable].
|
||||
Kubernetes is available in Docker for Mac (from [version 18.06.0-ce](https://docs.docker.com/docker-for-mac/release-notes/#stable-releases-of-2018))
|
||||
|
||||
[edge]: https://docs.docker.com/docker-for-mac/install/
|
||||
[enable]: https://docs.docker.com/docker-for-mac/#kubernetes
|
||||
|
||||
Create a service
|
||||
|
|
@ -42,7 +41,7 @@ Create a service
|
|||
kubectl apply -f https://raw.githubusercontent.com/kubernetes/ingress-nginx/master/deploy/provider/cloud-generic.yaml
|
||||
```
|
||||
|
||||
### minikube
|
||||
#### minikube
|
||||
|
||||
For standard usage:
|
||||
|
||||
|
|
@ -68,13 +67,13 @@ default-http-backend-66b447d9cf-rrlf9 1/1 Running 0 12s
|
|||
nginx-ingress-controller-fdcdcd6dd-vvpgs 1/1 Running 0 11s
|
||||
```
|
||||
|
||||
### AWS
|
||||
#### AWS
|
||||
|
||||
In AWS we use an Elastic Load Balancer (ELB) to expose the NGINX Ingress controller behind a Service of `Type=LoadBalancer`.
|
||||
Since Kubernetes v1.9.0 it is possible to use a classic load balancer (ELB) or network load balancer (NLB)
|
||||
Please check the [elastic load balancing AWS details page](https://aws.amazon.com/es/elasticloadbalancing/details/)
|
||||
|
||||
#### Elastic Load Balancer - ELB
|
||||
##### Elastic Load Balancer - ELB
|
||||
|
||||
This setup requires to choose in which layer (L4 or L7) we want to configure the ELB:
|
||||
|
||||
|
|
@ -102,7 +101,7 @@ This example creates an ELB with just two listeners, one in port 80 and another
|
|||
|
||||
|
||||
|
||||
#### Network Load Balancer (NLB)
|
||||
##### Network Load Balancer (NLB)
|
||||
|
||||
This type of load balancer is supported since v1.10.0 as an ALPHA feature.
|
||||
|
||||
|
|
@ -110,7 +109,7 @@ This type of load balancer is supported since v1.10.0 as an ALPHA feature.
|
|||
kubectl apply -f https://raw.githubusercontent.com/kubernetes/ingress-nginx/master/deploy/provider/aws/service-nlb.yaml
|
||||
```
|
||||
|
||||
### GCE - GKE
|
||||
#### GCE - GKE
|
||||
|
||||
```console
|
||||
kubectl apply -f https://raw.githubusercontent.com/kubernetes/ingress-nginx/master/deploy/provider/cloud-generic.yaml
|
||||
|
|
@ -118,16 +117,15 @@ kubectl apply -f https://raw.githubusercontent.com/kubernetes/ingress-nginx/mast
|
|||
|
||||
**Important Note:** proxy protocol is not supported in GCE/GKE
|
||||
|
||||
### Azure
|
||||
#### Azure
|
||||
|
||||
|
||||
```console
|
||||
kubectl apply -f https://raw.githubusercontent.com/kubernetes/ingress-nginx/master/deploy/provider/cloud-generic.yaml
|
||||
```
|
||||
|
||||
**Important Note:** proxy protocol is not supported in GCE/GKE
|
||||
|
||||
### Baremetal
|
||||
#### Baremetal
|
||||
|
||||
Using [NodePort](https://kubernetes.io/docs/concepts/services-networking/service/#type-nodeport):
|
||||
|
||||
|
|
@ -135,9 +133,30 @@ Using [NodePort](https://kubernetes.io/docs/concepts/services-networking/service
|
|||
kubectl apply -f https://raw.githubusercontent.com/kubernetes/ingress-nginx/master/deploy/provider/baremetal/service-nodeport.yaml
|
||||
```
|
||||
|
||||
### Verify installation
|
||||
|
||||
To check if the ingress controller pods have started, run the following command:
|
||||
|
||||
```console
|
||||
kubectl get pods --all-namespaces -l app=ingress-nginx --watch
|
||||
```
|
||||
|
||||
Once the operator pods are running, you can cancel the above command by typing `Ctrl+C`.
|
||||
Now, you are ready to create your first ingress.
|
||||
|
||||
### Detect installed version
|
||||
|
||||
To detect which version of the ingress controller is running, exec into the pod and run `nginx-ingress-controller version` command.
|
||||
|
||||
```console
|
||||
POD_NAMESPACE=ingress-nginx
|
||||
POD_NAME=$(kubectl get pods -n $POD_NAMESPACE -l app=ingress-nginx -o jsonpath={.items[0].metadata.name})
|
||||
kubectl exec -it $POD_NAME -n $POD_NAMESPACE -- /nginx-ingress-controller --version
|
||||
```
|
||||
|
||||
## Using Helm
|
||||
|
||||
NGINX Ingress controller can be installed via [Helm](https://helm.sh/) using the chart [stable/nginx](https://github.com/kubernetes/charts/tree/master/stable/nginx-ingress) from the official charts repository.
|
||||
NGINX Ingress controller can be installed via [Helm](https://helm.sh/) using the chart [stable/nginx-ingress](https://github.com/kubernetes/charts/tree/master/stable/nginx-ingress) from the official charts repository.
|
||||
To install the chart with the release name `my-nginx`:
|
||||
|
||||
```console
|
||||
|
|
@ -150,23 +169,10 @@ If the kubernetes cluster has RBAC enabled, then run:
|
|||
helm install stable/nginx-ingress --name my-nginx --set rbac.create=true
|
||||
```
|
||||
|
||||
## Verify installation
|
||||
|
||||
To check if the ingress controller pods have started, run the following command:
|
||||
Detect installed version:
|
||||
|
||||
```console
|
||||
kubectl get pods --all-namespaces -l app=ingress-nginx --watch
|
||||
POD_NAME=$(kubectl get pods -l app=nginx-ingress -o jsonpath={.items[0].metadata.name})
|
||||
kubectl exec -it $POD_NAME -- /nginx-ingress-controller --version
|
||||
```
|
||||
|
||||
Once the operator pods are running, you can cancel the above command by typing `Ctrl+C`.
|
||||
Now, you are ready to create your first ingress.
|
||||
|
||||
## Detect installed version
|
||||
|
||||
To detect which version of the ingress controller is running, exec into the pod and run `nginx-ingress-controller version` command.
|
||||
|
||||
```console
|
||||
POD_NAMESPACE=ingress-nginx
|
||||
POD_NAME=$(kubectl get pods -n $POD_NAMESPACE -l app=ingress-nginx -o jsonpath={.items[0].metadata.name})
|
||||
kubectl exec -it $POD_NAME -n $POD_NAMESPACE -- /nginx-ingress-controller --version
|
||||
```
|
||||
|
|
|
|||
|
|
@ -1,8 +1,8 @@
|
|||
# Upgrading
|
||||
|
||||
!!! important
|
||||
No matter the method you use for upgrading, *if you use template overrides,
|
||||
make sure your templates are compatible with the new version of ingress-nginx*.
|
||||
No matter the method you use for upgrading, _if you use template overrides,
|
||||
make sure your templates are compatible with the new version of ingress-nginx_.
|
||||
|
||||
## Without Helm
|
||||
|
||||
|
|
@ -33,12 +33,11 @@ The easiest way to do this is e.g. (do note you may need to change the name para
|
|||
|
||||
```
|
||||
kubectl set image deployment/nginx-ingress-controller \
|
||||
nginx-ingress-controller=nginx:quay.io/kubernetes-ingress-controller/nginx-ingress-controller:0.15.0
|
||||
nginx-ingress-controller=nginx:quay.io/kubernetes-ingress-controller/nginx-ingress-controller:0.18.0
|
||||
```
|
||||
|
||||
For interactive editing, use `kubectl edit deployment nginx-ingress-controller`.
|
||||
|
||||
|
||||
## With Helm
|
||||
|
||||
If you installed ingress-nginx using the Helm command in the deployment docs so its name is `ngx-ingress`,
|
||||
|
|
@ -47,4 +46,3 @@ you should be able to upgrade using
|
|||
```shell
|
||||
helm upgrade --reuse-values ngx-ingress stable/nginx-ingress
|
||||
```
|
||||
|
||||
|
|
|
|||
|
|
@ -40,7 +40,7 @@ metadata:
|
|||
# name of the secret that contains the user/password definitions
|
||||
nginx.ingress.kubernetes.io/auth-secret: basic-auth
|
||||
# message to display with an appropriate context why the authentication is required
|
||||
nginx.ingress.kubernetes.io/auth-realm: "Authentication Required - foo"
|
||||
nginx.ingress.kubernetes.io/auth-realm: 'Authentication Required - foo'
|
||||
spec:
|
||||
rules:
|
||||
- host: foo.bar.com
|
||||
|
|
|
|||
|
|
@ -24,8 +24,8 @@ Sample:
|
|||
metadata:
|
||||
name: application
|
||||
annotations:
|
||||
"nginx.ingress.kubernetes.io/auth-url": "https://$host/oauth2/auth"
|
||||
"nginx.ingress.kubernetes.io/auth-signin": "https://$host/oauth2/sign_in"
|
||||
nginx.ingress.kubernetes.io/auth-url: "https://$host/oauth2/auth"
|
||||
nginx.ingress.kubernetes.io/auth-signin: "https://$host/oauth2/start?rd=$escaped_request_uri"
|
||||
...
|
||||
```
|
||||
|
||||
|
|
|
|||
|
|
@ -2,8 +2,8 @@ apiVersion: extensions/v1beta1
|
|||
kind: Ingress
|
||||
metadata:
|
||||
annotations:
|
||||
nginx.ingress.kubernetes.io/auth-signin: https://$host/oauth2/start
|
||||
nginx.ingress.kubernetes.io/auth-url: https://$host/oauth2/auth
|
||||
nginx.ingress.kubernetes.io/auth-url: "https://$host/oauth2/auth"
|
||||
nginx.ingress.kubernetes.io/auth-signin: "https://$host/oauth2/start?rd=$escaped_request_uri"
|
||||
name: external-auth-oauth2
|
||||
namespace: kube-system
|
||||
spec:
|
||||
|
|
|
|||
|
|
@ -28,7 +28,7 @@ spec:
|
|||
value: <Client ID>
|
||||
- name: OAUTH2_PROXY_CLIENT_SECRET
|
||||
value: <Client Secret>
|
||||
# python -c 'import os,base64; print base64.b64encode(os.urandom(16))'
|
||||
# docker run -ti --rm python:3-alpine python -c 'import secrets,base64; print(base64.b64encode(base64.b64encode(secrets.token_bytes(16))));'
|
||||
- name: OAUTH2_PROXY_COOKIE_SECRET
|
||||
value: SECRET
|
||||
image: docker.io/colemickens/oauth2_proxy:latest
|
||||
|
|
|
|||
|
|
@ -2,7 +2,7 @@
|
|||
|
||||
## Ingress
|
||||
|
||||
The Ingress in this example adds a custom header to Nginx configuration that only applies to that specific Ingress. If you want to add headers that apply globally to all Ingresses, please have a look at [this example](/docs/examples/customization/custom-headers).
|
||||
The Ingress in this example adds a custom header to Nginx configuration that only applies to that specific Ingress. If you want to add headers that apply globally to all Ingresses, please have a look at [this example](/examples/customization/custom-headers/README).
|
||||
|
||||
```console
|
||||
$ kubectl apply -f ingress.yaml
|
||||
|
|
|
|||
|
|
@ -5,7 +5,6 @@ metadata:
|
|||
annotations:
|
||||
nginx.ingress.kubernetes.io/configuration-snippet: |
|
||||
more_set_headers "Request-Id: $req_id";
|
||||
|
||||
spec:
|
||||
rules:
|
||||
- host: custom.configuration.com
|
||||
|
|
|
|||
|
|
@ -1,82 +1,83 @@
|
|||
# Custom Errors
|
||||
|
||||
This example shows how is possible to use a custom backend to render custom error pages. The code of this example is located here [custom-error-pages](https://github.com/kubernetes/ingress-nginx/tree/master/docs/examples/customization/custom-errors)
|
||||
This example demonstrates how to use a custom backend to render custom error pages.
|
||||
|
||||
## Customized default backend
|
||||
|
||||
The idea is to use the headers `X-Code` and `X-Format` that NGINX pass to the backend in case of an error to find out the best existent representation of the response to be returned. i.e. if the request contains an `Accept` header of type `json` the error should be in that format and not in `html` (the default in NGINX).
|
||||
|
||||
First create the custom backend to use in the Ingress controller
|
||||
First, create the custom `default-backend`. It will be used by the Ingress controller later on.
|
||||
|
||||
```
|
||||
$ kubectl create -f custom-default-backend.yaml
|
||||
service "nginx-errors" created
|
||||
replicationcontroller "nginx-errors" created
|
||||
deployment.apps "nginx-errors" created
|
||||
```
|
||||
|
||||
```
|
||||
$ kubectl get svc
|
||||
NAME CLUSTER-IP EXTERNAL-IP PORT(S) AGE
|
||||
echoheaders 10.3.0.7 nodes 80/TCP 23d
|
||||
kubernetes 10.3.0.1 <none> 443/TCP 34d
|
||||
nginx-errors 10.3.0.102 <none> 80/TCP 11s
|
||||
```
|
||||
This should have created a Deployment and a Service with the name `nginx-errors`.
|
||||
|
||||
```
|
||||
$ kubectl get rc
|
||||
CONTROLLER REPLICAS AGE
|
||||
echoheaders 1 19d
|
||||
nginx-errors 1 19s
|
||||
$ kubectl get deploy,svc
|
||||
NAME DESIRED CURRENT READY AGE
|
||||
deployment.apps/nginx-errors 1 1 1 10s
|
||||
|
||||
NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE
|
||||
service/nginx-errors ClusterIP 10.0.0.12 <none> 80/TCP 10s
|
||||
```
|
||||
|
||||
Next create the Ingress controller executing
|
||||
```
|
||||
$ kubectl create -f rc-custom-errors.yaml
|
||||
```
|
||||
## Ingress controller configuration
|
||||
|
||||
Now to check if this is working we use curl:
|
||||
If you do not already have an instance of the the NGINX Ingress controller running, deploy it according to the
|
||||
[deployment guide][deploy], then follow these steps:
|
||||
|
||||
1. Edit the `nginx-ingress-controller` Deployment and set the value of the `--default-backend` flag to the name of the
|
||||
newly created error backend.
|
||||
|
||||
2. Edit the `nginx-configuration` ConfigMap and create the key `custom-http-errors` with a value of `404,503`.
|
||||
|
||||
3. Take note of the IP address assigned to the NGINX Ingress controller Service.
|
||||
```
|
||||
$ kubectl get svc ingress-nginx
|
||||
NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE
|
||||
ingress-nginx ClusterIP 10.0.0.13 <none> 80/TCP,443/TCP 10m
|
||||
```
|
||||
|
||||
!!! Note
|
||||
The `ingress-nginx` Service is of type `ClusterIP` in this example. This may vary depending on your environment.
|
||||
Make sure you can use the Service to reach NGINX before proceeding with the rest of this example.
|
||||
|
||||
[deploy]: ../../../deploy/
|
||||
|
||||
## Testing error pages
|
||||
|
||||
Let us send a couple of HTTP requests using cURL and validate everything is working as expected.
|
||||
|
||||
A request to the default backend returns a 404 error with a custom message:
|
||||
|
||||
```
|
||||
$ curl -v http://172.17.4.99/
|
||||
* Trying 172.17.4.99...
|
||||
* Connected to 172.17.4.99 (172.17.4.99) port 80 (#0)
|
||||
> GET / HTTP/1.1
|
||||
> Host: 172.17.4.99
|
||||
> User-Agent: curl/7.43.0
|
||||
> Accept: */*
|
||||
>
|
||||
< HTTP/1.1 404 Not Found
|
||||
< Server: nginx/1.10.0
|
||||
< Date: Wed, 04 May 2016 02:53:45 GMT
|
||||
< Content-Type: text/html
|
||||
< Transfer-Encoding: chunked
|
||||
< Connection: keep-alive
|
||||
< Vary: Accept-Encoding
|
||||
<
|
||||
$ curl -D- http://10.0.0.13/
|
||||
HTTP/1.1 404 Not Found
|
||||
Server: nginx/1.13.12
|
||||
Date: Tue, 12 Jun 2018 19:11:24 GMT
|
||||
Content-Type: */*
|
||||
Transfer-Encoding: chunked
|
||||
Connection: keep-alive
|
||||
|
||||
<span>The page you're looking for could not be found.</span>
|
||||
|
||||
* Connection #0 to host 172.17.4.99 left intact
|
||||
```
|
||||
|
||||
Specifying json as expected format:
|
||||
A request with a custom `Accept` header returns the corresponding document type (JSON):
|
||||
|
||||
```
|
||||
$ curl -v http://172.17.4.99/ -H 'Accept: application/json'
|
||||
* Trying 172.17.4.99...
|
||||
* Connected to 172.17.4.99 (172.17.4.99) port 80 (#0)
|
||||
> GET / HTTP/1.1
|
||||
> Host: 172.17.4.99
|
||||
> User-Agent: curl/7.43.0
|
||||
> Accept: application/json
|
||||
>
|
||||
< HTTP/1.1 404 Not Found
|
||||
< Server: nginx/1.10.0
|
||||
< Date: Wed, 04 May 2016 02:54:00 GMT
|
||||
< Content-Type: text/html
|
||||
< Transfer-Encoding: chunked
|
||||
< Connection: keep-alive
|
||||
< Vary: Accept-Encoding
|
||||
<
|
||||
$ curl -D- -H 'Accept: application/json' http://10.0.0.13/
|
||||
HTTP/1.1 404 Not Found
|
||||
Server: nginx/1.13.12
|
||||
Date: Tue, 12 Jun 2018 19:12:36 GMT
|
||||
Content-Type: application/json
|
||||
Transfer-Encoding: chunked
|
||||
Connection: keep-alive
|
||||
Vary: Accept-Encoding
|
||||
|
||||
{ "message": "The page you're looking for could not be found" }
|
||||
|
||||
* Connection #0 to host 172.17.4.99 left intact
|
||||
```
|
||||
|
||||
To go further with this example, feel free to deploy your own applications and Ingress objects, and validate that the
|
||||
responses are still in the correct format when a backend returns 503 (eg. if you scale a Deployment down to 0 replica).
|
||||
|
|
|
|||
|
|
@ -1,3 +1,4 @@
|
|||
---
|
||||
apiVersion: v1
|
||||
kind: Service
|
||||
metadata:
|
||||
|
|
@ -5,27 +6,35 @@ metadata:
|
|||
labels:
|
||||
app: nginx-errors
|
||||
spec:
|
||||
ports:
|
||||
- port: 80
|
||||
targetPort: 80
|
||||
protocol: TCP
|
||||
name: http
|
||||
selector:
|
||||
app: nginx-errors
|
||||
ports:
|
||||
- port: 80
|
||||
targetPort: 8080
|
||||
name: http
|
||||
---
|
||||
apiVersion: v1
|
||||
kind: ReplicationController
|
||||
apiVersion: apps/v1beta2
|
||||
kind: Deployment
|
||||
apiVersion: apps/v1beta2
|
||||
metadata:
|
||||
name: nginx-errors
|
||||
spec:
|
||||
replicas: 1
|
||||
selector:
|
||||
matchLabels:
|
||||
app: nginx-errors
|
||||
template:
|
||||
metadata:
|
||||
labels:
|
||||
app: nginx-errors
|
||||
spec:
|
||||
containers:
|
||||
- name: nginx-errors
|
||||
image: aledbf/nginx-error-server:0.1
|
||||
- name: nginx-error-server
|
||||
image: quay.io/kubernetes-ingress-controller/custom-error-pages-amd64:0.3
|
||||
ports:
|
||||
- containerPort: 80
|
||||
- containerPort: 8080
|
||||
# Setting the environment variable DEBUG we can see the headers sent
|
||||
# by the ingress controller to the backend in the client response.
|
||||
# env:
|
||||
# - name: DEBUG
|
||||
# value: "true"
|
||||
|
|
|
|||
|
|
@ -1,53 +0,0 @@
|
|||
apiVersion: v1
|
||||
kind: ReplicationController
|
||||
metadata:
|
||||
name: nginx-ingress-controller
|
||||
labels:
|
||||
k8s-app: nginx-ingress-lb
|
||||
spec:
|
||||
replicas: 1
|
||||
selector:
|
||||
k8s-app: nginx-ingress-lb
|
||||
template:
|
||||
metadata:
|
||||
labels:
|
||||
k8s-app: nginx-ingress-lb
|
||||
name: nginx-ingress-lb
|
||||
spec:
|
||||
terminationGracePeriodSeconds: 60
|
||||
containers:
|
||||
- image: quay.io/kubernetes-ingress-controller/nginx-ingress-controller:0.15.0
|
||||
name: nginx-ingress-lb
|
||||
imagePullPolicy: Always
|
||||
readinessProbe:
|
||||
httpGet:
|
||||
path: /healthz
|
||||
port: 10254
|
||||
scheme: HTTP
|
||||
livenessProbe:
|
||||
httpGet:
|
||||
path: /healthz
|
||||
port: 10254
|
||||
scheme: HTTP
|
||||
initialDelaySeconds: 10
|
||||
timeoutSeconds: 1
|
||||
# use downward API
|
||||
env:
|
||||
- name: POD_NAME
|
||||
valueFrom:
|
||||
fieldRef:
|
||||
fieldPath: metadata.name
|
||||
- name: POD_NAMESPACE
|
||||
valueFrom:
|
||||
fieldRef:
|
||||
fieldPath: metadata.namespace
|
||||
ports:
|
||||
- containerPort: 80
|
||||
hostPort: 80
|
||||
- containerPort: 443
|
||||
hostPort: 443
|
||||
args:
|
||||
- /nginx-ingress-controller
|
||||
- --default-backend-service=$(POD_NAMESPACE)/nginx-errors
|
||||
securityContext:
|
||||
runAsNonRoot: false
|
||||
|
|
@ -42,6 +42,3 @@ $ kubectl exec nginx-ingress-controller-v1ppm cat /etc/nginx/nginx.conf
|
|||
}
|
||||
....
|
||||
```
|
||||
|
||||
|
||||
|
||||
|
|
|
|||
Binary file not shown.
|
Before Width: | Height: | Size: 59 KiB |
|
|
@ -1,103 +0,0 @@
|
|||
# Custom VTS metrics with Prometheus
|
||||
|
||||
This example aims to demonstrate the deployment of an nginx ingress controller and use a ConfigMap to enable [nginx vts module](https://github.com/vozlt/nginx-module-vts
|
||||
) to export metrics in prometheus format.
|
||||
|
||||
## vts-metrics
|
||||
|
||||
Vts-metrics export NGINX metrics. To deploy all the files simply run `kubectl apply -f nginx`. A deployment and service will be
|
||||
created which already has a `prometheus.io/scrape: 'true'` annotation and if you added
|
||||
the recommended Prometheus service-endpoint scraping [configuration](https://raw.githubusercontent.com/prometheus/prometheus/master/documentation/examples/prometheus-kubernetes.yml),
|
||||
Prometheus will scrape it automatically and you start using the generated metrics right away.
|
||||
|
||||
## Custom configuration
|
||||
|
||||
```console
|
||||
apiVersion: v1
|
||||
data:
|
||||
enable-vts-status: "true"
|
||||
kind: ConfigMap
|
||||
metadata:
|
||||
name: nginx-configuration
|
||||
namespace: ingress-nginx
|
||||
labels:
|
||||
app: ingress-nginx
|
||||
```
|
||||
|
||||
```console
|
||||
$ kubectl apply -f nginx-vts-metrics-conf.yaml
|
||||
```
|
||||
|
||||
## Result
|
||||
|
||||
Check whether the ingress controller successfully generated the NGINX vts status:
|
||||
|
||||
```console
|
||||
$ kubectl exec nginx-ingress-controller-873061567-4n3k2 -n ingress-nginx cat /etc/nginx/nginx.conf|grep vhost_traffic_status_display
|
||||
vhost_traffic_status_display;
|
||||
vhost_traffic_status_display_format html;
|
||||
```
|
||||
|
||||
### NGINX vts dashboard
|
||||
|
||||
The vts dashboard provides real time metrics.
|
||||
|
||||
|
||||
|
||||
Because the vts port it's not yet exposed, you should forward the controller port to see it.
|
||||
|
||||
```console
|
||||
$ kubectl port-forward $(kubectl get pods --selector=k8s-app=nginx-ingress-controller -n ingress-nginx --output=jsonpath={.items..metadata.name}) -n ingress-nginx 18080
|
||||
```
|
||||
|
||||
Now open the url [http://localhost:18080/nginx_status](http://localhost:18080/nginx_status) in your browser.
|
||||
|
||||
### Prometheus metrics output
|
||||
|
||||
NGINX Ingress controller already has a parser to convert vts metrics to Prometheus format. It exports prometheus metrics to the address `:10254/metrics`.
|
||||
|
||||
```console
|
||||
$ kubectl exec -ti -n ingress-nginx $(kubectl get pods --selector=k8s-app=nginx-ingress-controller -n kube-system --output=jsonpath={.items..metadata.name}) curl localhost:10254/metrics
|
||||
ingress_controller_ssl_expire_time_seconds{host="foo.bar.com"} -6.21355968e+10
|
||||
# HELP ingress_controller_success Cumulative number of Ingress controller reload operations
|
||||
# TYPE ingress_controller_success counter
|
||||
ingress_controller_success{count="reloads"} 3
|
||||
# HELP nginx_bytes_total Nginx bytes count
|
||||
# TYPE nginx_bytes_total counter
|
||||
nginx_bytes_total{direction="in",ingress_class="nginx",namespace="",server_zone="*"} 3708
|
||||
nginx_bytes_total{direction="in",ingress_class="nginx",namespace="",server_zone="_"} 3708
|
||||
nginx_bytes_total{direction="out",ingress_class="nginx",namespace="",server_zone="*"} 5256
|
||||
nginx_bytes_total{direction="out",ingress_class="nginx",namespace="",server_zone="_"} 5256
|
||||
```
|
||||
|
||||
### Customize metrics
|
||||
|
||||
The default [vts vhost key](https://github.com/vozlt/nginx-module-vts#vhost_traffic_status_filter_by_set_key) is `$geoip_country_code country::*` that expose metrics grouped by server and country code. The example below show how to have metrics grouped by server and server path.
|
||||
|
||||
|
||||
|
||||
## NGINX custom configuration ( http level )
|
||||
|
||||
```
|
||||
apiVersion: v1
|
||||
kind: ConfigMap
|
||||
data:
|
||||
enable-vts-status: "true"
|
||||
vts-default-filter-key: "$server_name"
|
||||
...
|
||||
```
|
||||
|
||||
## Customize ingress
|
||||
|
||||
```
|
||||
apiVersion: extensions/v1beta1
|
||||
kind: Ingress
|
||||
metadata:
|
||||
annotations:
|
||||
nginx.ingress.kubernetes.io/vts-filter-key: $uri $server_name
|
||||
name: ingress
|
||||
```
|
||||
|
||||
## Result
|
||||
|
||||
|
||||
Binary file not shown.
|
Before Width: | Height: | Size: 969 KiB |
Binary file not shown.
|
Before Width: | Height: | Size: 451 KiB |
Binary file not shown.
|
Before Width: | Height: | Size: 244 KiB |
|
|
@ -1,9 +0,0 @@
|
|||
apiVersion: v1
|
||||
data:
|
||||
enable-vts-status: "true"
|
||||
kind: ConfigMap
|
||||
metadata:
|
||||
name: nginx-configuration
|
||||
namespace: ingress-nginx
|
||||
labels:
|
||||
app: ingress-nginx
|
||||
|
|
@ -14,7 +14,7 @@ spec:
|
|||
spec:
|
||||
containers:
|
||||
- name: http-svc
|
||||
image: gcr.io/google_containers/echoserver:1.8
|
||||
image: gcr.io/kubernetes-e2e-test-images/echoserver:2.1
|
||||
ports:
|
||||
- containerPort: 8080
|
||||
env:
|
||||
|
|
|
|||
|
|
@ -13,10 +13,9 @@ Auth | [OAuth external auth](auth/oauth-external-auth/README.md) | TODO | TODO
|
|||
Customization | [Configuration snippets](customization/configuration-snippets/README.md) | customize nginx location configuration using annotations | Advanced
|
||||
Customization | [Custom configuration](customization/custom-configuration/README.md) | TODO | TODO
|
||||
Customization | [Custom DH parameters for perfect forward secrecy](customization/ssl-dh-param/README.md) | TODO | TODO
|
||||
Customization | [Custom errors](customization/custom-errors/README.md) | TODO | TODO
|
||||
Customization | [Custom errors](customization/custom-errors/README.md) | serve custom error pages from the default backend | Intermediate
|
||||
Customization | [Custom headers](customization/custom-headers/README.md) | set custom headers before sending traffic to backends | Advanced
|
||||
Customization | [Custom upstream check](customization/custom-upstream-check/README.md) | TODO | TODO
|
||||
Customization | [Custom VTS metrics with Prometheus](customization/custom-vts-metrics-prometheus/README.md) | TODO | TODO
|
||||
Customization | [External authentication with response header propagation](customization/external-auth-headers/README.md) | TODO | TODO
|
||||
Customization | [Sysctl tuning](customization/sysctl/README.md) | TODO | TODO
|
||||
Features | [Rewrite](rewrite/README.md) | TODO | TODO
|
||||
|
|
|
|||
|
|
@ -58,7 +58,7 @@ spec:
|
|||
spec:
|
||||
containers:
|
||||
- name: http-svc
|
||||
image: gcr.io/google_containers/echoserver:1.8
|
||||
image: gcr.io/kubernetes-e2e-test-images/echoserver:2.1
|
||||
ports:
|
||||
- containerPort: 8080
|
||||
env:
|
||||
|
|
|
|||
|
|
@ -21,7 +21,7 @@ spec:
|
|||
# hostNetwork: true
|
||||
terminationGracePeriodSeconds: 60
|
||||
containers:
|
||||
- image: quay.io/kubernetes-ingress-controller/nginx-ingress-controller:0.15.0
|
||||
- image: quay.io/kubernetes-ingress-controller/nginx-ingress-controller:0.18.0
|
||||
name: nginx-ingress-controller
|
||||
readinessProbe:
|
||||
httpGet:
|
||||
|
|
@ -53,5 +53,3 @@ spec:
|
|||
- /nginx-ingress-controller
|
||||
- --default-backend-service=$(POD_NAMESPACE)/default-http-backend
|
||||
- --publish-service=$(POD_NAMESPACE)/nginx-ingress-lb
|
||||
securityContext:
|
||||
runAsNonRoot: false
|
||||
|
|
|
|||
|
|
@ -1,16 +1,16 @@
|
|||
# How it works
|
||||
|
||||
The objective of this document explains how the NGINX Ingress controller works, in particular how the NGINX model is built and why we need a one.
|
||||
The objective of this document is to explain how the NGINX Ingress controller works, in particular how the NGINX model is built and why we need one.
|
||||
|
||||
## NGINX configuration
|
||||
|
||||
The goal of this Ingress controller is the assembly of a configuration file (nginx.conf). The main implication of this requirement is the need to reload NGINX after any change in the configuration file.
|
||||
The goal of this Ingress controller is the assembly of a configuration file (nginx.conf). The main implication of this requirement is the need to reload NGINX after any change in the configuration file. _Though it is important to note that we don't reload Nginx on changes that impact only an `upstream` configuration (i.e Endpoints change when you deploy your app)_. We use https://github.com/openresty/lua-nginx-module to achieve this. Check [below](#avoiding-reloads-on-endpoints-changes) to learn more about how it's done.
|
||||
|
||||
## NGINX model
|
||||
|
||||
Usually, a Kubernetes Controller utilizes the [synchronization loop pattern](1) to check if the desired state in the controller is updated or a change is required. To this purpose, we need to build a model using different objects from the cluster, in particular (in no special order) Ingresses, Services, Endpoints, Secrets, and Configmaps to generate a point in time configuration file that reflects the state of the cluster.
|
||||
|
||||
To get this object from the cluster, we use [Kubernetes Informers](2), in particular, `FilteredSharedInformer`. This informers allows reacting to changes in using [callbacks](3) to individual changes when a new object is added, modified or removed. Unfortunately, there is no way to know if a particular change is going to affect the final configuration file. Therefore on every change, we have to rebuild a new model from scratch based on the state of cluster and compare it to the current model. If the new model equals to the current one, then we avoid generating a new NGINX configuration and [trigger a reload](7). Otherwise, we create a new NGINX configuration based on the new model, replace the current model and [trigger a reload](7).
|
||||
To get this object from the cluster, we use [Kubernetes Informers](2), in particular, `FilteredSharedInformer`. This informers allows reacting to changes in using [callbacks](3) to individual changes when a new object is added, modified or removed. Unfortunately, there is no way to know if a particular change is going to affect the final configuration file. Therefore on every change, we have to rebuild a new model from scratch based on the state of cluster and compare it to the current model. If the new model equals to the current one, then we avoid generating a new NGINX configuration and triggering a reload. Otherwise, we check if the difference is only about Endpoints. If so we then send the new list of Endpoints to a Lua handler running inside Nginx using HTTP POST request and again avoid generating a new NGINX configuration and triggering a reload. If the difference between running and new model is about more than just Endpoints we create a new NGINX configuration based on the new model, replace the current model and trigger a reload.
|
||||
|
||||
One of the uses of the model is to avoid unnecessary reloads when there's no change in the state and to detect conflicts in definitions.
|
||||
|
||||
|
|
@ -39,16 +39,21 @@ The next list describes the scenarios when a reload is required:
|
|||
|
||||
- New Ingress Resource Created.
|
||||
- TLS section is added to existing Ingress.
|
||||
- Change in Ingress annotations.
|
||||
- Change in Ingress annotations that impacts more than just upstream configuration. For instance `load-balance` annotation does not require a reload.
|
||||
- A path is added/removed from an Ingress.
|
||||
- An Ingress, Service, Secret is removed.
|
||||
- Some missing referenced object from the Ingress is available, like a Service, Secret or Endpoint.
|
||||
- Some missing referenced object from the Ingress is available, like a Service or Secret.
|
||||
- A Secret is updated.
|
||||
|
||||
## Avoiding reloads
|
||||
|
||||
In some cases, it is possible to avoid reloads, in particular when there is a change in the endpoints, i.e., a pod is started or replaced. It is out of the scope of this Ingress controller to remove reloads completely. This would require an incredible amount of work and at some point makes no sense. This can change only if NGINX changes the way new configurations are read, basically, new changes do not replace worker processes.
|
||||
|
||||
### Avoiding reloads on Endpoints changes
|
||||
On every endpoint change the controller fetches endpoints from all the services it sees and generates corresponding Backend objects. It then sends these objects to a Lua handler running inside Nginx. The Lua code in turn stores those backends in a shared memory zone. Then for every request Lua code running in [`balancer_by_lua`](https://github.com/openresty/lua-resty-core/blob/master/lib/ngx/balancer.md) context detects what endpoints it should choose upstream peer from and applies the configured load balancing algorithm to choose the peer. Then Nginx takes care of the rest. This way we avoid reloading Nginx on endpoint changes. _Note_ that this includes annotation changes that affects only `upstream` configuration in Nginx as well.
|
||||
|
||||
In a relatively big clusters with frequently deploying apps this feature saves significant number of Nginx reloads which can otherwise affect response latency, load balancing quality (after every reload Nginx resets the state of load balancing) and so on.
|
||||
|
||||
[0]: https://github.com/openresty/lua-nginx-module/pull/1259
|
||||
[1]: https://coreos.com/kubernetes/docs/latest/replication-controller.html#the-reconciliation-loop-in-detail
|
||||
[2]: https://godoc.org/k8s.io/client-go/informers#NewFilteredSharedInformerFactory
|
||||
|
|
@ -56,4 +61,4 @@ In some cases, it is possible to avoid reloads, in particular when there is a ch
|
|||
[4]: https://github.com/kubernetes/ingress-nginx/blob/master/internal/task/queue.go#L38
|
||||
[5]: https://golang.org/pkg/sync/#Mutex
|
||||
[6]: https://github.com/kubernetes/ingress-nginx/blob/master/rootfs/etc/nginx/template/nginx.tmpl
|
||||
[7]: http://nginx.org/en/docs/beginners_guide.html#control
|
||||
[7]: http://nginx.org/en/docs/beginners_guide.html#control
|
||||
|
|
|
|||
BIN
docs/images/grafana.png
Normal file
BIN
docs/images/grafana.png
Normal file
Binary file not shown.
|
After Width: | Height: | Size: 84 KiB |
BIN
docs/images/jaeger-demo.png
Normal file
BIN
docs/images/jaeger-demo.png
Normal file
Binary file not shown.
|
After Width: | Height: | Size: 141 KiB |
BIN
docs/images/prometheus-dashboard.png
Normal file
BIN
docs/images/prometheus-dashboard.png
Normal file
Binary file not shown.
|
After Width: | Height: | Size: 112 KiB |
|
|
@ -1,14 +0,0 @@
|
|||
# Ingress Controller Catalog
|
||||
|
||||
This is a non-comprehensive list of existing ingress controllers.
|
||||
|
||||
* [Dummy controller backend](/examples/custom-controller)
|
||||
* [HAProxy Ingress controller](https://github.com/jcmoraisjr/haproxy-ingress)
|
||||
* [Linkerd](https://linkerd.io/config/0.9.1/linkerd/index.html#ingress-identifier)
|
||||
* [traefik](https://docs.traefik.io/configuration/backends/kubernetes/)
|
||||
* [AWS Application Load Balancer Ingress Controller](https://github.com/coreos/alb-ingress-controller)
|
||||
* [kube-ingress-aws-controller](https://github.com/zalando-incubator/kube-ingress-aws-controller)
|
||||
* [Voyager: HAProxy Ingress Controller](https://github.com/appscode/voyager)
|
||||
* [External Nginx Ingress Controller](https://github.com/unibet/ext_nginx)
|
||||
* [Heptio Contour controller](https://github.com/heptio/contour)
|
||||
* [LemonLDAP::NG kubernetes controller](https://github.com/lemonldap-ng-controller/lemonldap-ng-controller) adds WebSSO, Access Management and Identity Federation to NGINX Ingress Controller
|
||||
|
|
@ -1,48 +1,48 @@
|
|||
# Command line arguments
|
||||
|
||||
The following command line arguments are accepted by the main controller executable.
|
||||
The following command line arguments are accepted by the Ingress controller executable.
|
||||
|
||||
They are set in the container spec of the `nginx-ingress-controller` Deployment object (see `deploy/with-rbac.yaml` or `deploy/without-rbac.yaml`).
|
||||
They are set in the container spec of the `nginx-ingress-controller` Deployment manifest (see `deploy/with-rbac.yaml` or `deploy/without-rbac.yaml`).
|
||||
|
||||
|
||||
| Argument | Description |
|
||||
|----------|-------------|
|
||||
| `--alsologtostderr` | log to standard error as well as files |
|
||||
| `--annotations-prefix string` | Prefix of the ingress annotations. (default "nginx.ingress.kubernetes.io") |
|
||||
| `--apiserver-host string` | The address of the Kubernetes Apiserver to connect to in the format of protocol://address:port, e.g., http://localhost:8080. If not specified, the assumption is that the binary runs inside a Kubernetes cluster and local discovery is attempted. |
|
||||
| `--configmap string` | Name of the ConfigMap that contains the custom configuration to use |
|
||||
| `--default-backend-service string` | Service used to serve a 404 page for the default backend. Takes the form namespace/name. The controller uses the first node port of this Service for the default backend. |
|
||||
| `--default-server-port int` | Default port to use for exposing the default server (catch all) (default 8181) |
|
||||
| `--default-ssl-certificate string` | Name of the secret that contains a SSL certificate to be used as default for a HTTPS catch-all server. Takes the form <namespace>/<secret name>. |
|
||||
| `--election-id string` | Election id to use for status update. (default "ingress-controller-leader") |
|
||||
| `--enable-dynamic-configuration` | When enabled controller will try to avoid Nginx reloads as much as possible by using Lua. Disabled by default. |
|
||||
| `--enable-ssl-chain-completion` | Defines if the nginx ingress controller should check the secrets for missing intermediate CA certificates. If the certificate contain issues chain issues is not possible to enable OCSP. Default is true. (default true) |
|
||||
| `--enable-ssl-passthrough` | Enable SSL passthrough feature. Default is disabled |
|
||||
| `--force-namespace-isolation` | Force namespace isolation. This flag is required to avoid the reference of secrets or configmaps located in a different namespace than the specified in the flag --watch-namespace. |
|
||||
| `--health-check-path string` | Defines the URL to be used as health check inside in the default server in NGINX. (default "/healthz") |
|
||||
| `--healthz-port int` | port for healthz endpoint. (default 10254) |
|
||||
| `--http-port int` | Indicates the port to use for HTTP traffic (default 80) |
|
||||
| `--https-port int` | Indicates the port to use for HTTPS traffic (default 443) |
|
||||
| `--ingress-class string` | Name of the ingress class to route through this controller. |
|
||||
| `--kubeconfig string` | Path to kubeconfig file with authorization and master location information. |
|
||||
| `--log_backtrace_at traceLocation` | when logging hits line file:N, emit a stack trace (default :0) |
|
||||
| `--log_dir string` | If non-empty, write log files in this directory |
|
||||
| `--logtostderr` | log to standard error instead of files (default true) |
|
||||
| `--profiling` | Enable profiling via web interface host:port/debug/pprof/ (default true) |
|
||||
| `--publish-service string` | Service fronting the ingress controllers. Takes the form namespace/name. The controller will set the endpoint records on the ingress objects to reflect those on the service. |
|
||||
| `--publish-status-address string` | User customized address to be set in the status of ingress resources. The controller will set the endpoint records on the ingress using this address. |
|
||||
| `--report-node-internal-ip-address` | Defines if the nodes IP address to be returned in the ingress status should be the internal instead of the external IP address |
|
||||
| `--sort-backends` | Defines if backends and its endpoints should be sorted |
|
||||
| `--ssl-passtrough-proxy-port int` | Default port to use internally for SSL when SSL Passthgough is enabled (default 442) |
|
||||
| `--status-port int` | Indicates the TCP port to use for exposing the nginx status page (default 18080) |
|
||||
| `--stderrthreshold severity` | logs at or above this threshold go to stderr (default 2) |
|
||||
| `--sync-period duration` | Relist and confirm cloud resources this often. Default is 10 minutes (default 10m0s) |
|
||||
| `--sync-rate-limit float32` | Define the sync frequency upper limit (default 0.3) |
|
||||
| `--tcp-services-configmap string` | Name of the ConfigMap that contains the definition of the TCP services to expose. The key in the map indicates the external port to be used. The value is the name of the service with the format namespace/serviceName and the port of the service could be a number of the name of the port. The ports 80 and 443 are not allowed as external ports. This ports are reserved for the backend |
|
||||
| `--udp-services-configmap string` | Name of the ConfigMap that contains the definition of the UDP services to expose. The key in the map indicates the external port to be used. The value is the name of the service with the format namespace/serviceName and the port of the service could be a number of the name of the port. |
|
||||
| `--update-status` | Indicates if the ingress controller should update the Ingress status IP/hostname. Default is true (default true) |
|
||||
| `--update-status-on-shutdown` | Indicates if the ingress controller should update the Ingress status IP/hostname when the controller is being stopped. Default is true (default true) |
|
||||
| `-v`, `--v Level` | log level for V logs |
|
||||
| `--version` | Shows release information about the NGINX Ingress controller |
|
||||
| `--vmodule moduleSpec` | comma-separated list of pattern=N settings for file-filtered logging |
|
||||
| `--watch-namespace string` | Namespace to watch for Ingress. Default is to watch all namespaces |
|
||||
| --alsologtostderr | log to standard error as well as files |
|
||||
| --annotations-prefix string | Prefix of the Ingress annotations specific to the NGINX controller. (default "nginx.ingress.kubernetes.io") |
|
||||
| --apiserver-host string | Address of the Kubernetes API server. Takes the form "protocol://address:port". If not specified, it is assumed the program runs inside a Kubernetes cluster and local discovery is attempted. |
|
||||
| --configmap string | Name of the ConfigMap containing custom global configurations for the controller. |
|
||||
| --default-backend-service string | Service used to serve HTTP requests not matching any known server name (catch-all). Takes the form "namespace/name". The controller configures NGINX to forward requests to the first port of this Service. |
|
||||
| --default-server-port int | Port to use for exposing the default server (catch-all). (default 8181) |
|
||||
| --default-ssl-certificate string | Secret containing a SSL certificate to be used by the default HTTPS server (catch-all). Takes the form "namespace/name". |
|
||||
| --election-id string | Election id to use for Ingress status updates. (default "ingress-controller-leader") |
|
||||
| --enable-dynamic-configuration | Dynamically refresh backends on topology changes instead of reloading NGINX. Feature backed by OpenResty Lua libraries. (enabled by default) |
|
||||
| --enable-ssl-chain-completion | Autocomplete SSL certificate chains with missing intermediate CA certificates. A valid certificate chain is required to enable OCSP stapling. Certificates uploaded to Kubernetes must have the "Authority Information Access" X.509 v3 extension for this to succeed. (default true) |
|
||||
| --enable-ssl-passthrough | Enable SSL Passthrough. |
|
||||
| --force-namespace-isolation | Force namespace isolation. Prevents Ingress objects from referencing Secrets and ConfigMaps located in a different namespace than their own. May be used together with watch-namespace. |
|
||||
| --health-check-path string | URL path of the health check endpoint. Configured inside the NGINX status server. All requests received on the port defined by the healthz-port parameter are forwarded internally to this path. (default "/healthz") |
|
||||
| --healthz-port int | Port to use for the healthz endpoint. (default 10254) |
|
||||
| --http-port int | Port to use for servicing HTTP traffic. (default 80) |
|
||||
| --https-port int | Port to use for servicing HTTPS traffic. (default 443) |
|
||||
| --ingress-class string | Name of the ingress class this controller satisfies. The class of an Ingress object is set using the annotation "kubernetes.io/ingress.class". All ingress classes are satisfied if this parameter is left empty. |
|
||||
| --kubeconfig string | Path to a kubeconfig file containing authorization and API server information. |
|
||||
| --log_backtrace_at traceLocation | when logging hits line file:N, emit a stack trace (default :0) |
|
||||
| --log_dir string | If non-empty, write log files in this directory |
|
||||
| --logtostderr | log to standard error instead of files (default true) |
|
||||
| --profiling | Enable profiling via web interface host:port/debug/pprof/ (default true) |
|
||||
| --publish-service string | Service fronting the Ingress controller. Takes the form "namespace/name". When used together with update-status, the controller mirrors the address of this service's endpoints to the load-balancer status of all Ingress objects it satisfies. |
|
||||
| --publish-status-address string | Customized address to set as the load-balancer status of Ingress objects this controller satisfies. Requires the update-status parameter. |
|
||||
| --report-node-internal-ip-address | Set the load-balancer status of Ingress objects to internal Node addresses instead of external. Requires the update-status parameter. |
|
||||
| --sort-backends | Sort servers inside NGINX upstreams. |
|
||||
| --ssl-passthrough-proxy-port int | Port to use internally for SSL Passthrough. (default 442) |
|
||||
| --status-port int | Port to use for exposing NGINX status pages. (default 18080) |
|
||||
| --stderrthreshold severity | logs at or above this threshold go to stderr (default 2) |
|
||||
| --sync-period duration | Period at which the controller forces the repopulation of its local object stores. (default is 0) |
|
||||
| --sync-rate-limit float32 | Define the sync frequency upper limit (default 0.3) |
|
||||
| --tcp-services-configmap string | Name of the ConfigMap containing the definition of the TCP services to expose. The key in the map indicates the external port to be used. The value is a reference to a Service in the form "namespace/name:port", where "port" can either be a port number or name. TCP ports 80 and 443 are reserved by the controller for servicing HTTP traffic. |
|
||||
| --udp-services-configmap string | Name of the ConfigMap containing the definition of the UDP services to expose. The key in the map indicates the external port to be used. The value is a reference to a Service in the form "namespace/name:port", where "port" can either be a port name or number.
|
||||
| --update-status | Update the load-balancer status of Ingress objects this controller satisfies. Requires setting the publish-service parameter to a valid Service reference. (default true) |
|
||||
| --update-status-on-shutdown | Update the load-balancer status of Ingress objects when the controller shuts down. Requires the update-status parameter. (default true) |
|
||||
| --v Level | log level for V logs |
|
||||
| --version | Show release information about the NGINX Ingress controller and exit. |
|
||||
| --vmodule moduleSpec | comma-separated list of pattern=N settings for file-filtered logging |
|
||||
| --watch-namespace string | Namespace the controller watches for updates to Kubernetes objects. This includes Ingresses, Services and all configuration resources. All namespaces are watched if this parameter is left empty. |
|
||||
|
|
|
|||
|
|
@ -1,19 +1,30 @@
|
|||
# Custom errors
|
||||
|
||||
In case of an error in a request the body of the response is obtained from the `default backend`.
|
||||
Each request to the default backend includes two headers:
|
||||
When the [`custom-http-errors`][cm-custom-http-errors] option is enabled, the Ingress controller configures NGINX so
|
||||
that it passes several HTTP headers down to its `default-backend` in case of error:
|
||||
|
||||
- `X-Code` indicates the HTTP code to be returned to the client.
|
||||
- `X-Format` the value of the `Accept` header.
|
||||
| Header | Value |
|
||||
| ---------------- | ------------------------------------------------ |
|
||||
| `X-Code` | HTTP status code retuned by the request |
|
||||
| `X-Format` | Value of the `Accept` header sent by the client |
|
||||
| `X-Original-URI` | URI that caused the error |
|
||||
| `X-Namespace` | Namespace where the backend Service is located |
|
||||
| `X-Ingress-Name` | Name of the Ingress where the backend is defined |
|
||||
| `X-Service-Name` | Name of the Service backing the backend |
|
||||
| `X-Service-Port` | Port number of the Service backing the backend |
|
||||
|
||||
A custom error backend can use this information to return the best possible representation of an error page. For
|
||||
example, if the value of the `Accept` header send by the client was `application/json`, a carefully crafted backend
|
||||
could decide to return the error payload as a JSON document instead of HTML.
|
||||
|
||||
!!! Important
|
||||
The custom backend must return the correct HTTP status code to be returned. NGINX does not change the response from the custom default backend.
|
||||
The custom backend is expected to return the correct HTTP status code instead of `200`. NGINX does not change
|
||||
the response from the custom default backend.
|
||||
|
||||
Using these two headers it's possible to use a custom backend service like [this one](https://github.com/kubernetes/ingress-nginx/tree/master/images/custom-error-pages) that inspects each request and returns a custom error page with the format expected by the client. Please check the example [custom-errors](https://github.com/kubernetes/ingress-nginx/tree/master/docs/examples/customization/custom-errors).
|
||||
An example of such custom backend is available inside the source repository at [images/custom-error-pages][img-custom-error-pages].
|
||||
|
||||
NGINX sends additional headers that can be used to build custom response:
|
||||
See also the [Custom errors][example-custom-errors] example.
|
||||
|
||||
- X-Original-URI
|
||||
- X-Namespace
|
||||
- X-Ingress-Name
|
||||
- X-Service-Name
|
||||
[cm-custom-http-errors]: ./nginx-configuration/configmap.md#custom-http-errors
|
||||
[img-custom-error-pages]: https://github.com/kubernetes/ingress-nginx/tree/master/images/custom-error-pages
|
||||
[example-custom-errors]: ../examples/customization/custom-errors
|
||||
|
|
|
|||
|
|
@ -12,7 +12,8 @@ The next example shows how to expose the service `example-go` running in the nam
|
|||
apiVersion: v1
|
||||
kind: ConfigMap
|
||||
metadata:
|
||||
name: tcp-configmap-example
|
||||
name: tcp-services
|
||||
namespace: ingress-nginx
|
||||
data:
|
||||
9000: "default/example-go:8080"
|
||||
```
|
||||
|
|
@ -24,7 +25,8 @@ The next example shows how to expose the service `kube-dns` running in the names
|
|||
apiVersion: v1
|
||||
kind: ConfigMap
|
||||
metadata:
|
||||
name: udp-configmap-example
|
||||
name: udp-services
|
||||
namespace: ingress-nginx
|
||||
data:
|
||||
53: "kube-system/kube-dns:53"
|
||||
```
|
||||
|
|
|
|||
90
docs/user-guide/monitoring.md
Normal file
90
docs/user-guide/monitoring.md
Normal file
|
|
@ -0,0 +1,90 @@
|
|||
# Prometheus and Grafana installation
|
||||
|
||||
This tutorial will show you how to install [Prometheus](https://prometheus.io/) and [Grafana](https://grafana.com/) for scraping the metrics of the NGINX Ingress controller.
|
||||
|
||||
!!! Important: this example uses `emptyDir` volumes for Prometheus and Grafana. This means once the pod gets terminated you will lose all the data.
|
||||
|
||||
## Before You Begin
|
||||
|
||||
The NGINX Ingress controller should already be deployed according to the deployment instructions [here](../deploy/index.md).
|
||||
|
||||
Note that the yaml files used in this tutorial are stored in the [deploy/monitoring](https://github.com/kubernetes/ingress-nginx/tree/master/deploy/monitoring) folder of the GitHub repository [kubernetes/ingress-nginx](https://github.com/kubernetes/ingress-nginx).
|
||||
|
||||
## Deploy and configure Prometheus Server
|
||||
|
||||
The Prometheus server must be configured so that it can discover endpoints of services. If a Prometheus server is already running in the cluster and if it is configured in a way that it can find the ingress controller pods, no extra configuration is needed.
|
||||
|
||||
If there is no existing Prometheus server running, the rest of this tutorial will guide you through the steps needed to deploy a properly configured Prometheus server.
|
||||
|
||||
Running the following command deploys the prometheus configuration in Kubernetes:
|
||||
|
||||
```console
|
||||
kubectl create -f https://raw.githubusercontent.com/kubernetes/ingress-nginx/master/deploy/monitoring/configuration.yaml
|
||||
configmap "prometheus-configuration" created
|
||||
```
|
||||
|
||||
Running the following command deploys prometheus in Kubernetes:
|
||||
|
||||
```console
|
||||
kubectl create -f https://raw.githubusercontent.com/kubernetes/ingress-nginx/master/deploy/monitoring/prometheus.yaml
|
||||
clusterrole "prometheus-server" created
|
||||
serviceaccount "prometheus-server" created
|
||||
clusterrolebinding "prometheus-server" created
|
||||
deployment "prometheus-server" created
|
||||
service "prometheus-service" created
|
||||
```
|
||||
|
||||
### Prometheus Dashboard
|
||||
|
||||
Open Prometheus dashboard in a web browser:
|
||||
|
||||
```console
|
||||
kubectl get svc -n ingress-nginx
|
||||
NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE
|
||||
default-http-backend ClusterIP 10.103.59.201 <none> 80/TCP 3d
|
||||
ingress-nginx NodePort 10.97.44.72 <none> 80:30100/TCP,443:30154/TCP,10254:32049/TCP 5h
|
||||
prometheus NodePort 10.98.233.86 <none> 9090:32630/TCP 1m
|
||||
```
|
||||
|
||||
Obtain the IP address of the nodes in the running cluster:
|
||||
|
||||
```console
|
||||
kubectl get nodes -o wide
|
||||
```
|
||||
|
||||
In some cases where the node only have internal IP adresses we need to execute:
|
||||
|
||||
```console
|
||||
kubectl get nodes --selector=kubernetes.io/role!=master -o jsonpath={.items[*].status.addresses[?\(@.type==\"InternalIP\"\)].address}
|
||||
10.192.0.2 10.192.0.3 10.192.0.4
|
||||
```
|
||||
|
||||
Open your browser and visit the following URL: _http://{node IP address}:{prometheus-svc-nodeport}_ to load the Prometheus Dashboard.
|
||||
|
||||
According to the above example, this URL will be http://10.192.0.3:32630
|
||||
|
||||
|
||||
|
||||
### Grafana
|
||||
|
||||
```console
|
||||
kubectl create -f https://raw.githubusercontent.com/kubernetes/ingress-nginx/master/deploy/monitoring/grafana.yaml
|
||||
```
|
||||
|
||||
```console
|
||||
kubectl get svc -n ingress-nginx
|
||||
NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE
|
||||
default-http-backend ClusterIP 10.103.59.201 <none> 80/TCP 3d
|
||||
ingress-nginx NodePort 10.97.44.72 <none> 80:30100/TCP,443:30154/TCP,10254:32049/TCP 5h
|
||||
prometheus NodePort 10.98.233.86 <none> 9090:32630/TCP 10m
|
||||
grafana NodePort 10.98.233.86 <none> 9090:31086/TCP 10m
|
||||
```
|
||||
|
||||
Open your browser and visit the following URL: _http://{node IP address}:{grafana-svc-nodeport}_ to load the Grafana Dashboard.
|
||||
According to the above example, this URL will be http://10.192.0.3:31086
|
||||
|
||||
The username and password is `admin`
|
||||
|
||||
After the login you can import the Grafana dashboard from _https://github.com/kubernetes/ingress-nginx/tree/master/deploy/grafana/dashboards_
|
||||
|
||||
|
||||
|
|
@ -27,6 +27,7 @@ You can add these Kubernetes annotations to specific Ingress objects to customiz
|
|||
|[nginx.ingress.kubernetes.io/auth-tls-error-page](#client-certificate-authentication)|string|
|
||||
|[nginx.ingress.kubernetes.io/auth-tls-pass-certificate-to-upstream](#client-certificate-authentication)|"true" or "false"|
|
||||
|[nginx.ingress.kubernetes.io/auth-url](#external-authentication)|string|
|
||||
|[nginx.ingress.kubernetes.io/backend-protocol](#backend-protocol)|string|HTTP,HTTPS,GRPC,GRPCS,AJP|
|
||||
|[nginx.ingress.kubernetes.io/base-url-scheme](#rewrite)|string|
|
||||
|[nginx.ingress.kubernetes.io/client-body-buffer-size](#client-body-buffer-size)|string|
|
||||
|[nginx.ingress.kubernetes.io/configuration-snippet](#configuration-snippet)|string|
|
||||
|
|
@ -43,7 +44,9 @@ You can add these Kubernetes annotations to specific Ingress objects to customiz
|
|||
|[nginx.ingress.kubernetes.io/limit-connections](#rate-limiting)|number|
|
||||
|[nginx.ingress.kubernetes.io/limit-rps](#rate-limiting)|number|
|
||||
|[nginx.ingress.kubernetes.io/permanent-redirect](#permanent-redirect)|string|
|
||||
|[nginx.ingress.kubernetes.io/permanent-redirect-code](#permanent-redirect-code)|number|
|
||||
|[nginx.ingress.kubernetes.io/proxy-body-size](#custom-max-body-size)|string|
|
||||
|[nginx.ingress.kubernetes.io/proxy-cookie-domain](#proxy-cookie-domain)|string|
|
||||
|[nginx.ingress.kubernetes.io/proxy-connect-timeout](#custom-timeouts)|number|
|
||||
|[nginx.ingress.kubernetes.io/proxy-send-timeout](#custom-timeouts)|number|
|
||||
|[nginx.ingress.kubernetes.io/proxy-read-timeout](#custom-timeouts)|number|
|
||||
|
|
@ -70,6 +73,7 @@ You can add these Kubernetes annotations to specific Ingress objects to customiz
|
|||
|[nginx.ingress.kubernetes.io/upstream-vhost](#custom-nginx-upstream-vhost)|string|
|
||||
|[nginx.ingress.kubernetes.io/whitelist-source-range](#whitelist-source-range)|CIDR|
|
||||
|[nginx.ingress.kubernetes.io/proxy-buffering](#proxy-buffering)|string|
|
||||
|[nginx.ingress.kubernetes.io/proxy-buffer-size](#proxy-buffer-size)|string|
|
||||
|[nginx.ingress.kubernetes.io/ssl-ciphers](#ssl-ciphers)|string|
|
||||
|[nginx.ingress.kubernetes.io/connection-proxy-header](#connection-proxy-header)|string|
|
||||
|[nginx.ingress.kubernetes.io/enable-access-log](#enable-access-log)|"true" or "false"|
|
||||
|
|
@ -150,7 +154,7 @@ nginx.ingress.kubernetes.io/auth-realm: "realm string"
|
|||
NGINX exposes some flags in the [upstream configuration](http://nginx.org/en/docs/http/ngx_http_upstream_module.html#upstream) that enable the configuration of each server in the upstream. The Ingress controller allows custom `max_fails` and `fail_timeout` parameters in a global context using `upstream-max-fails` and `upstream-fail-timeout` in the NGINX ConfigMap or in a particular Ingress rule. `upstream-max-fails` defaults to 0. This means NGINX will respect the container's `readinessProbe` if it is defined. If there is no probe and no values for `upstream-max-fails` NGINX will continue to send traffic to the container.
|
||||
|
||||
|
||||
!!! tip
|
||||
!!! tip
|
||||
With the default configuration NGINX will not health check your backends. Whenever the endpoints controller notices a readiness probe failure, that pod's IP will be removed from the list of endpoints. This will trigger the NGINX controller to also remove it from the upstreams.**
|
||||
|
||||
To use custom values in an Ingress rule define these annotations:
|
||||
|
|
@ -208,9 +212,9 @@ The annotations are:
|
|||
|
||||
!!! attention
|
||||
TLS with Client Authentication is **not** possible in Cloudflare and might result in unexpected behavior.
|
||||
|
||||
|
||||
Cloudflare only allows Authenticated Origin Pulls and is required to use their own certificate: [https://blog.cloudflare.com/protecting-the-origin-with-tls-authenticated-origin-pulls/](https://blog.cloudflare.com/protecting-the-origin-with-tls-authenticated-origin-pulls/)
|
||||
|
||||
|
||||
Only Authenticated Origin Pulls are allowed and can be configured by following their tutorial: [https://support.cloudflare.com/hc/en-us/articles/204494148-Setting-up-NGINX-to-use-TLS-Authenticated-Origin-Pulls](https://support.cloudflare.com/hc/en-us/articles/204494148-Setting-up-NGINX-to-use-TLS-Authenticated-Origin-Pulls)
|
||||
|
||||
|
||||
|
|
@ -364,6 +368,10 @@ To configure this setting globally for all Ingress rules, the `limit-rate-after`
|
|||
|
||||
This annotation allows to return a permanent redirect instead of sending data to the upstream. For example `nginx.ingress.kubernetes.io/permanent-redirect: https://www.google.com` would redirect everything to Google.
|
||||
|
||||
### Permanent Redirect Code
|
||||
|
||||
This annotation allows you to modify the status code used for permanent redirects. For example `nginx.ingress.kubernetes.io/permanent-redirect-code: '308'` would return your permanent-redirect with a 308.
|
||||
|
||||
### SSL Passthrough
|
||||
|
||||
The annotation `nginx.ingress.kubernetes.io/ssl-passthrough` allows to configure TLS termination in the pod and not in NGINX.
|
||||
|
|
@ -375,7 +383,9 @@ The annotation `nginx.ingress.kubernetes.io/ssl-passthrough` allows to configure
|
|||
!!! attention
|
||||
The use of this annotation requires the flag `--enable-ssl-passthrough` (By default it is disabled).
|
||||
|
||||
### Secure backends
|
||||
### Secure backends DEPRECATED (since 0.18.0)
|
||||
|
||||
Please use `nginx.ingress.kubernetes.io/backend-protocol: "HTTPS"`
|
||||
|
||||
By default NGINX uses plain HTTP to reach the services.
|
||||
Adding the annotation `nginx.ingress.kubernetes.io/secure-backends: "true"` in the Ingress rule changes the protocol to HTTPS.
|
||||
|
|
@ -464,6 +474,12 @@ To use custom values in an Ingress rule define these annotation:
|
|||
nginx.ingress.kubernetes.io/proxy-body-size: 8m
|
||||
```
|
||||
|
||||
### Proxy cookie domain
|
||||
|
||||
Sets a text that [should be changed in the domain attribute](http://nginx.org/en/docs/http/ngx_http_proxy_module.html#proxy_cookie_domain) of the "Set-Cookie" header fields of a proxied server response.
|
||||
|
||||
To configure this setting globally for all Ingress rules, the `proxy-cookie-domain` value may be set in the [NGINX ConfigMap][configmap].
|
||||
|
||||
### Proxy buffering
|
||||
|
||||
Enable or disable proxy buffering [`proxy_buffering`](http://nginx.org/en/docs/http/ngx_http_proxy_module.html#proxy_buffering).
|
||||
|
|
@ -476,6 +492,16 @@ To use custom values in an Ingress rule define these annotation:
|
|||
nginx.ingress.kubernetes.io/proxy-buffering: "on"
|
||||
```
|
||||
|
||||
### Proxy buffer size
|
||||
|
||||
Sets the size of the buffer [`proxy_buffer_size`](http://nginx.org/en/docs/http/ngx_http_proxy_module.html#proxy_buffer_size) used for reading the first part of the response received from the proxied server.
|
||||
By default proxy buffer size is set as "4k"
|
||||
|
||||
To configure this setting globally, set `proxy-buffer-size` in [NGINX ConfigMap][configmap]. To use custom values in an Ingress rule, define this annotation:
|
||||
```yaml
|
||||
nginx.ingress.kubernetes.io/proxy-buffer-size: "8k"
|
||||
```
|
||||
|
||||
### SSL ciphers
|
||||
|
||||
Specifies the [enabled ciphers](http://nginx.org/en/docs/http/ngx_http_ssl_module.html#ssl_ciphers).
|
||||
|
|
@ -546,7 +572,9 @@ nginx.ingress.kubernetes.io/lua-resty-waf-extra-rules: '[=[ { "access": [ { "act
|
|||
|
||||
For details on how to write WAF rules, please refer to [https://github.com/p0pr0ck5/lua-resty-waf](https://github.com/p0pr0ck5/lua-resty-waf).
|
||||
|
||||
### gRPC backend
|
||||
### gRPC backend DEPRECATED (since 0.18.0)
|
||||
|
||||
Please use `nginx.ingress.kubernetes.io/backend-protocol: "GRPC"` or `nginx.ingress.kubernetes.io/backend-protocol: "GRPCS"`
|
||||
|
||||
Since NGINX 1.13.10 it is possible to expose [gRPC services natively](http://nginx.org/en/docs/http/ngx_http_grpc_module.html)
|
||||
|
||||
|
|
@ -568,15 +596,29 @@ using the [nginx-influxdb-module](https://github.com/influxdata/nginx-influxdb-m
|
|||
nginx.ingress.kubernetes.io/enable-influxdb: "true"
|
||||
nginx.ingress.kubernetes.io/influxdb-measurement: "nginx-reqs"
|
||||
nginx.ingress.kubernetes.io/influxdb-port: "8089"
|
||||
nginx.ingress.kubernetes.io/influxdb-host: "influxdb"
|
||||
nginx.ingress.kubernetes.io/influxdb-host: "127.0.0.1"
|
||||
nginx.ingress.kubernetes.io/influxdb-server-name: "nginx-ingress"
|
||||
```
|
||||
|
||||
For the `influxdb-host` parameter you have two options:
|
||||
|
||||
To use the module in the Kubernetes Nginx ingress controller, you have two options:
|
||||
|
||||
- Use an InfluxDB server configured to enable the [UDP protocol](https://docs.influxdata.com/influxdb/v1.5/supported_protocols/udp/).
|
||||
- Use an InfluxDB server configured with the [UDP protocol](https://docs.influxdata.com/influxdb/v1.5/supported_protocols/udp/) enabled.
|
||||
- Deploy Telegraf as a sidecar proxy to the Ingress controller configured to listen UDP with the [socket listener input](https://github.com/influxdata/telegraf/tree/release-1.6/plugins/inputs/socket_listener) and to write using
|
||||
anyone of the [outputs plugins](https://github.com/influxdata/telegraf/tree/release-1.6/plugins/outputs)
|
||||
anyone of the [outputs plugins](https://github.com/influxdata/telegraf/tree/release-1.7/plugins/outputs) like InfluxDB, Apache Kafka,
|
||||
Prometheus, etc.. (recommended)
|
||||
|
||||
It's important to remember that there's no DNS resolver at this stage so you will have to configure
|
||||
an ip address to `nginx.ingress.kubernetes.io/influxdb-host`. If you deploy Influx or Telegraf as sidecar (another container in the same pod) this becomes straightforward since you can directly use `127.0.0.1`.
|
||||
|
||||
### Backend Protocol
|
||||
|
||||
Using `backend-protocol` annotations is possible to indicate how NGINX should communicate with the backend service.
|
||||
Valid Values: HTTP, HTTPS, GRPC, GRPCS and AJP
|
||||
|
||||
By default NGINX uses `HTTP`.
|
||||
|
||||
Example:
|
||||
|
||||
```yaml
|
||||
nginx.ingress.kubernetes.io/backend-protocol: "HTTPS"
|
||||
```
|
||||
|
|
|
|||
|
|
@ -44,10 +44,6 @@ The following table shows a configuration option's name, type, and the default v
|
|||
|[disable-ipv6-dns](#disable-ipv6-dns)|bool|false|
|
||||
|[enable-underscores-in-headers](#enable-underscores-in-headers)|bool|false|
|
||||
|[ignore-invalid-headers](#ignore-invalid-headers)|bool|true|
|
||||
|[enable-vts-status](#enable-vts-status)|bool|false|
|
||||
|[vts-status-zone-size](#vts-status-zone-size)|string|"10m"|
|
||||
|[vts-sum-key](#vts-sum-key)|string|"*"|
|
||||
|[vts-default-filter-key](#vts-default-filter-key)|string|"$geoip_country_code country::*"|
|
||||
|[retry-non-idempotent](#retry-non-idempotent)|bool|"false"|
|
||||
|[error-log-level](#error-log-level)|string|"notice"|
|
||||
|[http2-max-field-size](#http2-max-field-size)|string|"4k"|
|
||||
|
|
@ -62,6 +58,7 @@ The following table shows a configuration option's name, type, and the default v
|
|||
|[log-format-escape-json](#log-format-escape-json)|bool|"false"|
|
||||
|[log-format-upstream](#log-format-upstream)|string|`%v - [$the_real_ip] - $remote_user [$time_local] "$request" $status $body_bytes_sent "$http_referer" "$http_user_agent" $request_length $request_time [$proxy_upstream_name] $upstream_addr $upstream_response_length $upstream_response_time $upstream_status`|
|
||||
|[log-format-stream](#log-format-stream)|string|`[$time_local] $protocol $status $bytes_sent $bytes_received $session_time`|
|
||||
|[enable-multi-accept](#enable-multi-accept)|bool|"true"|
|
||||
|[max-worker-connections](#max-worker-connections)|int|16384|
|
||||
|[map-hash-bucket-size](#max-worker-connections)|int|64|
|
||||
|[nginx-status-ipv4-whitelist](#nginx-status-ipv4-whitelist)|[]string|"127.0.0.1"|
|
||||
|
|
@ -72,6 +69,7 @@ The following table shows a configuration option's name, type, and the default v
|
|||
|[server-name-hash-bucket-size](#server-name-hash-bucket-size)|int|`<size of the processor’s cache line>`
|
||||
|[proxy-headers-hash-max-size](#proxy-headers-hash-max-size)|int|512|
|
||||
|[proxy-headers-hash-bucket-size](#proxy-headers-hash-bucket-size)|int|64|
|
||||
|[reuse-port](#reuse-port)|bool|"true"|
|
||||
|[server-tokens](#server-tokens)|bool|"true"|
|
||||
|[ssl-ciphers](#ssl-ciphers)|string|"ECDHE-ECDSA-AES256-GCM-SHA384:ECDHE-RSA-AES256-GCM-SHA384:ECDHE-ECDSA-CHACHA20-POLY1305:ECDHE-RSA-CHACHA20-POLY1305:ECDHE-ECDSA-AES128-GCM-SHA256:ECDHE-RSA-AES128-GCM-SHA256:ECDHE-ECDSA-AES256-SHA384:ECDHE-RSA-AES256-SHA384:ECDHE-ECDSA-AES128-SHA256:ECDHE-RSA-AES128-SHA256"|
|
||||
|[ssl-ecdh-curve](#ssl-ecdh-curve)|string|"auto"|
|
||||
|
|
@ -91,11 +89,12 @@ The following table shows a configuration option's name, type, and the default v
|
|||
|[brotli-level](#brotli-level)|int|4|
|
||||
|[brotli-types](#brotli-types)|string|"application/xml+rss application/atom+xml application/javascript application/x-javascript application/json application/rss+xml application/vnd.ms-fontobject application/x-font-ttf application/x-web-app-manifest+json application/xhtml+xml application/xml font/opentype image/svg+xml image/x-icon text/css text/plain text/x-component"|
|
||||
|[use-http2](#use-http2)|bool|"true"|
|
||||
|[gzip-level](#gzip-level)|int|5|
|
||||
|[gzip-types](#gzip-types)|string|"application/atom+xml application/javascript application/x-javascript application/json application/rss+xml application/vnd.ms-fontobject application/x-font-ttf application/x-web-app-manifest+json application/xhtml+xml application/xml font/opentype image/svg+xml image/x-icon text/css text/plain text/x-component"|
|
||||
|[worker-processes](#worker-processes)|string|`<Number of CPUs>`|
|
||||
|[worker-cpu-affinity](#worker-cpu-affinity)|string|""|
|
||||
|[worker-shutdown-timeout](#worker-shutdown-timeout)|string|"10s"|
|
||||
|[load-balance](#load-balance)|string|"least_conn"|
|
||||
|[load-balance](#load-balance)|string|"round_robin"|
|
||||
|[variables-hash-bucket-size](#variables-hash-bucket-size)|int|128|
|
||||
|[variables-hash-max-size](#variables-hash-max-size)|int|2048|
|
||||
|[upstream-keepalive-connections](#upstream-keepalive-connections)|int|32|
|
||||
|
|
@ -112,15 +111,17 @@ The following table shows a configuration option's name, type, and the default v
|
|||
|[zipkin-collector-host](#zipkin-collector-host)|string|""|
|
||||
|[zipkin-collector-port](#zipkin-collector-port)|int|9411|
|
||||
|[zipkin-service-name](#zipkin-service-name)|string|"nginx"|
|
||||
|[zipkin-sample-rate](#zipkin-sample-rate)|float|1.0|
|
||||
|[jaeger-collector-host](#jaeger-collector-host)|string|""|
|
||||
|[jaeger-collector-port](#jaeger-collector-port)|int|6831|
|
||||
|[jaeger-service-name](#jaeger-service-name)|string|"nginx"|
|
||||
|[jaeger-sampler-type](#jaeger-sampler-type)|string|"const"|
|
||||
|[jaeger-sampler-param](#jaeger-sampler-param)|string|"1"|
|
||||
|[main-snippet](#main-snippet)|string|""|
|
||||
|[http-snippet](#http-snippet)|string|""|
|
||||
|[server-snippet](#server-snippet)|string|""|
|
||||
|[location-snippet](#location-snippet)|string|""|
|
||||
|[custom-http-errors](#custom-http-errors)|[]int]|[]int{}|
|
||||
|[custom-http-errors](#custom-http-errors)|[]int|[]int{}|
|
||||
|[proxy-body-size](#proxy-body-size)|string|"1m"|
|
||||
|[proxy-connect-timeout](#proxy-connect-timeout)|int|5|
|
||||
|[proxy-read-timeout](#proxy-read-timeout)|int|60|
|
||||
|
|
@ -241,32 +242,6 @@ Enables underscores in header names. _**default:**_ is disabled
|
|||
Set if header fields with invalid names should be ignored.
|
||||
_**default:**_ is enabled
|
||||
|
||||
## enable-vts-status
|
||||
|
||||
Allows the replacement of the default status page with a third party module named [nginx-module-vts](https://github.com/vozlt/nginx-module-vts).
|
||||
_**default:**_ is disabled
|
||||
|
||||
## vts-status-zone-size
|
||||
|
||||
Vts config on http level sets parameters for a shared memory zone that will keep states for various keys. The cache is shared between all worker processes. _**default:**_ 10m
|
||||
|
||||
_References:_
|
||||
[https://github.com/vozlt/nginx-module-vts#vhost_traffic_status_zone](https://github.com/vozlt/nginx-module-vts#vhost_traffic_status_zone)
|
||||
|
||||
## vts-default-filter-key
|
||||
|
||||
Vts config on http level enables the keys by user defined variable. The key is a key string to calculate traffic. The name is a group string to calculate traffic. The key and name can contain variables such as $host, $server_name. The name's group belongs to filterZones if specified. The key's group belongs to serverZones if not specified second argument name. _**default:**_ $geoip_country_code country::*
|
||||
|
||||
_References:_
|
||||
[https://github.com/vozlt/nginx-module-vts#vhost_traffic_status_filter_by_set_key](https://github.com/vozlt/nginx-module-vts#vhost_traffic_status_filter_by_set_key)
|
||||
|
||||
## vts-sum-key
|
||||
|
||||
For metrics keyed (or when using Prometheus, labeled) by server zone, this value is used to indicate metrics for all server zones combined. _**default:**_ *
|
||||
|
||||
_References:_
|
||||
[https://github.com/vozlt/nginx-module-vts#vhost_traffic_status_display_sum_key](https://github.com/vozlt/nginx-module-vts#vhost_traffic_status_display_sum_key)
|
||||
|
||||
## retry-non-idempotent
|
||||
|
||||
Since 1.9.13 NGINX will not retry non-idempotent requests (POST, LOCK, PATCH) in case of an error in the upstream server. The previous behavior can be restored using the value "true".
|
||||
|
|
@ -360,6 +335,14 @@ Please check the [log-format](log-format.md) for definition of each field.
|
|||
|
||||
Sets the nginx [stream format](https://nginx.org/en/docs/stream/ngx_stream_log_module.html#log_format).
|
||||
|
||||
## enable-multi-accept
|
||||
|
||||
If disabled, a worker process will accept one new connection at a time. Otherwise, a worker process will accept all new connections at a time.
|
||||
_**default:**_ true
|
||||
|
||||
_References:_
|
||||
[http://nginx.org/en/docs/ngx_core_module.html#multi_accept](http://nginx.org/en/docs/ngx_core_module.html#multi_accept)
|
||||
|
||||
## max-worker-connections
|
||||
|
||||
Sets the maximum number of simultaneous connections that can be opened by each [worker process](http://nginx.org/en/docs/ngx_core_module.html#worker_connections)
|
||||
|
|
@ -401,7 +384,12 @@ _References:_
|
|||
- [http://nginx.org/en/docs/hash.html](http://nginx.org/en/docs/hash.html)
|
||||
- [https://nginx.org/en/docs/http/ngx_http_proxy_module.html#proxy_headers_hash_max_size](https://nginx.org/en/docs/http/ngx_http_proxy_module.html#proxy_headers_hash_max_size)
|
||||
|
||||
## proxy-headers-hash-bucket-size
|
||||
## reuse-port
|
||||
|
||||
Instructs NGINX to create an individual listening socket for each worker process (using the SO_REUSEPORT socket option), allowing a kernel to distribute incoming connections between worker processes
|
||||
_**default:**_ true
|
||||
|
||||
## proxy-headers-hash-bucket-size
|
||||
|
||||
Sets the size of the bucket for the proxy headers hash tables.
|
||||
|
||||
|
|
@ -463,8 +451,9 @@ Enables or disables session resumption through [TLS session tickets](http://ngin
|
|||
## ssl-session-ticket-key
|
||||
|
||||
Sets the secret key used to encrypt and decrypt TLS session tickets. The value must be a valid base64 string.
|
||||
To create a ticket: `openssl rand 80 | openssl enc -A -base64`
|
||||
|
||||
[TLS session ticket-key](http://nginx.org/en/docs/http/ngx_http_ssl_module.html#ssl_session_tickets), by default, a randomly generated key is used. To create a ticket: `openssl rand 80 | base64 -w0`
|
||||
[TLS session ticket-key](http://nginx.org/en/docs/http/ngx_http_ssl_module.html#ssl_session_tickets), by default, a randomly generated key is used.
|
||||
|
||||
## ssl-session-timeout
|
||||
|
||||
|
|
@ -483,7 +472,7 @@ Enables or disables the [PROXY protocol](https://www.nginx.com/resources/admin-g
|
|||
|
||||
## proxy-protocol-header-timeout
|
||||
|
||||
Sets the timeout value for receiving the proxy-protocol headers. The default of 5 seconds prevents the TLS passthrough handler from waiting indefinetly on a dropped connection.
|
||||
Sets the timeout value for receiving the proxy-protocol headers. The default of 5 seconds prevents the TLS passthrough handler from waiting indefinitely on a dropped connection.
|
||||
_**default:**_ 5s
|
||||
|
||||
## use-gzip
|
||||
|
|
@ -516,6 +505,10 @@ _**default:**_ `application/xml+rss application/atom+xml application/javascript
|
|||
|
||||
Enables or disables [HTTP/2](http://nginx.org/en/docs/http/ngx_http_v2_module.html) support in secure connections.
|
||||
|
||||
## gzip-level
|
||||
|
||||
Sets the gzip Compression Level that will be used. _**default:**_ 5
|
||||
|
||||
## gzip-types
|
||||
|
||||
Sets the MIME types in addition to "text/html" to compress. The special value "\*" matches any MIME type. Responses with the "text/html" type are always compressed if `use-gzip` is enabled.
|
||||
|
|
@ -544,11 +537,11 @@ Sets the algorithm to use for load balancing.
|
|||
The value can either be:
|
||||
|
||||
- round_robin: to use the default round robin loadbalancer
|
||||
- least_conn: to use the least connected method
|
||||
- ip_hash: to use a hash of the server for routing.
|
||||
- ewma: to use the peak ewma method for routing (only available with `enable-dynamic-configuration` flag)
|
||||
- least_conn: to use the least connected method (_note_ that this is available only in non-dynamic mode: `--enable-dynamic-configuration=false`)
|
||||
- ip_hash: to use a hash of the server for routing (_note_ that this is available only in non-dynamic mode: `--enable-dynamic-configuration=false`, but alternatively you can consider using `nginx.ingress.kubernetes.io/upstream-hash-by`)
|
||||
- ewma: to use the Peak EWMA method for routing ([implementation](https://github.com/kubernetes/ingress-nginx/blob/master/rootfs/etc/nginx/lua/balancer/ewma.lua))
|
||||
|
||||
The default is least_conn.
|
||||
The default is `round_robin`.
|
||||
|
||||
_References:_
|
||||
[http://nginx.org/en/docs/http/load_balancing.html](http://nginx.org/en/docs/http/load_balancing.html)
|
||||
|
|
@ -638,6 +631,10 @@ Specifies the port to use when uploading traces. _**default:**_ 9411
|
|||
|
||||
Specifies the service name to use for any traces created. _**default:**_ nginx
|
||||
|
||||
## zipkin-sample-rate
|
||||
|
||||
Specifies sample rate for any traces created. _**default:**_ 1.0
|
||||
|
||||
## jaeger-collector-host
|
||||
|
||||
Specifies the host to use when uploading traces. It must be a valid URL.
|
||||
|
|
@ -659,20 +656,21 @@ Specifies the sampler to be used when sampling traces. The available samplers ar
|
|||
Specifies the argument to be passed to the sampler constructor. Must be a number.
|
||||
For const this should be 0 to never sample and 1 to always sample. _**default:**_ 1
|
||||
|
||||
## main-snippet
|
||||
|
||||
Adds custom configuration to the main section of the nginx configuration.
|
||||
|
||||
## http-snippet
|
||||
|
||||
Adds custom configuration to the http section of the nginx configuration.
|
||||
_**default:**_ ""
|
||||
|
||||
## server-snippet
|
||||
|
||||
Adds custom configuration to all the servers in the nginx configuration.
|
||||
_**default:**_ ""
|
||||
|
||||
## location-snippet
|
||||
|
||||
Adds custom configuration to all the locations in the nginx configuration.
|
||||
_**default:**_ ""
|
||||
|
||||
## custom-http-errors
|
||||
|
||||
|
|
@ -761,7 +759,7 @@ _References:_
|
|||
## http-redirect-code
|
||||
|
||||
Sets the HTTP status code to be used in redirects.
|
||||
Supported codes are [301](https://developer.mozilla.org/es/docs/Web/HTTP/Status/301),[302](https://developer.mozilla.org/es/docs/Web/HTTP/Status/302),[307](https://developer.mozilla.org/es/docs/Web/HTTP/Status/307) and [308](https://developer.mozilla.org/es/docs/Web/HTTP/Status/308)
|
||||
Supported codes are [301](https://developer.mozilla.org/docs/Web/HTTP/Status/301),[302](https://developer.mozilla.org/docs/Web/HTTP/Status/302),[307](https://developer.mozilla.org/docs/Web/HTTP/Status/307) and [308](https://developer.mozilla.org/docs/Web/HTTP/Status/308)
|
||||
_**default:**_ 308
|
||||
|
||||
> __Why the default code is 308?__
|
||||
|
|
|
|||
|
|
@ -31,6 +31,15 @@ log_format upstreaminfo
|
|||
| `$upstream_response_time` | time spent on receiving the response from the upstream server as seconds with millisecond resolution |
|
||||
| `$upstream_status` | status code of the response obtained from the upstream server |
|
||||
|
||||
Additional available variables:
|
||||
|
||||
| Placeholder | Description |
|
||||
|-------------|-------------|
|
||||
| `$namespace` | namespace of the ingress |
|
||||
| `$ingress_name` | name of the ingress |
|
||||
| `$service_name` | name of the service |
|
||||
| `$service_port` | port of the service |
|
||||
|
||||
|
||||
Sources:
|
||||
|
||||
|
|
|
|||
|
|
@ -1,11 +0,0 @@
|
|||
# NGINX status page
|
||||
|
||||
The [ngx_http_stub_status_module](http://nginx.org/en/docs/http/ngx_http_stub_status_module.html) module provides access to basic status information.
|
||||
This is the default module active in the url `/nginx_status` in the status port (default is 18080).
|
||||
|
||||
This controller provides an alternative to this module using the [nginx-module-vts](https://github.com/vozlt/nginx-module-vts) module.
|
||||
To use this module just set in the configuration configmap `enable-vts-status: "true"`.
|
||||
|
||||
|
||||
|
||||
To extract the information in JSON format the module provides a custom URL: `/nginx_status/format/json`
|
||||
|
|
@ -7,7 +7,7 @@ The [ModSecurity-nginx](https://github.com/SpiderLabs/ModSecurity-nginx) connect
|
|||
The default ModSecurity configuration file is located in `/etc/nginx/modsecurity/modsecurity.conf`. This is the only file located in this directory and contains the default recommended configuration. Using a volume we can replace this file with the desired configuration.
|
||||
To enable the ModSecurity feature we need to specify `enable-modsecurity: "true"` in the configuration configmap.
|
||||
|
||||
>__Note:__ the default configuration use detection only, because that minimises the chances of post-installation disruption.
|
||||
>__Note:__ the default configuration use detection only, because that minimizes the chances of post-installation disruption.
|
||||
The file `/var/log/modsec_audit.log` contains the log of ModSecurity.
|
||||
|
||||
|
||||
|
|
|
|||
|
|
@ -1,9 +1,59 @@
|
|||
# OpenTracing
|
||||
|
||||
Enables requests served by nginx for distributed tracing via The OpenTracing Project.
|
||||
|
||||
Using the third party module [opentracing-contrib/nginx-opentracing](https://github.com/opentracing-contrib/nginx-opentracing) the NGINX ingress controller can configure NGINX to enable [OpenTracing](http://opentracing.io) instrumentation.
|
||||
By default this feature is disabled.
|
||||
|
||||
To enable the instrumentation we just need to enable the instrumentation in the configuration configmap and set the host where we should send the traces.
|
||||
## Usage
|
||||
|
||||
To enable the instrumentation we must enable opentracing in the configuration configmap:
|
||||
```
|
||||
data:
|
||||
enable-opentracing: "true"
|
||||
```
|
||||
|
||||
We must also set the host to use when uploading traces:
|
||||
|
||||
```
|
||||
zipkin-collector-host: zipkin.default.svc.cluster.local
|
||||
jaeger-collector-host: jaeger-collector.default.svc.cluster.local
|
||||
```
|
||||
|
||||
Next you will need to deploy a distributed tracing system which uses OpenTracing. Both [Zipkin](https://github.com/openzipkin/zipkin) and
|
||||
[Jaeger](https://github.com/jaegertracing/jaeger) have been tested.
|
||||
|
||||
Other optional configuration options:
|
||||
```
|
||||
# specifies the port to use when uploading traces
|
||||
zipkin-collector-port
|
||||
|
||||
# specifies the service name to use for any traces created, Default: nginx
|
||||
zipkin-service-name
|
||||
|
||||
# specifies sample rate for any traces created. Default: 1.0
|
||||
zipkin-sample-rate
|
||||
|
||||
# specifies the port to use when uploading traces
|
||||
jaeger-collector-port
|
||||
|
||||
# specifies the service name to use for any traces created, Default: nginx
|
||||
jaeger-service-name
|
||||
|
||||
# specifies the sampler to be used when sampling traces.
|
||||
# The available samplers are: const, probabilistic, ratelimiting, remote, Default: const
|
||||
jaeger-sampler-type
|
||||
|
||||
# specifies the argument to be passed to the sampler constructor, Default: 1
|
||||
jaeger-sampler-param
|
||||
```
|
||||
|
||||
## Examples
|
||||
|
||||
The following examples show how to deploy and test different distributed tracing systems. These example can be performed
|
||||
using Minikube.
|
||||
|
||||
### Zipkin
|
||||
|
||||
In the [rnburn/zipkin-date-server](https://github.com/rnburn/zipkin-date-server)
|
||||
github repository is an example of a dockerized date service. To install the example and zipkin collector run:
|
||||
|
|
@ -23,20 +73,111 @@ data:
|
|||
enable-opentracing: "true"
|
||||
zipkin-collector-host: zipkin.default.svc.cluster.local
|
||||
metadata:
|
||||
name: nginx-configuration
|
||||
namespace: ingress-nginx
|
||||
labels:
|
||||
app: ingress-nginx
|
||||
name: nginx-load-balancer-conf
|
||||
namespace: kube-system
|
||||
' | kubectl replace -f -
|
||||
```
|
||||
|
||||
Using curl we can generate some traces:
|
||||
|
||||
```console
|
||||
$ curl -v http://$(minikube ip)
|
||||
$ curl -v http://$(minikube ip)
|
||||
```
|
||||
|
||||
In the zipkin interface we can see the details:
|
||||
|
||||
|
||||
|
||||
### Jaeger
|
||||
|
||||
1. Enable Ingress addon in minikube:
|
||||
```
|
||||
$ minikube addons enable ingress
|
||||
```
|
||||
|
||||
2. Add minikube IP to /etc/hosts:
|
||||
```
|
||||
$ echo "$(minikube ip) example.com" | sudo tee -a /etc/hosts
|
||||
```
|
||||
|
||||
3. Apply a Basic Service and Ingress Resource:
|
||||
```
|
||||
# Create Echoheaders Deployment
|
||||
$ kubectl run echoheaders --image=k8s.gcr.io/echoserver:1.4 --replicas=1 --port=8080
|
||||
|
||||
# Expose as a Cluster-IP
|
||||
$ kubectl expose deployment echoheaders --port=80 --target-port=8080 --name=echoheaders-x
|
||||
|
||||
# Apply the Ingress Resource
|
||||
$ echo '
|
||||
apiVersion: extensions/v1beta1
|
||||
kind: Ingress
|
||||
metadata:
|
||||
name: echo-ingress
|
||||
spec:
|
||||
rules:
|
||||
- host: example.com
|
||||
http:
|
||||
paths:
|
||||
- backend:
|
||||
serviceName: echoheaders-x
|
||||
servicePort: 80
|
||||
path: /echo
|
||||
' | kubectl apply -f -
|
||||
```
|
||||
|
||||
4. Enable OpenTracing and set the zipkin-collector-host:
|
||||
```
|
||||
$ echo '
|
||||
apiVersion: v1
|
||||
kind: ConfigMap
|
||||
data:
|
||||
enable-opentracing: "true"
|
||||
zipkin-collector-host: zipkin.default.svc.cluster.local
|
||||
jaeger-collector-host: jaeger-collector.default.svc.cluster.local
|
||||
metadata:
|
||||
name: nginx-load-balancer-conf
|
||||
namespace: kube-system
|
||||
' | kubectl replace -f -
|
||||
```
|
||||
|
||||
5. Apply the Jaeger All-In-One Template:
|
||||
```
|
||||
$ kubectl apply -f https://raw.githubusercontent.com/jaegertracing/jaeger-kubernetes/master/all-in-one/jaeger-all-in-one-template.yml
|
||||
```
|
||||
|
||||
6. Make a few requests to the Service:
|
||||
```
|
||||
$ curl example.com/echo -d "meow"
|
||||
|
||||
CLIENT VALUES:
|
||||
client_address=172.17.0.5
|
||||
command=POST
|
||||
real path=/echo
|
||||
query=nil
|
||||
request_version=1.1
|
||||
request_uri=http://example.com:8080/echo
|
||||
|
||||
SERVER VALUES:
|
||||
server_version=nginx: 1.10.0 - lua: 10001
|
||||
|
||||
HEADERS RECEIVED:
|
||||
accept=*/*
|
||||
connection=close
|
||||
content-length=4
|
||||
content-type=application/x-www-form-urlencoded
|
||||
host=example.com
|
||||
user-agent=curl/7.54.0
|
||||
x-forwarded-for=192.168.99.1
|
||||
x-forwarded-host=example.com
|
||||
x-forwarded-port=80
|
||||
x-forwarded-proto=http
|
||||
x-original-uri=/echo
|
||||
x-real-ip=192.168.99.1
|
||||
x-scheme=http
|
||||
BODY:
|
||||
meow
|
||||
```
|
||||
|
||||
7. View the Jaeger UI:
|
||||
```
|
||||
$ minikube service jaeger-query --url
|
||||
|
||||
http://192.168.99.100:30183
|
||||
```
|
||||
|
||||
In the jaeger interface we can see the details:
|
||||
|
||||
|
|
|
|||
Loading…
Add table
Add a link
Reference in a new issue