Merge remote-tracking branch 'master/master' into refactor-template-headers
This commit is contained in:
Manuel de Brito Fontes
commit
3ed6019f9f
33 changed files with 576 additions and 104 deletions
|
|
@ -17,6 +17,7 @@ This is an nginx Ingress controller that uses [ConfigMap](https://kubernetes.io/
|
|||
* [TCP Services](#exposing-tcp-services)
|
||||
* [UDP Services](#exposing-udp-services)
|
||||
* [Proxy Protocol](#proxy-protocol)
|
||||
* [Opentracing](#opentracing)
|
||||
* [NGINX customization](configuration.md)
|
||||
* [Custom errors](#custom-errors)
|
||||
* [NGINX status page](#nginx-status-page)
|
||||
|
|
@ -334,8 +335,8 @@ version to fully support Kube-Lego is nginx Ingress controller 0.8.
|
|||
|
||||
## Exposing TCP services
|
||||
|
||||
Ingress does not support TCP services (yet). For this reason this Ingress controller uses the flag `--tcp-services-configmap` to point to an existing config map where the key is the external port to use and the value is `<namespace/service name>:<service port>:[PROXY]`
|
||||
It is possible to use a number or the name of the port. The last field is optional. Adding `PROXY` in the last field we can enable Proxy Protocol in a TCP service.
|
||||
Ingress does not support TCP services (yet). For this reason this Ingress controller uses the flag `--tcp-services-configmap` to point to an existing config map where the key is the external port to use and the value is `<namespace/service name>:<service port>:[PROXY]:[PROXY]`
|
||||
It is possible to use a number or the name of the port. The two last fields are optional. Adding `PROXY` in either or both of the two last fields we can use Proxy Protocol decoding (listen) and/or encoding (proxy_pass) in a TCP service (https://www.nginx.com/resources/admin-guide/proxy-protocol/).
|
||||
|
||||
The next example shows how to expose the service `example-go` running in the namespace `default` in the port `8080` using the port `9000`
|
||||
```
|
||||
|
|
@ -378,14 +379,63 @@ Amongst others [ELBs in AWS](http://docs.aws.amazon.com/ElasticLoadBalancing/lat
|
|||
|
||||
Please check the [proxy-protocol](examples/proxy-protocol/) example
|
||||
|
||||
### Opentracing
|
||||
|
||||
Using the third party module [rnburn/nginx-opentracing](https://github.com/rnburn/nginx-opentracing) the NGINX ingress controller can configure NGINX to enable [OpenTracing](http://opentracing.io) instrumentation.
|
||||
By default this feature is disabled.
|
||||
|
||||
To enable the instrumentation we just need to enable the instrumentation in the configuration configmap and set the host where we should send the traces.
|
||||
|
||||
In the [aledbf/zipkin-js-example](https://github.com/aledbf/zipkin-js-example) github repository is possible to see a dockerized version of zipkin-js-example with the required Kubernetes descriptors.
|
||||
To install the example and the zipkin collector we just need to run:
|
||||
|
||||
```
|
||||
$ kubectl create -f https://raw.githubusercontent.com/aledbf/zipkin-js-example/kubernetes/kubernetes/zipkin.yaml
|
||||
$ kubectl create -f https://raw.githubusercontent.com/aledbf/zipkin-js-example/kubernetes/kubernetes/deployment.yaml
|
||||
```
|
||||
|
||||
Also we need to configure the NGINX controller configmap with the required values:
|
||||
|
||||
```
|
||||
apiVersion: v1
|
||||
data:
|
||||
enable-opentracing: "true"
|
||||
zipkin-collector-host: zipkin.default.svc.cluster.local
|
||||
kind: ConfigMap
|
||||
metadata:
|
||||
labels:
|
||||
k8s-app: nginx-ingress-controller
|
||||
name: nginx-custom-configuration
|
||||
```
|
||||
|
||||
Using curl we can generate some traces:
|
||||
```
|
||||
$ curl -v http://$(minikube ip)/api -H 'Host: zipkin-js-example'
|
||||
$ curl -v http://$(minikube ip)/api -H 'Host: zipkin-js-example'
|
||||
```
|
||||
|
||||
In the zipkin inteface we can see the details:
|
||||
|
||||
|
||||
|
||||
### Custom errors
|
||||
|
||||
In case of an error in a request the body of the response is obtained from the `default backend`. Each request to the default backend includes two headers:
|
||||
- `X-Code` indicates the HTTP code
|
||||
- `X-Format` the value of the `Accept` header
|
||||
In case of an error in a request the body of the response is obtained from the `default backend`.
|
||||
Each request to the default backend includes two headers:
|
||||
|
||||
Using this two headers is possible to use a custom backend service like [this one](https://github.com/aledbf/contrib/tree/nginx-debug-server/Ingress/images/nginx-error-server) that inspect each request and returns a custom error page with the format expected by the client. Please check the example [custom-errors](examples/custom-errors/README.md)
|
||||
- `X-Code` indicates the HTTP code to be returned to the client.
|
||||
- `X-Format` the value of the `Accept` header.
|
||||
|
||||
**Important:** the custom backend must return the correct HTTP status code to be returned. NGINX do not changes the reponse from the custom default backend.
|
||||
|
||||
Using this two headers is possible to use a custom backend service like [this one](https://github.com/kubernetes/ingress/tree/master/examples/customization/custom-errors/nginx) that inspect each request and returns a custom error page with the format expected by the client. Please check the example [custom-errors](examples/customization/custom-errors/nginx/README.md)
|
||||
|
||||
NGINX sends aditional headers that can be used to build custom response:
|
||||
|
||||
- X-Original-URI
|
||||
- X-Namespace
|
||||
- X-Ingress-Name
|
||||
- X-Service-Name
|
||||
|
||||
### NGINX status page
|
||||
|
||||
|
|
|
|||
|
|
@ -135,14 +135,14 @@ Please check the [auth](/examples/auth/basic/nginx/README.md) example.
|
|||
|
||||
### Certificate Authentication
|
||||
|
||||
It's possible to enable Certificate based authentication using additional annotations in Ingress Rule.
|
||||
It's possible to enable Certificate-Based Authentication (Mutual Authentication) using additional annotations in Ingress Rule.
|
||||
|
||||
The annotations are:
|
||||
```
|
||||
ingress.kubernetes.io/auth-tls-secret: secretName
|
||||
```
|
||||
|
||||
The name of the secret that contains the full Certificate Authority chain that is enabled to authenticate against this ingress. It's composed of namespace/secretName
|
||||
The name of the secret that contains the full Certificate Authority chain `ca.crt` that is enabled to authenticate against this ingress. It's composed of namespace/secretName.
|
||||
|
||||
```
|
||||
ingress.kubernetes.io/auth-tls-verify-depth
|
||||
|
|
@ -487,6 +487,17 @@ The default mime type list to compress is: `application/atom+xml application/jav
|
|||
|
||||
**bind-address:** Sets the addresses on which the server will accept requests instead of *. It should be noted that these addresses must exist in the runtime environment or the controller will crash loop.
|
||||
|
||||
**enable-opentracing:** enables the nginx Opentracing extension https://github.com/rnburn/nginx-opentracing
|
||||
Default is "false"
|
||||
|
||||
**zipkin-collector-host:** specifies the host to use when uploading traces. It must be a valid URL
|
||||
|
||||
**zipkin-collector-port:** specifies the port to use when uploading traces
|
||||
Default: 9411
|
||||
|
||||
**zipkin-service-name:** specifies the service name to use for any traces created
|
||||
Default: nginx
|
||||
|
||||
### Default configuration options
|
||||
|
||||
The following table shows the options, the default value and a description.
|
||||
|
|
|
|||
BIN
controllers/nginx/docs/images/zipkin-demo.png
Normal file
BIN
controllers/nginx/docs/images/zipkin-demo.png
Normal file
Binary file not shown.
|
After Width: | Height: | Size: 18 KiB |
|
|
@ -35,6 +35,7 @@ import (
|
|||
"github.com/spf13/pflag"
|
||||
|
||||
proxyproto "github.com/armon/go-proxyproto"
|
||||
"github.com/ncabatoff/process-exporter/proc"
|
||||
apiv1 "k8s.io/api/core/v1"
|
||||
extensions "k8s.io/api/extensions/v1beta1"
|
||||
|
||||
|
|
@ -62,7 +63,7 @@ const (
|
|||
var (
|
||||
tmplPath = "/etc/nginx/template/nginx.tmpl"
|
||||
cfgPath = "/etc/nginx/nginx.conf"
|
||||
binary = "/usr/sbin/nginx"
|
||||
nginxBinary = "/usr/sbin/nginx"
|
||||
defIngressClass = "nginx"
|
||||
)
|
||||
|
||||
|
|
@ -72,7 +73,7 @@ var (
|
|||
func newNGINXController() *NGINXController {
|
||||
ngx := os.Getenv("NGINX_BINARY")
|
||||
if ngx == "" {
|
||||
ngx = binary
|
||||
ngx = nginxBinary
|
||||
}
|
||||
|
||||
h, err := dns.GetSystemNameServers()
|
||||
|
|
@ -200,7 +201,26 @@ NGINX master process died (%v): %v
|
|||
break
|
||||
}
|
||||
conn.Close()
|
||||
time.Sleep(1 * time.Second)
|
||||
// kill nginx worker processes
|
||||
fs, err := proc.NewFS("/proc")
|
||||
procs, _ := fs.FS.AllProcs()
|
||||
for _, p := range procs {
|
||||
pn, err := p.Comm()
|
||||
if err != nil {
|
||||
glog.Errorf("unexpected error obtaining process information: %v", err)
|
||||
continue
|
||||
}
|
||||
|
||||
if pn == "nginx" {
|
||||
osp, err := os.FindProcess(p.PID)
|
||||
if err != nil {
|
||||
glog.Errorf("unexpected error obtaining process information: %v", err)
|
||||
continue
|
||||
}
|
||||
osp.Signal(syscall.SIGQUIT)
|
||||
}
|
||||
}
|
||||
time.Sleep(100 * time.Millisecond)
|
||||
}
|
||||
// restart a new nginx master process if the controller
|
||||
// is not being stopped
|
||||
|
|
@ -710,6 +730,28 @@ func (n NGINXController) Check(_ *http.Request) error {
|
|||
if res.StatusCode != 200 {
|
||||
return fmt.Errorf("ingress controller is not healthy")
|
||||
}
|
||||
|
||||
// check the nginx master process is running
|
||||
fs, err := proc.NewFS("/proc")
|
||||
if err != nil {
|
||||
glog.Errorf("%v", err)
|
||||
return err
|
||||
}
|
||||
f, err := ioutil.ReadFile("/run/nginx.pid")
|
||||
if err != nil {
|
||||
glog.Errorf("%v", err)
|
||||
return err
|
||||
}
|
||||
pid, err := strconv.Atoi(strings.TrimRight(string(f), "\r\n"))
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
_, err = fs.NewProc(int(pid))
|
||||
if err != nil {
|
||||
glog.Errorf("%v", err)
|
||||
return err
|
||||
}
|
||||
|
||||
return nil
|
||||
}
|
||||
|
||||
|
|
|
|||
|
|
@ -22,7 +22,6 @@ import (
|
|||
"encoding/json"
|
||||
"fmt"
|
||||
"net"
|
||||
"net/url"
|
||||
"os"
|
||||
"os/exec"
|
||||
"strconv"
|
||||
|
|
@ -151,7 +150,6 @@ var (
|
|||
"serverConfig": func(all config.TemplateConfig, server *ingress.Server) interface{} {
|
||||
return struct{ First, Second interface{} }{all, server}
|
||||
},
|
||||
"buildAuthSignURL": buildAuthSignURL,
|
||||
"isValidClientBodyBufferSize": isValidClientBodyBufferSize,
|
||||
"buildForwardedFor": buildForwardedFor,
|
||||
"trustHTTPHeaders": trustHTTPHeaders,
|
||||
|
|
@ -570,22 +568,6 @@ func buildNextUpstream(input interface{}) string {
|
|||
return strings.Join(nextUpstreamCodes, " ")
|
||||
}
|
||||
|
||||
func buildAuthSignURL(input interface{}) string {
|
||||
s, ok := input.(string)
|
||||
if !ok {
|
||||
glog.Errorf("expected an 'string' type but %T was returned", input)
|
||||
return ""
|
||||
}
|
||||
|
||||
u, _ := url.Parse(s)
|
||||
q := u.Query()
|
||||
if len(q) == 0 {
|
||||
return fmt.Sprintf("%v?rd=$request_uri", s)
|
||||
}
|
||||
|
||||
return fmt.Sprintf("%v&rd=$request_uri", s)
|
||||
}
|
||||
|
||||
// buildRandomUUID return a random string to be used in the template
|
||||
func buildRandomUUID() string {
|
||||
s := uuid.New()
|
||||
|
|
|
|||
|
|
@ -310,24 +310,6 @@ func TestBuildResolvers(t *testing.T) {
|
|||
}
|
||||
}
|
||||
|
||||
func TestBuildAuthSignURL(t *testing.T) {
|
||||
urlOne := "http://google.com"
|
||||
validUrlOne := "http://google.com?rd=$request_uri"
|
||||
|
||||
urlTwo := "http://google.com?cat"
|
||||
validUrlTwo := "http://google.com?cat&rd=$request_uri"
|
||||
|
||||
authSignURLOne := buildAuthSignURL(urlOne)
|
||||
if authSignURLOne != validUrlOne {
|
||||
t.Errorf("Expected '%v' but returned '%v'", validUrlOne, authSignURLOne)
|
||||
}
|
||||
|
||||
authSignURLTwo := buildAuthSignURL(urlTwo)
|
||||
if authSignURLTwo != validUrlTwo {
|
||||
t.Errorf("Expected '%v' but returned '%v'", validUrlTwo, authSignURLTwo)
|
||||
}
|
||||
}
|
||||
|
||||
func TestBuildNextUpstream(t *testing.T) {
|
||||
nextUpstream := "timeout http_500 http_502 non_idempotent"
|
||||
validNextUpstream := "timeout http_500 http_502"
|
||||
|
|
|
|||
|
|
@ -461,19 +461,22 @@ stream {
|
|||
}
|
||||
server {
|
||||
{{ range $address := $all.Cfg.BindAddressIpv4 }}
|
||||
listen {{ $address }}:{{ $tcpServer.Port }}{{ if $tcpServer.Backend.UseProxyProtocol }} proxy_protocol{{ end }};
|
||||
listen {{ $address }}:{{ $tcpServer.Port }}{{ if $tcpServer.Backend.ProxyProtocol.Decode }} proxy_protocol{{ end }};
|
||||
{{ else }}
|
||||
listen {{ $tcpServer.Port }}{{ if $tcpServer.Backend.UseProxyProtocol }} proxy_protocol{{ end }};
|
||||
listen {{ $tcpServer.Port }}{{ if $tcpServer.Backend.ProxyProtocol.Decode }} proxy_protocol{{ end }};
|
||||
{{ end }}
|
||||
{{ if $IsIPV6Enabled }}
|
||||
{{ range $address := $all.Cfg.BindAddressIpv6 }}
|
||||
listen {{ $address }}:{{ $tcpServer.Port }}{{ if $tcpServer.Backend.UseProxyProtocol }} proxy_protocol{{ end }};
|
||||
listen {{ $address }}:{{ $tcpServer.Port }}{{ if $tcpServer.Backend.ProxyProtocol.Decode }} proxy_protocol{{ end }};
|
||||
{{ else }}
|
||||
listen [::]:{{ $tcpServer.Port }}{{ if $tcpServer.Backend.UseProxyProtocol }} proxy_protocol{{ end }};
|
||||
listen [::]:{{ $tcpServer.Port }}{{ if $tcpServer.Backend.ProxyProtocol.Decode }} proxy_protocol{{ end }};
|
||||
{{ end }}
|
||||
{{ end }}
|
||||
proxy_timeout {{ $cfg.ProxyStreamTimeout }};
|
||||
proxy_pass tcp-{{ $tcpServer.Port }}-{{ $tcpServer.Backend.Namespace }}-{{ $tcpServer.Backend.Name }}-{{ $tcpServer.Backend.Port }};
|
||||
{{ if $tcpServer.Backend.ProxyProtocol.Encode }}
|
||||
proxy_protocol on;
|
||||
{{ end }}
|
||||
}
|
||||
|
||||
{{ end }}
|
||||
|
|
@ -514,6 +517,8 @@ stream {
|
|||
location @custom_{{ $errCode }} {
|
||||
internal;
|
||||
|
||||
proxy_intercept_errors off;
|
||||
|
||||
proxy_set_header X-Code {{ $errCode }};
|
||||
proxy_set_header X-Format $http_accept;
|
||||
proxy_set_header X-Original-URI $request_uri;
|
||||
|
|
@ -521,6 +526,7 @@ stream {
|
|||
proxy_set_header X-Ingress-Name $ingress_name;
|
||||
proxy_set_header X-Service-Name $service_name;
|
||||
|
||||
rewrite (.*) / break;
|
||||
proxy_pass http://upstream-default-backend;
|
||||
}
|
||||
{{ end }}
|
||||
|
|
@ -626,6 +632,10 @@ stream {
|
|||
{{ end }}
|
||||
{{ end }}
|
||||
|
||||
{{ if not (empty $server.ServerSnippet) }}
|
||||
{{ $server.ServerSnippet }}
|
||||
{{ end }}
|
||||
|
||||
{{ range $location := $server.Locations }}
|
||||
{{ $path := buildLocation $location }}
|
||||
{{ $authPath := buildAuthLocation $location }}
|
||||
|
|
@ -704,7 +714,7 @@ stream {
|
|||
{{ end }}
|
||||
|
||||
{{ if not (empty $location.ExternalAuth.SigninURL) }}
|
||||
error_page 401 = {{ buildAuthSignURL $location.ExternalAuth.SigninURL }};
|
||||
error_page 401 = $location.ExternalAuth.SigninURL;
|
||||
{{ end }}
|
||||
|
||||
{{/* if the location contains a rate limit annotation, create one */}}
|
||||
|
|
@ -763,6 +773,9 @@ stream {
|
|||
proxy_set_header X-Original-URI $request_uri;
|
||||
proxy_set_header X-Scheme $pass_access_scheme;
|
||||
|
||||
{{/* This header is used for external authentication */}}
|
||||
proxy_set_header X-Auth-Request-Redirect $request_uri;
|
||||
|
||||
# mitigate HTTPoxy Vulnerability
|
||||
# https://www.nginx.com/blog/mitigating-the-httpoxy-vulnerability-with-nginx/
|
||||
proxy_set_header Proxy "";
|
||||
|
|
|
|||
Loading…
Add table
Add a link
Reference in a new issue