Remove global-rate-limit feature (#11851)
This commit is contained in:
Ricardo Katz
GitHub
parent
5243b9b90a
commit
21cd966d1c
25 changed files with 28 additions and 1326 deletions
|
|
@ -7,7 +7,6 @@ Do not try to edit it manually.
|
|||
|
||||
|
||||
### [[Admission] admission controller](https://github.com/kubernetes/ingress-nginx/tree/main//test/e2e/admission/admission.go#L39)
|
||||
- [reject ingress with global-rate-limit annotations when memcached is not configured](https://github.com/kubernetes/ingress-nginx/tree/main//test/e2e/admission/admission.go#L47)
|
||||
- [should not allow overlaps of host and paths without canary annotations](https://github.com/kubernetes/ingress-nginx/tree/main//test/e2e/admission/admission.go#L74)
|
||||
- [should allow overlaps of host and paths with canary annotation](https://github.com/kubernetes/ingress-nginx/tree/main//test/e2e/admission/admission.go#L91)
|
||||
- [should block ingress with invalid path](https://github.com/kubernetes/ingress-nginx/tree/main//test/e2e/admission/admission.go#L112)
|
||||
|
|
@ -173,8 +172,6 @@ Do not try to edit it manually.
|
|||
### [from-to-www-redirect](https://github.com/kubernetes/ingress-nginx/tree/main//test/e2e/annotations/fromtowwwredirect.go#L31)
|
||||
- [should redirect from www HTTP to HTTP](https://github.com/kubernetes/ingress-nginx/tree/main//test/e2e/annotations/fromtowwwredirect.go#L38)
|
||||
- [should redirect from www HTTPS to HTTPS](https://github.com/kubernetes/ingress-nginx/tree/main//test/e2e/annotations/fromtowwwredirect.go#L64)
|
||||
### [annotation-global-rate-limit](https://github.com/kubernetes/ingress-nginx/tree/main//test/e2e/annotations/globalratelimit.go#L30)
|
||||
- [generates correct configuration](https://github.com/kubernetes/ingress-nginx/tree/main//test/e2e/annotations/globalratelimit.go#L38)
|
||||
### [backend-protocol - GRPC](https://github.com/kubernetes/ingress-nginx/tree/main//test/e2e/annotations/grpc.go#L45)
|
||||
- [should use grpc_pass in the configuration file](https://github.com/kubernetes/ingress-nginx/tree/main//test/e2e/annotations/grpc.go#L48)
|
||||
- [should return OK for service with backend protocol GRPC](https://github.com/kubernetes/ingress-nginx/tree/main//test/e2e/annotations/grpc.go#L71)
|
||||
|
|
@ -420,8 +417,6 @@ Do not try to edit it manually.
|
|||
### [global-options](https://github.com/kubernetes/ingress-nginx/tree/main//test/e2e/settings/global_options.go#L28)
|
||||
- [should have worker_rlimit_nofile option](https://github.com/kubernetes/ingress-nginx/tree/main//test/e2e/settings/global_options.go#L31)
|
||||
- [should have worker_rlimit_nofile option and be independent on amount of worker processes](https://github.com/kubernetes/ingress-nginx/tree/main//test/e2e/settings/global_options.go#L37)
|
||||
### [settings-global-rate-limit](https://github.com/kubernetes/ingress-nginx/tree/main//test/e2e/settings/globalratelimit.go#L30)
|
||||
- [generates correct NGINX configuration](https://github.com/kubernetes/ingress-nginx/tree/main//test/e2e/settings/globalratelimit.go#L38)
|
||||
### [GRPC](https://github.com/kubernetes/ingress-nginx/tree/main//test/e2e/settings/grpc.go#L39)
|
||||
- [should set the correct GRPC Buffer Size](https://github.com/kubernetes/ingress-nginx/tree/main//test/e2e/settings/grpc.go#L42)
|
||||
### [gzip](https://github.com/kubernetes/ingress-nginx/tree/main//test/e2e/settings/gzip.go#L30)
|
||||
|
|
|
|||
|
|
@ -55,10 +55,6 @@
|
|||
| ExternalAuth | auth-url | High | location |
|
||||
| FastCGI | fastcgi-index | Medium | location |
|
||||
| FastCGI | fastcgi-params-configmap | Medium | location |
|
||||
| GlobalRateLimit | global-rate-limit | Low | ingress |
|
||||
| GlobalRateLimit | global-rate-limit-ignored-cidrs | Medium | ingress |
|
||||
| GlobalRateLimit | global-rate-limit-key | High | ingress |
|
||||
| GlobalRateLimit | global-rate-limit-window | Low | ingress |
|
||||
| HTTP2PushPreload | http2-push-preload | Low | location |
|
||||
| LoadBalancing | load-balance | Low | location |
|
||||
| Logs | enable-access-log | Low | location |
|
||||
|
|
|
|||
|
|
@ -64,10 +64,6 @@ You can add these Kubernetes annotations to specific Ingress objects to customiz
|
|||
|[nginx.ingress.kubernetes.io/http2-push-preload](#http2-push-preload)|"true" or "false"|
|
||||
|[nginx.ingress.kubernetes.io/limit-connections](#rate-limiting)|number|
|
||||
|[nginx.ingress.kubernetes.io/limit-rps](#rate-limiting)|number|
|
||||
|[nginx.ingress.kubernetes.io/global-rate-limit](#global-rate-limiting)|number|
|
||||
|[nginx.ingress.kubernetes.io/global-rate-limit-window](#global-rate-limiting)|duration|
|
||||
|[nginx.ingress.kubernetes.io/global-rate-limit-key](#global-rate-limiting)|string|
|
||||
|[nginx.ingress.kubernetes.io/global-rate-limit-ignored-cidrs](#global-rate-limiting)|string|
|
||||
|[nginx.ingress.kubernetes.io/permanent-redirect](#permanent-redirect)|string|
|
||||
|[nginx.ingress.kubernetes.io/permanent-redirect-code](#permanent-redirect-code)|number|
|
||||
|[nginx.ingress.kubernetes.io/temporal-redirect](#temporal-redirect)|string|
|
||||
|
|
@ -560,46 +556,6 @@ To configure settings globally for all Ingress rules, the `limit-rate-after` and
|
|||
|
||||
The client IP address will be set based on the use of [PROXY protocol](./configmap.md#use-proxy-protocol) or from the `X-Forwarded-For` header value when [use-forwarded-headers](./configmap.md#use-forwarded-headers) is enabled.
|
||||
|
||||
### Global Rate Limiting
|
||||
|
||||
**Note:** Be careful when configuring both (Local) Rate Limiting and Global Rate Limiting at the same time.
|
||||
They are two completely different rate limiting implementations. Whichever limit exceeds first will reject the
|
||||
requests. It might be a good idea to configure both of them to ease load on Global Rate Limiting backend
|
||||
in cases of spike in traffic.
|
||||
|
||||
The stock NGINX rate limiting does not share its counters among different NGINX instances.
|
||||
Given that most ingress-nginx deployments are elastic and number of replicas can change any day
|
||||
it is impossible to configure a proper rate limit using stock NGINX functionalities.
|
||||
Global Rate Limiting overcome this by using [lua-resty-global-throttle](https://github.com/ElvinEfendi/lua-resty-global-throttle). `lua-resty-global-throttle` shares its counters via a central store such as `memcached`.
|
||||
The obvious shortcoming of this is users have to deploy and operate a `memcached` instance
|
||||
in order to benefit from this functionality. Configure the `memcached`
|
||||
using [these configmap settings](./configmap.md#global-rate-limit).
|
||||
|
||||
**Here are a few remarks for ingress-nginx integration of `lua-resty-global-throttle`:**
|
||||
|
||||
1. We minimize `memcached` access by caching exceeding limit decisions. The expiry of
|
||||
cache entry is the desired delay `lua-resty-global-throttle` calculates for us.
|
||||
The Lua Shared Dictionary used for that is `global_throttle_cache`. Currently its size defaults to 10M.
|
||||
Customize it as per your needs using [lua-shared-dicts](./configmap.md#lua-shared-dicts).
|
||||
When we fail to cache the exceeding limit decision then we log an NGINX error. You can monitor
|
||||
for that error to decide if you need to bump the cache size. Without cache the cost of processing a
|
||||
request is two memcached commands: `GET`, and `INCR`. With the cache it is only `INCR`.
|
||||
1. Log NGINX variable `$global_rate_limit_exceeding`'s value to have some visibility into
|
||||
what portion of requests are rejected (value `y`), whether they are rejected using cached decision (value `c`),
|
||||
or if they are not rejected (default value `n`). You can use [log-format-upstream](./configmap.md#log-format-upstream)
|
||||
to include that in access logs.
|
||||
1. In case of an error it will log the error message and **fail open**.
|
||||
1. The annotations below creates Global Rate Limiting instance per ingress.
|
||||
That means if there are multiple paths configured under the same ingress,
|
||||
the Global Rate Limiting will count requests to all the paths under the same counter.
|
||||
Extract a path out into its own ingress if you need to isolate a certain path.
|
||||
|
||||
|
||||
* `nginx.ingress.kubernetes.io/global-rate-limit`: Configures maximum allowed number of requests per window. Required.
|
||||
* `nginx.ingress.kubernetes.io/global-rate-limit-window`: Configures a time window (i.e `1m`) that the limit is applied. Required.
|
||||
* `nginx.ingress.kubernetes.io/global-rate-limit-key`: Configures a key for counting the samples. Defaults to `$remote_addr`. You can also combine multiple NGINX variables here, like `${remote_addr}-${http_x_api_client}` which would mean the limit will be applied to requests coming from the same API client (indicated by `X-API-Client` HTTP request header) with the same source IP address.
|
||||
* `nginx.ingress.kubernetes.io/global-rate-limit-ignored-cidrs`: comma separated list of IPs and CIDRs to match client IP against. When there's a match request is not considered for rate limiting.
|
||||
|
||||
### Permanent Redirect
|
||||
|
||||
This annotation allows to return a permanent redirect (Return Code 301) instead of sending data to the upstream. For example `nginx.ingress.kubernetes.io/permanent-redirect: https://www.google.com` would redirect everything to Google.
|
||||
|
|
|
|||
|
|
@ -218,12 +218,6 @@ The following table shows a configuration option's name, type, and the default v
|
|||
| [block-referers](#block-referers) | []string | "" | |
|
||||
| [proxy-ssl-location-only](#proxy-ssl-location-only) | bool | "false" | |
|
||||
| [default-type](#default-type) | string | "text/html" | |
|
||||
| [global-rate-limit-memcached-host](#global-rate-limit) | string | "" | |
|
||||
| [global-rate-limit-memcached-port](#global-rate-limit) | int | 11211 | |
|
||||
| [global-rate-limit-memcached-connect-timeout](#global-rate-limit) | int | 50 | |
|
||||
| [global-rate-limit-memcached-max-idle-timeout](#global-rate-limit) | int | 10000 | |
|
||||
| [global-rate-limit-memcached-pool-size](#global-rate-limit) | int | 50 | |
|
||||
| [global-rate-limit-status-code](#global-rate-limit) | int | 429 | |
|
||||
| [service-upstream](#service-upstream) | bool | "false" | |
|
||||
| [ssl-reject-handshake](#ssl-reject-handshake) | bool | "false" | |
|
||||
| [debug-connections](#debug-connections) | []string | "127.0.0.1,1.1.1.1/24" | |
|
||||
|
|
@ -1349,22 +1343,6 @@ _**default:**_ text/html
|
|||
_References:_
|
||||
[https://nginx.org/en/docs/http/ngx_http_core_module.html#default_type](https://nginx.org/en/docs/http/ngx_http_core_module.html#default_type)
|
||||
|
||||
## global-rate-limit
|
||||
|
||||
* `global-rate-limit-status-code`: configure HTTP status code to return when rejecting requests. Defaults to 429.
|
||||
|
||||
Configure `memcached` client for [Global Rate Limiting](https://github.com/kubernetes/ingress-nginx/blob/main/docs/user-guide/nginx-configuration/annotations.md#global-rate-limiting).
|
||||
|
||||
* `global-rate-limit-memcached-host`: IP/FQDN of memcached server to use. Required to enable Global Rate Limiting.
|
||||
* `global-rate-limit-memcached-port`: port of memcached server to use. Defaults default memcached port of `11211`.
|
||||
* `global-rate-limit-memcached-connect-timeout`: configure timeout for connect, send and receive operations. Unit is millisecond. Defaults to 50ms.
|
||||
* `global-rate-limit-memcached-max-idle-timeout`: configure timeout for cleaning idle connections. Unit is millisecond. Defaults to 50ms.
|
||||
* `global-rate-limit-memcached-pool-size`: configure number of max connections to keep alive. Make sure your `memcached` server can handle
|
||||
`global-rate-limit-memcached-pool-size * worker-processes * <number of ingress-nginx replicas>` simultaneous connections.
|
||||
|
||||
These settings get used by [lua-resty-global-throttle](https://github.com/ElvinEfendi/lua-resty-global-throttle)
|
||||
that ingress-nginx includes. Refer to the link to learn more about `lua-resty-global-throttle`.
|
||||
|
||||
## service-upstream
|
||||
|
||||
Set if the service's Cluster IP and port should be used instead of a list of all endpoints. This can be overwritten by an annotation on an Ingress rule.
|
||||
|
|
|
|||
Loading…
Add table
Add a link
Reference in a new issue