ingress-nginx-helm/enhancements/20190815-zone-aware-routing/index.html
2020-02-24 22:42:17 +00:00

1320 lines
No EOL
34 KiB
HTML

<!doctype html>
<html lang="en" class="no-js">
<head>
<meta charset="utf-8">
<meta name="viewport" content="width=device-width,initial-scale=1">
<meta http-equiv="x-ua-compatible" content="ie=edge">
<link rel="canonical" href="https://kubernetes.github.io/ingress-nginx/enhancements/20190815-zone-aware-routing/">
<meta name="lang:clipboard.copy" content="Copy to clipboard">
<meta name="lang:clipboard.copied" content="Copied to clipboard">
<meta name="lang:search.language" content="en">
<meta name="lang:search.pipeline.stopwords" content="True">
<meta name="lang:search.pipeline.trimmer" content="True">
<meta name="lang:search.result.none" content="No matching documents">
<meta name="lang:search.result.one" content="1 matching document">
<meta name="lang:search.result.other" content="# matching documents">
<meta name="lang:search.tokenizer" content="[\s\-]+">
<link rel="shortcut icon" href="../../assets/images/favicon.png">
<meta name="generator" content="mkdocs-1.0.4, mkdocs-material-4.6.3">
<title>Availability zone aware routing</title>
<link rel="stylesheet" href="../../assets/stylesheets/application.adb8469c.css">
<link rel="stylesheet" href="../../assets/stylesheets/application-palette.a8b3c06d.css">
<meta name="theme-color" content="#009688">
<script src="../../assets/javascripts/modernizr.86422ebf.js"></script>
<link href="https://fonts.gstatic.com" rel="preconnect" crossorigin>
<link rel="stylesheet" href="https://fonts.googleapis.com/css?family=Roboto:300,400,400i,700%7CRoboto+Mono&display=fallback">
<style>body,input{font-family:"Roboto","Helvetica Neue",Helvetica,Arial,sans-serif}code,kbd,pre{font-family:"Roboto Mono","Courier New",Courier,monospace}</style>
<link rel="stylesheet" href="../../assets/fonts/material-icons.css">
<link rel="stylesheet" href="../../extra.css">
<script>
window.ga = window.ga || function() {
(ga.q = ga.q || []).push(arguments)
}
ga.l = +new Date
/* Setup integration and send page view */
ga("create", "UA-118407822-1", "kubernetes.github.io")
ga("set", "anonymizeIp", true)
ga("send", "pageview")
/* Register handler to log search on blur */
document.addEventListener("DOMContentLoaded", () => {
if (document.forms.search) {
var query = document.forms.search.query
query.addEventListener("blur", function() {
if (this.value) {
var path = document.location.pathname;
ga("send", "pageview", path + "?q=" + this.value)
}
})
}
})
</script>
<script async src="https://www.google-analytics.com/analytics.js"></script>
</head>
<body dir="ltr" data-md-color-primary="teal" data-md-color-accent="green">
<svg class="md-svg">
<defs>
<svg xmlns="http://www.w3.org/2000/svg" width="416" height="448" viewBox="0 0 416 448" id="__github"><path fill="currentColor" d="M160 304q0 10-3.125 20.5t-10.75 19T128 352t-18.125-8.5-10.75-19T96 304t3.125-20.5 10.75-19T128 256t18.125 8.5 10.75 19T160 304zm160 0q0 10-3.125 20.5t-10.75 19T288 352t-18.125-8.5-10.75-19T256 304t3.125-20.5 10.75-19T288 256t18.125 8.5 10.75 19T320 304zm40 0q0-30-17.25-51T296 232q-10.25 0-48.75 5.25Q229.5 240 208 240t-39.25-2.75Q130.75 232 120 232q-29.5 0-46.75 21T56 304q0 22 8 38.375t20.25 25.75 30.5 15 35 7.375 37.25 1.75h42q20.5 0 37.25-1.75t35-7.375 30.5-15 20.25-25.75T360 304zm56-44q0 51.75-15.25 82.75-9.5 19.25-26.375 33.25t-35.25 21.5-42.5 11.875-42.875 5.5T212 416q-19.5 0-35.5-.75t-36.875-3.125-38.125-7.5-34.25-12.875T37 371.5t-21.5-28.75Q0 312 0 260q0-59.25 34-99-6.75-20.5-6.75-42.5 0-29 12.75-54.5 27 0 47.5 9.875t47.25 30.875Q171.5 96 212 96q37 0 70 8 26.25-20.5 46.75-30.25T376 64q12.75 25.5 12.75 54.5 0 21.75-6.75 42 34 40 34 99.5z"/></svg>
</defs>
</svg>
<input class="md-toggle" data-md-toggle="drawer" type="checkbox" id="__drawer" autocomplete="off">
<input class="md-toggle" data-md-toggle="search" type="checkbox" id="__search" autocomplete="off">
<label class="md-overlay" data-md-component="overlay" for="__drawer"></label>
<a href="#availability-zone-aware-routing" tabindex="0" class="md-skip">
Skip to content
</a>
<header class="md-header" data-md-component="header">
<nav class="md-header-nav md-grid">
<div class="md-flex">
<div class="md-flex__cell md-flex__cell--shrink">
<a href="https://kubernetes.github.io/ingress-nginx" title="NGINX Ingress Controller" aria-label="NGINX Ingress Controller" class="md-header-nav__button md-logo">
<i class="md-icon">public</i>
</a>
</div>
<div class="md-flex__cell md-flex__cell--shrink">
<label class="md-icon md-icon--menu md-header-nav__button" for="__drawer"></label>
</div>
<div class="md-flex__cell md-flex__cell--stretch">
<div class="md-flex__ellipsis md-header-nav__title" data-md-component="title">
<span class="md-header-nav__topic">
NGINX Ingress Controller
</span>
<span class="md-header-nav__topic">
Availability zone aware routing
</span>
</div>
</div>
<div class="md-flex__cell md-flex__cell--shrink">
<label class="md-icon md-icon--search md-header-nav__button" for="__search"></label>
<div class="md-search" data-md-component="search" role="dialog">
<label class="md-search__overlay" for="__search"></label>
<div class="md-search__inner" role="search">
<form class="md-search__form" name="search">
<input type="text" class="md-search__input" aria-label="search" name="query" placeholder="Search" autocapitalize="off" autocorrect="off" autocomplete="off" spellcheck="false" data-md-component="query" data-md-state="active">
<label class="md-icon md-search__icon" for="__search"></label>
<button type="reset" class="md-icon md-search__icon" data-md-component="reset" tabindex="-1">
&#xE5CD;
</button>
</form>
<div class="md-search__output">
<div class="md-search__scrollwrap" data-md-scrollfix>
<div class="md-search-result" data-md-component="result">
<div class="md-search-result__meta">
Type to start searching
</div>
<ol class="md-search-result__list"></ol>
</div>
</div>
</div>
</div>
</div>
</div>
<div class="md-flex__cell md-flex__cell--shrink">
<div class="md-header-nav__source">
<a href="https://github.com/kubernetes/ingress-nginx/" title="Go to repository" class="md-source" data-md-source="github">
<div class="md-source__icon">
<svg viewBox="0 0 24 24" width="24" height="24">
<use xlink:href="#__github" width="24" height="24"></use>
</svg>
</div>
<div class="md-source__repository">
kubernetes/ingress-nginx
</div>
</a>
</div>
</div>
</div>
</nav>
</header>
<div class="md-container">
<nav class="md-tabs" data-md-component="tabs">
<div class="md-tabs__inner md-grid">
<ul class="md-tabs__list">
<li class="md-tabs__item">
<a href="../.." class="md-tabs__link">
Welcome
</a>
</li>
<li class="md-tabs__item">
<a href="../../deploy/" class="md-tabs__link">
Deployment
</a>
</li>
<li class="md-tabs__item">
<a href="../../user-guide/nginx-configuration/" class="md-tabs__link">
User guide
</a>
</li>
<li class="md-tabs__item">
<a href="../../examples/" class="md-tabs__link">
Examples
</a>
</li>
</ul>
</div>
</nav>
<main class="md-main" role="main">
<div class="md-main__inner md-grid" data-md-component="container">
<div class="md-sidebar md-sidebar--primary" data-md-component="navigation">
<div class="md-sidebar__scrollwrap">
<div class="md-sidebar__inner">
<nav class="md-nav md-nav--primary" data-md-level="0">
<label class="md-nav__title md-nav__title--site" for="__drawer">
<a href="https://kubernetes.github.io/ingress-nginx" title="NGINX Ingress Controller" class="md-nav__button md-logo">
<i class="md-icon">public</i>
</a>
NGINX Ingress Controller
</label>
<div class="md-nav__source">
<a href="https://github.com/kubernetes/ingress-nginx/" title="Go to repository" class="md-source" data-md-source="github">
<div class="md-source__icon">
<svg viewBox="0 0 24 24" width="24" height="24">
<use xlink:href="#__github" width="24" height="24"></use>
</svg>
</div>
<div class="md-source__repository">
kubernetes/ingress-nginx
</div>
</a>
</div>
<ul class="md-nav__list" data-md-scrollfix>
<li class="md-nav__item md-nav__item--nested">
<input class="md-toggle md-nav__toggle" data-md-toggle="nav-1" type="checkbox" id="nav-1">
<label class="md-nav__link" for="nav-1">
Welcome
</label>
<nav class="md-nav" data-md-component="collapsible" data-md-level="1">
<label class="md-nav__title" for="nav-1">
Welcome
</label>
<ul class="md-nav__list" data-md-scrollfix>
<li class="md-nav__item">
<a href="../.." title="Welcome" class="md-nav__link">
Welcome
</a>
</li>
<li class="md-nav__item">
<a href="../../how-it-works/" title="How it works" class="md-nav__link">
How it works
</a>
</li>
<li class="md-nav__item">
<a href="../../troubleshooting/" title="Troubleshooting" class="md-nav__link">
Troubleshooting
</a>
</li>
<li class="md-nav__item">
<a href="../../kubectl-plugin/" title="kubectl plugin" class="md-nav__link">
kubectl plugin
</a>
</li>
<li class="md-nav__item">
<a href="../../development/" title="Development" class="md-nav__link">
Development
</a>
</li>
</ul>
</nav>
</li>
<li class="md-nav__item md-nav__item--nested">
<input class="md-toggle md-nav__toggle" data-md-toggle="nav-2" type="checkbox" id="nav-2">
<label class="md-nav__link" for="nav-2">
Deployment
</label>
<nav class="md-nav" data-md-component="collapsible" data-md-level="1">
<label class="md-nav__title" for="nav-2">
Deployment
</label>
<ul class="md-nav__list" data-md-scrollfix>
<li class="md-nav__item">
<a href="../../deploy/" title="Installation Guide" class="md-nav__link">
Installation Guide
</a>
</li>
<li class="md-nav__item">
<a href="../../deploy/baremetal/" title="Bare-metal considerations" class="md-nav__link">
Bare-metal considerations
</a>
</li>
<li class="md-nav__item">
<a href="../../deploy/rbac/" title="Role Based Access Control (RBAC)" class="md-nav__link">
Role Based Access Control (RBAC)
</a>
</li>
<li class="md-nav__item">
<a href="../../deploy/validating-webhook/" title="Validating Webhook (admission controller)" class="md-nav__link">
Validating Webhook (admission controller)
</a>
</li>
<li class="md-nav__item">
<a href="../../deploy/upgrade/" title="Upgrade" class="md-nav__link">
Upgrade
</a>
</li>
</ul>
</nav>
</li>
<li class="md-nav__item md-nav__item--nested">
<input class="md-toggle md-nav__toggle" data-md-toggle="nav-3" type="checkbox" id="nav-3">
<label class="md-nav__link" for="nav-3">
User guide
</label>
<nav class="md-nav" data-md-component="collapsible" data-md-level="1">
<label class="md-nav__title" for="nav-3">
User guide
</label>
<ul class="md-nav__list" data-md-scrollfix>
<li class="md-nav__item md-nav__item--nested">
<input class="md-toggle md-nav__toggle" data-md-toggle="nav-3-1" type="checkbox" id="nav-3-1">
<label class="md-nav__link" for="nav-3-1">
NGINX Configuration
</label>
<nav class="md-nav" data-md-component="collapsible" data-md-level="2">
<label class="md-nav__title" for="nav-3-1">
NGINX Configuration
</label>
<ul class="md-nav__list" data-md-scrollfix>
<li class="md-nav__item">
<a href="../../user-guide/nginx-configuration/" title="Introduction" class="md-nav__link">
Introduction
</a>
</li>
<li class="md-nav__item">
<a href="../../user-guide/basic-usage/" title="Basic usage" class="md-nav__link">
Basic usage
</a>
</li>
<li class="md-nav__item">
<a href="../../user-guide/nginx-configuration/annotations/" title="Annotations" class="md-nav__link">
Annotations
</a>
</li>
<li class="md-nav__item">
<a href="../../user-guide/nginx-configuration/configmap/" title="ConfigMap" class="md-nav__link">
ConfigMap
</a>
</li>
<li class="md-nav__item">
<a href="../../user-guide/nginx-configuration/custom-template/" title="Custom NGINX template" class="md-nav__link">
Custom NGINX template
</a>
</li>
<li class="md-nav__item">
<a href="../../user-guide/nginx-configuration/log-format/" title="Log format" class="md-nav__link">
Log format
</a>
</li>
</ul>
</nav>
</li>
<li class="md-nav__item">
<a href="../../user-guide/cli-arguments/" title="Command line arguments" class="md-nav__link">
Command line arguments
</a>
</li>
<li class="md-nav__item">
<a href="../../user-guide/custom-errors/" title="Custom errors" class="md-nav__link">
Custom errors
</a>
</li>
<li class="md-nav__item">
<a href="../../user-guide/default-backend/" title="Default backend" class="md-nav__link">
Default backend
</a>
</li>
<li class="md-nav__item">
<a href="../../user-guide/exposing-tcp-udp-services/" title="Exposing TCP and UDP services" class="md-nav__link">
Exposing TCP and UDP services
</a>
</li>
<li class="md-nav__item">
<a href="../../user-guide/fcgi-services/" title="Exposing FCGI services" class="md-nav__link">
Exposing FCGI services
</a>
</li>
<li class="md-nav__item">
<a href="../../user-guide/ingress-path-matching/" title="Regular expressions in paths" class="md-nav__link">
Regular expressions in paths
</a>
</li>
<li class="md-nav__item">
<a href="../../user-guide/external-articles/" title="External Articles" class="md-nav__link">
External Articles
</a>
</li>
<li class="md-nav__item">
<a href="../../user-guide/miscellaneous/" title="Miscellaneous" class="md-nav__link">
Miscellaneous
</a>
</li>
<li class="md-nav__item">
<a href="../../user-guide/monitoring/" title="Prometheus and Grafana installation" class="md-nav__link">
Prometheus and Grafana installation
</a>
</li>
<li class="md-nav__item">
<a href="../../user-guide/multiple-ingress/" title="Multiple Ingress controllers" class="md-nav__link">
Multiple Ingress controllers
</a>
</li>
<li class="md-nav__item">
<a href="../../user-guide/tls/" title="TLS/HTTPS" class="md-nav__link">
TLS/HTTPS
</a>
</li>
<li class="md-nav__item md-nav__item--nested">
<input class="md-toggle md-nav__toggle" data-md-toggle="nav-3-13" type="checkbox" id="nav-3-13">
<label class="md-nav__link" for="nav-3-13">
Third party addons
</label>
<nav class="md-nav" data-md-component="collapsible" data-md-level="2">
<label class="md-nav__title" for="nav-3-13">
Third party addons
</label>
<ul class="md-nav__list" data-md-scrollfix>
<li class="md-nav__item">
<a href="../../user-guide/third-party-addons/modsecurity/" title="ModSecurity Web Application Firewall" class="md-nav__link">
ModSecurity Web Application Firewall
</a>
</li>
<li class="md-nav__item">
<a href="../../user-guide/third-party-addons/opentracing/" title="OpenTracing" class="md-nav__link">
OpenTracing
</a>
</li>
</ul>
</nav>
</li>
</ul>
</nav>
</li>
<li class="md-nav__item md-nav__item--nested">
<input class="md-toggle md-nav__toggle" data-md-toggle="nav-4" type="checkbox" id="nav-4">
<label class="md-nav__link" for="nav-4">
Examples
</label>
<nav class="md-nav" data-md-component="collapsible" data-md-level="1">
<label class="md-nav__title" for="nav-4">
Examples
</label>
<ul class="md-nav__list" data-md-scrollfix>
<li class="md-nav__item">
<a href="../../examples/" title="Introduction" class="md-nav__link">
Introduction
</a>
</li>
<li class="md-nav__item">
<a href="../../examples/PREREQUISITES/" title="Prerequisites" class="md-nav__link">
Prerequisites
</a>
</li>
<li class="md-nav__item">
<a href="../../examples/affinity/cookie/" title="Sticky Sessions" class="md-nav__link">
Sticky Sessions
</a>
</li>
<li class="md-nav__item md-nav__item--nested">
<input class="md-toggle md-nav__toggle" data-md-toggle="nav-4-4" type="checkbox" id="nav-4-4">
<label class="md-nav__link" for="nav-4-4">
Auth
</label>
<nav class="md-nav" data-md-component="collapsible" data-md-level="2">
<label class="md-nav__title" for="nav-4-4">
Auth
</label>
<ul class="md-nav__list" data-md-scrollfix>
<li class="md-nav__item">
<a href="../../examples/auth/basic/" title="Basic Authentication" class="md-nav__link">
Basic Authentication
</a>
</li>
<li class="md-nav__item">
<a href="../../examples/auth/client-certs/" title="Client Certificate Authentication" class="md-nav__link">
Client Certificate Authentication
</a>
</li>
<li class="md-nav__item">
<a href="../../examples/auth/external-auth/" title="External Basic Authentication" class="md-nav__link">
External Basic Authentication
</a>
</li>
<li class="md-nav__item">
<a href="../../examples/auth/oauth-external-auth/" title="External OAUTH Authentication" class="md-nav__link">
External OAUTH Authentication
</a>
</li>
</ul>
</nav>
</li>
<li class="md-nav__item md-nav__item--nested">
<input class="md-toggle md-nav__toggle" data-md-toggle="nav-4-5" type="checkbox" id="nav-4-5">
<label class="md-nav__link" for="nav-4-5">
Customization
</label>
<nav class="md-nav" data-md-component="collapsible" data-md-level="2">
<label class="md-nav__title" for="nav-4-5">
Customization
</label>
<ul class="md-nav__list" data-md-scrollfix>
<li class="md-nav__item">
<a href="../../examples/customization/configuration-snippets/" title="Configuration Snippets" class="md-nav__link">
Configuration Snippets
</a>
</li>
<li class="md-nav__item">
<a href="../../examples/customization/custom-configuration/" title="Custom Configuration" class="md-nav__link">
Custom Configuration
</a>
</li>
<li class="md-nav__item">
<a href="../../examples/customization/custom-errors/" title="Custom Errors" class="md-nav__link">
Custom Errors
</a>
</li>
<li class="md-nav__item">
<a href="../../examples/customization/custom-headers/" title="Custom Headers" class="md-nav__link">
Custom Headers
</a>
</li>
<li class="md-nav__item">
<a href="../../examples/customization/external-auth-headers/" title="External authentication" class="md-nav__link">
External authentication
</a>
</li>
<li class="md-nav__item">
<a href="../../examples/customization/ssl-dh-param/" title="Custom DH parameters for perfect forward secrecy" class="md-nav__link">
Custom DH parameters for perfect forward secrecy
</a>
</li>
<li class="md-nav__item">
<a href="../../examples/customization/sysctl/" title="Sysctl tuning" class="md-nav__link">
Sysctl tuning
</a>
</li>
</ul>
</nav>
</li>
<li class="md-nav__item">
<a href="../../examples/docker-registry/" title="Docker registry" class="md-nav__link">
Docker registry
</a>
</li>
<li class="md-nav__item">
<a href="../../examples/grpc/" title="gRPC" class="md-nav__link">
gRPC
</a>
</li>
<li class="md-nav__item">
<a href="../../examples/multi-tls/" title="Multi TLS certificate termination" class="md-nav__link">
Multi TLS certificate termination
</a>
</li>
<li class="md-nav__item">
<a href="../../examples/rewrite/" title="Rewrite" class="md-nav__link">
Rewrite
</a>
</li>
<li class="md-nav__item">
<a href="../../examples/static-ip/" title="Static IPs" class="md-nav__link">
Static IPs
</a>
</li>
<li class="md-nav__item">
<a href="../../examples/tls-termination/" title="TLS termination" class="md-nav__link">
TLS termination
</a>
</li>
<li class="md-nav__item">
<a href="../../examples/psp/" title="Pod Security Policy (PSP)" class="md-nav__link">
Pod Security Policy (PSP)
</a>
</li>
</ul>
</nav>
</li>
</ul>
</nav>
</div>
</div>
</div>
<div class="md-sidebar md-sidebar--secondary" data-md-component="toc">
<div class="md-sidebar__scrollwrap">
<div class="md-sidebar__inner">
<nav class="md-nav md-nav--secondary">
<label class="md-nav__title" for="__toc">Table of contents</label>
<ul class="md-nav__list" data-md-scrollfix>
<li class="md-nav__item">
<a href="#table-of-contents" class="md-nav__link">
Table of Contents
</a>
</li>
<li class="md-nav__item">
<a href="#summary" class="md-nav__link">
Summary
</a>
</li>
<li class="md-nav__item">
<a href="#motivation" class="md-nav__link">
Motivation
</a>
<nav class="md-nav">
<ul class="md-nav__list">
<li class="md-nav__item">
<a href="#goals" class="md-nav__link">
Goals
</a>
</li>
<li class="md-nav__item">
<a href="#non-goals" class="md-nav__link">
Non-Goals
</a>
</li>
</ul>
</nav>
</li>
<li class="md-nav__item">
<a href="#proposal" class="md-nav__link">
Proposal
</a>
</li>
<li class="md-nav__item">
<a href="#implementation-history" class="md-nav__link">
Implementation History
</a>
</li>
<li class="md-nav__item">
<a href="#drawbacks-optional" class="md-nav__link">
Drawbacks [optional]
</a>
</li>
</ul>
</nav>
</div>
</div>
</div>
<div class="md-content">
<article class="md-content__inner md-typeset">
<a href="https://github.com/kubernetes/ingress-nginx/edit/master/docs/enhancements/20190815-zone-aware-routing.md" title="Edit this page" class="md-icon md-content__icon">&#xE3C9;</a>
<h1 id="availability-zone-aware-routing">Availability zone aware routing<a class="headerlink" href="#availability-zone-aware-routing" title="Permanent link"></a></h1>
<h2 id="table-of-contents">Table of Contents<a class="headerlink" href="#table-of-contents" title="Permanent link"></a></h2>
<!-- toc -->
<ul>
<li><a href="#summary">Summary</a></li>
<li><a href="#motivation">Motivation</a></li>
<li><a href="#goals">Goals</a></li>
<li><a href="#non-goals">Non-Goals</a></li>
<li><a href="#proposal">Proposal</a></li>
<li><a href="#implementation-history">Implementation History</a></li>
<li><a href="#drawbacks-optional">Drawbacks [optional]</a>
<!-- /toc --></li>
</ul>
<h2 id="summary">Summary<a class="headerlink" href="#summary" title="Permanent link"></a></h2>
<p>Teach ingress-nginx about availability zones where endpoints are running in. This way ingress-nginx pod will do its best to proxy to zone-local endpoint.</p>
<h2 id="motivation">Motivation<a class="headerlink" href="#motivation" title="Permanent link"></a></h2>
<p>When users run their services across multiple availability zones they usually pay for egress traffic between zones. Providers such as GCP, Amazon EC charges money for that.
ingress-nginx when picking an endpoint to route request to does not consider whether the endpoint is in different zone or the same one. That means it's at least equally likely
that it will pick an endpoint from another zone and proxy the request to it. In this situation response from the endpoint to ingress-nginx pod is considered as
inter zone traffic and costs money.</p>
<p>At the time of this writing GCP charges $0.01 per GB of inter zone egress traffic according to https://cloud.google.com/compute/network-pricing.
According to https://datapath.io/resources/blog/what-are-aws-data-transfer-costs-and-how-to-minimize-them/ Amazon also charges the same amount of money sa GCP for cross zone, egress traffic.</p>
<p>This can be a lot of money depending on once's traffic. By teaching ingress-nginx about zones we can eliminate or at least decrease this cost.</p>
<p>Arguably inter-zone network latency should also be better than cross zone.</p>
<h3 id="goals">Goals<a class="headerlink" href="#goals" title="Permanent link"></a></h3>
<ul>
<li>Given a regional cluster running ingress-nginx, ingress-nginx should do best effort to pick zone-local endpoint when proxying</li>
<li>This should not impact canary feature</li>
<li>ingress-nginx should be able to operate successfully if there's no zonal endpoints</li>
</ul>
<h3 id="non-goals">Non-Goals<a class="headerlink" href="#non-goals" title="Permanent link"></a></h3>
<ul>
<li>This feature inherently assumes that endpoints are distributed across zones in a way that they can handle all the traffic from ingress-nginx pod(s) in that zone</li>
<li>This feature will be relying on https://kubernetes.io/docs/reference/kubernetes-api/labels-annotations-taints/#failure-domainbetakubernetesiozone, it is not this KEP's goal to support other cases</li>
</ul>
<h2 id="proposal">Proposal<a class="headerlink" href="#proposal" title="Permanent link"></a></h2>
<p>The idea here is to have controller part of ingress-nginx to (1) detect what zone its current pod is running in and (2) detect the zone for every endpoints it knows about.
After that it will post that data as part of endpoints to Lua land. Then Lua balancer when picking an endpoint will try to pick zone-local endpoint first and
if there is no zone-local endpoint then it will fallback to current behaviour.</p>
<p>This feature at least in the beginning should be optional since it is going to make it harder to reason about the load balancing and not everyone might want that.</p>
<p><strong>How does controller know what zone it runs in?</strong>
We can have the pod spec do pass node name using downward API as an environment variable.
Then on start controller can get node details from the API based on node name. Once the node details is obtained
we can extract the zone from <code class="codehilite"><span class="err">failure-domain.beta.kubernetes.io/zone</span></code> annotation. Then we can pass that value to Lua land through Nginx configuration
when loading <code class="codehilite"><span class="err">lua_ingress.lua</span></code> module in <code class="codehilite"><span class="err">init_by_lua</span></code> phase.</p>
<p><strong>How do we extract zones for endpoints?</strong>
We can have the controller watch create and update events on nodes in the entire cluster and based on that keep the map of nodes to zones in the memory.
And when we generate endpoints list, we can access node name using <code class="codehilite"><span class="p">.</span><span class="n">subsets</span><span class="p">.</span><span class="n">addresses</span><span class="o">[</span><span class="n">i</span><span class="o">]</span><span class="p">.</span><span class="n">nodeName</span><span class="w"></span></code>
and based on that fetch zone from the map in memory and store it as a field on the endpoint.
<strong>This solution assumes <code class="codehilite"><span class="err">failure-domain.beta.kubernetes.io/zone</span></code></strong> annotation does not change until the end of node's life. Otherwise we have to
watch update events as well on the nodes and that'll add even more overhead.</p>
<p>Alternatively, we can get the list of nodes only when there's no node in the memory for given node name. This is probably a better solution
because then we would avoid watching for API changes on node resources. We can eagrly fetch all the nodes and build node name to zone mapping on start.
And from thereon sync it during endpoints building in the main event loop iff there's no entry exist for the node of an endpoint.
This means an extra API call in case cluster has expanded.</p>
<p><strong>How do we make sure we do our best to choose zone-local endpoint?</strong>
This will be done on Lua side. For every backend we will initialize two balancer instances: (1) with all endpoints
(2) with all endpoints corresponding to current zone for the backend. Then given the request once we choose what backend
needs to serve the request, we will first try to use zonal balancer for that backend. If zonal balancer does not exist (i.e there's no zonal endpoint)
then we will use general balancer. In case of zonal outages we assume that readiness probe will fail and controller will
see no endpoints for the backend and therefore we will use general balancer.</p>
<p>We can enable the feature using a configmap setting. Doing it this way makes it easier to rollback in case of a problem.</p>
<h2 id="implementation-history">Implementation History<a class="headerlink" href="#implementation-history" title="Permanent link"></a></h2>
<ul>
<li>initial version of KEP is shipped</li>
<li>proposal and implementation details is done</li>
</ul>
<h2 id="drawbacks-optional">Drawbacks [optional]<a class="headerlink" href="#drawbacks-optional" title="Permanent link"></a></h2>
<p>More load on the Kubernetes API server.</p>
</article>
</div>
</div>
</main>
<footer class="md-footer">
<div class="md-footer-meta md-typeset">
<div class="md-footer-meta__inner md-grid">
<div class="md-footer-copyright">
powered by
<a href="https://www.mkdocs.org" target="_blank" rel="noopener">MkDocs</a>
and
<a href="https://squidfunk.github.io/mkdocs-material/" target="_blank" rel="noopener">
Material for MkDocs</a>
</div>
</div>
</div>
</footer>
</div>
<script src="../../assets/javascripts/application.df00da5d.js"></script>
<script>app.initialize({version:"1.0.4",url:{base:"../.."}})</script>
</body>
</html>