Have you found that NGINX is returning 502 unless you update the service every time you update the Django container image while running a docker swarm? I have, here’s a solution.
I inherited a project that is running as docker swarm in multiple virtual machines.
The core service is a Django website being served over NGINX, let’s ignore all the other microservices for today's blog.
The documentation left behind told me, after updating the Django container image, to update the NGINX service too.
Not a big deal, nevertheless annoying, and at least on one occasion I forgot and the website returned 502 for some ten minutes.
Now, *surely* there’s a better way to do this. Once I got through the pile of higher priority tasks in my to-do list I set out to find said better way. Mind I had no NGINX familiarity to guide me.
Long story short, I found many similar questions in my search (from how to start containers in a specific order on), but struggled to find a solution.
So let’s cut to the chase and here’s how I fixed it:
The issue: NGINX resolving the IP only once
The underlying issue turned out to be that NGINX was only resolving the IP of the container at start time (or reloading the service).
Here’s what was happening: my website is running as a docker container, NGINX is started as a docker container and finds out my website IP is 10.0.0.x.
I do my coding, have my new feature ready and push it to github. My new image is automatically built and I update the service in my swarm. My website is now a new container and got a new 10.0.0.y IP, but NGINX is still redirecting to (the now non-existent) 10.0.0.x.
The solution: use variables
A workaround to make NGINX resolve the IP at every request is to use variables.
So I created a variable to reference the Django port.
You’ll also have to make sure you’re using the docker DNS resolver (default IP is 127.0.0.11)
location / {
resolver 127.0.0.11;
set $portdocker 8000;
proxy_pass http://my-website:$portdocker;
[extra config as required]
}Fancy addition: don’t have your user reload the page
No more need to manually update the NGINX service!
But in the unlikely event (in my case) a user was to navigate to the website just during the update they would get 502 and would have to reload the page a few seconds/minutes later.
Instead of this I created a custom 502.html with a meta tag
<meta http-equiv="refresh" content="60;url=http://my-website-landing-page.com/" /> pointing to the landing page of my website.
Now if a user lands on the website just during the update, they’ll still get error 502, but the page will reload automatically in a minute and take them to the entry page.
Is there an even better way? Probably.
Is there an even better way considering my/your time constraints? You judge.
Comments