I have Nginx running as a reverse proxy on a computer with only one open port. Through this port and Nginx I redirect the received requests to several internal servers. Now I need to run InfluxDB on this computer, but the client writing to InfluxDB is on another computer.
My first idea was to add a new location to redirect input requests since port 8086 is closed, for example:
location /databasets {
proxy_set_header Host $host;
proxy_set_header X-Real-IP $remote_addr;
proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
proxy_set_header X-NginX-Proxy true;
proxy_pass http://localhost:8086;
}
and then, with Python, I use:
client = InfluxDBClient(host='https://myurl', port=10000, 'root', 'root', dbname='mydb', path='databasets', ssl=True, proxies={"https": "https://myurl:10000/databasets"})
But so far it doesn't work, I have tried a couple of ways of configuring the nginx.conf file that I have seen on the internet and also changing the host / port in the Python client. I don't know if this is not possible, or on which side is the error, any ideas?
Thanks in advance
Add the following config in your nginx config
location /databasets/ {
proxy_pass http://localhost:8086;
rewrite `^/databasets/(.*) /$1 break`;
proxy_set_header Host $host;
}
The input url needs to be rewritten
I have an API on FastAPI and i need to get the client real IP address when he request my page.
I'm ty to use starlette Request. But it returns my server IP, not client remote IP.
My code:
#app.post('/my-endpoint')
async def my_endpoint(stats: Stats, request: Request):
ip = request.client.host
print(ip)
return {'status': 1, 'message': 'ok'}
What i'm doing wrong? How to get real IP (like in Flask request.remote_addr)?
request.client should work, unless you're running behind a proxy (e.g. nginx) in that case use uvicorn's --proxy-headers flag to accept these incoming headers and make sure the proxy forwards them.
The FastAPI using-request-directly doc page shows this example:
from fastapi import FastAPI, Request
app = FastAPI()
#app.get("/items/{item_id}")
def read_root(item_id: str, request: Request):
client_host = request.client.host
return {"client_host": client_host, "item_id": item_id}
Having had this example would have saved me ten minutes of mussing with Starlette's Request class
You don't need to set --proxy-headers bc it is enabled by default, but it only trusts IPs from --forwarded-allow-ips which defaults to 127.0.0.1
To be safe, you should only trust proxy headers from the ip of your reverse proxy (instead of trust all with '*'). If it's on the same machine then the defaults should work. Although I noticed from my nginx logs that it was using ip6 to communicate with uvicorn so I had to use --forwarded-allow-ips='[::1]' then I could see the ip addresses in FastAPI. You can also use --forwarded-allow-ips='127.0.0.1,[::1]' to catch both ip4 and ip6 on localhost.
--proxy-headers / --no-proxy-headers - Enable/Disable X-Forwarded-Proto, X-Forwarded-For, X-Forwarded-Port to populate remote address info. Defaults to enabled, but is restricted to only trusting connecting IPs in the forwarded-allow-ips configuration.
--forwarded-allow-ips - Comma separated list of IPs to trust with proxy headers. Defaults to the $FORWARDED_ALLOW_IPS environment variable if available, or '127.0.0.1'. A wildcard '*' means always trust.
Ref: https://www.uvicorn.org/settings/#http
if you use the nginx and uvicorn，you should set proxy-headers for uvicorn，and your nginx config should be add Host、X-Real-IPand X-Forwarded-For.
e.g.
server {
# the port your site will be served on
listen 80;
# the domain name it will serve for
server_name <your_host_name>; # substitute your machine's IP address or FQDN
# add_header Access-Control-Allow-Origin *;
# add_header Access-Control-Allow-Credentials: true;
add_header Access-Control-Allow-Headers Content-Type,XFILENAME,XFILECATEGORY,XFILESIZE;
add_header access-control-allow-headers authorization;
# Finally, send all non-media requests to the Django server.
location / {
proxy_pass http://127.0.0.1:8000/; # the uvicorn server address
proxy_set_header Host $host;
proxy_set_header X-Real-IP $remote_addr;
proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
}
}
on the nginx document:
This middleware can be applied to add HTTP proxy support to an
application that was not designed with HTTP proxies in mind. It
sets REMOTE_ADDR, HTTP_HOST from X-Forwarded headers. While
Werkzeug-based applications already can use
:py:func:werkzeug.wsgi.get_host to retrieve the current host even if
behind proxy setups, this middleware can be used for applications which
access the WSGI environment directly。
If you have more than one proxy server in front of your app, set
num_proxies accordingly.
Do not use this middleware in non-proxy setups for security reasons.
The original values of REMOTE_ADDR and HTTP_HOST are stored in
the WSGI environment as werkzeug.proxy_fix.orig_remote_addr and
werkzeug.proxy_fix.orig_http_host
:param app: the WSGI application
:param num_proxies: the number of proxy servers in front of the app.
If you have configured your nginx configuration properly based on #AllenRen's answer,
Try using --proxy-headers and also --forwarded-allow-ips='*' flags for uvicorn.
You would use the below code to getting the real-IP address from the client. If you have using reverse proxying and port forwarding
#app.post('/my-endpoint')
async def my_endpoint(stats: Stats, request: Request):
x = 'x-forwarded-for'.encode('utf-8')
for header in request.headers.raw:
if header[0] == x:
print("Find out the forwarded-for ip address")
origin_ip, forward_ip = re.split(', ', header[1].decode('utf-8'))
print(f"origin_ip:\t{origin_ip}")
print(f"forward_ip:\t{forward_ip}")
return {'status': 1, 'message': 'ok'}
I have deployed with docker-compose file and changes are
nginx. conf file
location / {
proxy_set_header Host $host;
proxy_set_header X-Real-IP $remote_addr;
proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
proxy_pass http://localhost:8000;
}
Changes in Dockerfile
EXPOSE 8000
CMD ["uvicorn", "app.main:app", "--host", "0.0.0.0"]
Changes in docker-compose.yaml file
version: "3.7"
services:
app:
build: ./fastapi
container_name: ipinfo
restart: always
ports:
- "8000:8000"
network_mode: host
nginx:
build: ./nginx
container_name: nginx
restart: always
ports:
- "80:80"
- "443:443"
network_mode: host
After these changes got client external IP correctly
Sharing what has worked for me on an Apache server setup on a stand-alone ubuntu-based web-server instance/droplet (Amazon EC2 / DigitalOcean / Hetzner / SSDnodes). TL;DR : use X_Forwarded_For
I'm assuming you have a domain name registered and are pinning your server to it.
In the code
from fastapi import FastAPI, Header
app = FastAPI()
#app.get("/API/path1")
def path1(X_Forwarded_For: Optional[str] = Header(None)):
print("X_Forwarded_For:",X_Forwarded_For)
return { "X_Forwarded_For":X_Forwarded_For }
This gives a null when running in local machine and hitting localhost:port/API/path1 , but in my deployed site it's properly giving my IP address when I hit the API.
In the program launch command
uvicorn launch1:app --port 5010 --host 0.0.0.0 --root-path /site1
main program is in launch1.py . Note the --root-path arg here - that's important if your application is going to deployed not at root level of a URL.
This takes care of url mappings, so in the program code above we didn't need to include it in the #app.get line. Makes the program portable - tomorrow you can move it from /site1 to /site2 path without having to edit the code.
In the server setup
The setting on my web-server:
Apache server is setup
LetsEncrypt SSL is enabled
Edit /etc/apache2/sites-available/[sitename]-le-ssl.conf
Add these lines inside <VirtualHost *:443> tag:
ProxyPreserveHost On
ProxyPass /site1/ http://127.0.0.1:5010/
ProxyPassReverse /site1/ http://127.0.0.1:5010/
Enable proxy_http and restart Apache
a2enmod proxy_http
systemctl restart apache2
some good guides for server setup:
https://www.digitalocean.com/community/tutorials/how-to-secure-apache-with-let-s-encrypt-on-ubuntu-20-04
https://www.digitalocean.com/community/tutorials/how-to-use-apache-http-server-as-reverse-proxy-using-mod_proxy-extension-ubuntu-20-04
With this all setup, you can hit your api endpoint on https://[sitename]/site1/API/path1 and should see the same IP address in the response as what you see on https://www.whatismyip.com/ .
I have docker-compose and nginx proxy. The following helped:
in forwarded-allow-ips specified '*' (environment variable in docker-compose.yml file)
- FORWARDED_ALLOW_IPS=*
Added the code to the nginx.conf file as recommended by #allenren
location /api/ {
proxy_pass http://backend:8000/;
proxy_set_header Host $host;
proxy_set_header X-Real-IP $remote_addr;
proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
}
Using the Header dependency should let you access the X-Real-IP header.
from fastapi import FastAPI, Depends, Header
app = FastAPI()
#app.get('/')
def index(real_ip: str = Header(None, alias='X-Real-IP')):
return real_ip
Now if you start the server (in this case on port 8000) and hit it with a request with that X-Real-IP header set you should see it echo back.
http :8000/ X-Real-IP:111.222.333.444
HTTP/1.1 200 OK
content-length: 17
content-type: application/json
server: uvicorn
"111.222.333.444"
If you are using nginx as a reverse proxy; the direct solution is to include the proxy_params file like so:
location /api {
include proxy_params;
proxy_pass http://localhost:8000;
}
I am simply running a flask app and not using nginx and uwsgi, yes my host is behind the load blancer .
I am trying to read all the keys which can read the IP address, but I am not getting the actual IP of the client.
X-Real-IP is changing on every request and X-Forwarded-For has only one IP address which is the loadbalancer IP.
Same issue with bottle. When I started the application directly python app.py , I am not able to get the real IP address.
Is this must to use uwsgi and nginx for a sample app to read IP?
If I use below configuration and forward the uwsgi_param I can read the list of IP address in the response.
Below wsgi_file.ini
[uwsgi]
socket = 127.0.0.1:8000
plugin = python
wsgi-file = app/app.py
process = 3
callable = app
nginx.conf
server {
listen 3000;
access_log /var/log/nginx/access.log;
error_log /var/log/nginx/error.log;
location / {
uwsgi_pass 0.0.0.0:8000; #unix:///tmp/uwsgi.sock;
include /etc/nginx/uwsgi_params;
uwsgi_param X-Real-IP $remote_addr;
uwsgi_param X-Forwarded-For $proxy_add_x_forwarded_for;
uwsgi_param X-Forwarded-Proto $http_x_forwarded_proto;
}
}
I started the nginx server and ran the application using command:
uwsgi --ini wsgi_file.ini.
The IP address of the client can be obtained in Flask with request.remote_addr.
Note that if you are using a reverse proxy, load balancer or any other intermediary between the client and the server, then this is going to return the IP address of the last intermediary, the one that sends requests directly into the Flask server. If the intermediaries include the X-Real-IP, X-Forwarded-For or Forwarded headers, then you can still figure out the IP address of the client.
I have a Flask website running inside of a docker container. I would like to display the user's ip address to them, however it currently displays the IP address of the docker container, probably due to it being forwarded from docker.
I retrieve the IP address using Flask's requests module.
It shouldn't make a difference, but I am using docker-compose.
version: '2'
services:
web:
build: .
ports:
- "5000:5000"
volumes:
- .:/code
redis:
image: "redis:alpine"
This is the code for the website using Flask:
from flask import Flask, request
from redis import Redis
app = Flask(__name__)
redis = Redis(host='redis', port=6379)
#app.route('/')
def hello():
count = redis.incr('hits')
return 'Your IP is: {0}. I have been visited {1} times and \'foo\' is set to {2}.\n'.format(request.remote_addr,count, redis.get('foo').decode('utf-8'))
if __name__ == "__main__":
app.run(host="0.0.0.0", debug=True)
I know that Nginx gives the request a header with the ip address before it was forwarded that you can use. Does Docker have anything similar that I can use or is there another approach. Get IP address of visitors using Python + Flask
Example of current behavior:
Thank You.
I disagree with the comments. Depending on how much of the deployment you are in control of this is doable. You should add something like the following to the NGINX config:
location / {
proxy_pass http://web;
proxy_set_header Connection "";
proxy_http_version 1.1;
proxy_set_header Host $host;
proxy_set_header X-Real-IP $remote_addr;
proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
}
I am setting up a flask+gunicorn+nginx webapp and during my initial prototyping stage I have got response from the gunicorn through nginx, however in the process nginx adds the following to the body of the http response, corrupting the html code to be interpreted by the browser.
This is added to top of the body:
Server: gunicorn/19.7.1
Date: Thu, 27 Apr 2017 02:05:11 GMT
Connection: close
Content-Type: text/html; charset=utf-8
Content-Length: 12
My hello world flask code test.py is:
from flask import Flask
# Import the fixer
from werkzeug.contrib.fixers import ProxyFix
app = Flask(__name__)
# Use the fixer
app.wsgi_app = ProxyFix(app.wsgi_app)
#app.route('/')
def index():
return "Hello World!"
I run the following in the directory of test.py to start gunicorn:
gunicorn test:app
and finally restart nginx v1.8.1 with the following nginx.conf file:
worker_processes 1;
events {
worker_connections 1024;
}
http {
server {
listen 80;
server_name localhost;
location / {
# Pass the request to Gunicorn
proxy_pass http://127.0.0.1:8000;
# Set some HTTP headers so that our app knows where the
# request really came from
proxy_set_header Host $host;
proxy_set_header X-Real-IP $remote_addr;
proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
}
}
}
What is the proper way for disabling this modification to the response? Am I missing somthing in my nginx conf file?