Flask : double slash in urls - python

I'm having some trouble with Flask deployed under gunicorn + nginx. My website is under a reverse proxy to go on the url : http://example.com/identity/.
I have several urls in my Flask application and for each one (for example "index") :
when I try to access to http://example.com/identity/index/ (with the trailing slash) it goes right to the url
when I try to access to http://example.com/identity/index (without the trailing slash) it goes to http://example.com/identity//index/? (notice the double slash and question mark).
In Flask, the route associated to index is : #route('/identity/index'). I guess my problem is a "normal" Flask behaviour but I would like to access index without any slash and have my normal url. Same goes with redirections using redirect(url_for('index')) for example.

Actually it was a problem with my Nginx configuration, I was running under https from the outside and under http between my servers. So I used this snippet : http://flask.pocoo.org/snippets/35/ to force https urls in my application and now it works !

Related

has been blocked by CORS policy: Request header field access-control-allow-origin is not allowed by Access-Control-Allow-Headers in preflight response [duplicate]

I am working on an app using Vue js.
According to my setting I need to pass to a variable to my URL when setting change.
<!-- language: lang-js -->
$.get('http://172.16.1.157:8002/firstcolumn/' + c1v + '/' + c1b, function (data) {
// some code...
});
But when my app hit on URL, it shows the following message.
Failed to load http://172.16.1.157:8002/firstcolumn/2017-03-01/2017-10-26: Redirect from 'http://172.16.1.157:8002/firstcolumn/2017-03-01/2017-10-26' to 'http://172.16.1.157:8002/firstcolumn/2017-03-01/2017-10-26/' has been blocked by CORS policy: No 'Access-Control-Allow-Origin' header is present on the requested resource. Origin 'http://localhost:8080' is therefore not allowed access.
In addition to what awd mentioned about getting the person responsible for the server to reconfigure (an impractical solution for local development) I use a change-origin chrome plugin like this:
Moesif Orign & CORS Changer (use to be free but now wants a work email address >_>)
Allow CORS: Access-Control-Allow-Origin
You can make your local dev server (ex: localhost:8080) to appear to be coming from 172.16.1.157:8002 or any other domain.
In case the 2nd plugin link breaks in the future or the plugin writer decides to capitalize off the fame of this thread, open your browser's
plugin marketplace and search "allow cors", there's going to be a
bunch of them.
Thanks all, I solved by this extension on chrome.
Allow CORS: Access-Control-Allow-Origin
If you have control over your server, you can use PHP:
<?PHP
header('Access-Control-Allow-Origin: *');
?>
Ask the person maintaining the server at http://172.16.1.157:8002/ to add your hostname to Access-Control-Allow-Origin hosts, the server should return a header similar to the following with the response-
Access-Control-Allow-Origin: yourhostname:port
Using npm:
To allow cross-origin requests install 'cors':
npm i cors
Add this in the server-side:
let cors = require("cors");
app.use(cors());
When you have this problem with Chrome, you don't need an Extension.
Start Chrome from the Console:
chrome.exe --user-data-dir="C:/Chrome dev session" --disable-web-security
Maybe you have to close all Tabs in Chrome and restart it.
I will assume that you're a front-end developer only and that you don't have access to the backend of the application (regarding the tags of the question).
Short answer on how to properly solve this in your case? You can't, you'll need somebody else.
What is this about?
You need to understand that CORS is a security thing, it's not just here to annoy you just for fun.
It's purpose is to mainly prevent the usage of a (malicious) HTTP call from a non-whitelisted frontend to your backend with some critical mutation.
You could give a look to this YouTube video or any other one really, but I recommend a visual video because text-based explanation can be quite hard to understand.
You also need to understand that if you use Postman or any other tool to try your API call, you will not get the CORS issue. The reason being that those tools are not Web frontends but rather some server-based tools.
Hence, don't be surprised if something is working there but not in your Vue app, the context is different.
Now, how to solve this?
Depending of the framework used by your backend team, the syntax may be quite different but overall, you'll need to tell them to provide something like Access-Control-Allow-Origin: http://localhost:3000 (or any other port you'll be using).
PS: Using Access-Control-Allow-Origin: * would be quite risky because it would allow anybody to access it, hence why a stricter rule is recommended.
If you're using a service, like an API to send SMS, payment, some Google console or something else really, you'll need to allow your localhost in the dashboard of the service. Ask for credentials to your manager or Tech Lead.
If you have access to the backend, you could it yourself as shown here (ExpressJS in this example): https://flaviocopes.com/cors/
How to hack it in a dirty way?
If you're in a damn hurry and want to get something really dirty, you could use a lot of various hacks a listed in the other answers, here's a quick list:
use any extension who is able to create a middleware and forward the request to the backend (it will work because it's not directly coming from your frontend)
force your browser to disable CORS, not sure how this would actually solve the issue
use a proxy, if you're using Nuxt2, #nuxtjs/proxy is a popular one but any kind of proxy (even a real backend will do the job)
any other hack related somehow to the 3 listed above...
At the end, solving the CORS issue can be done quite fast and easily. You only need to communicate with your team or find something on your side (if you have access to the backend/admin dashboard of some service).
I heavily do recommend trying get it right from the beginning because it's related to security and that it may be forgotten down the road...
The approved answer to this question is not valid.
You need to set headers on your server-side code
app.use((req,res,next)=>{
res.setHeader('Access-Control-Allow-Origin','*');
res.setHeader('Access-Control-Allow-Methods','GET,POST,PUT,PATCH,DELETE');
res.setHeader('Access-Control-Allow-Methods','Content-Type','Authorization');
next();
})
You can also try a chrome extension to add these headers automatically.
Hello If I understood it right you are doing an XMLHttpRequest to a different domain than your page is on. So the browser is blocking it as it usually allows a request in the same origin for security reasons. You need to do something different when you want to do a cross-domain request. A tutorial about how to achieve that is Using CORS.
When you are using postman they are not restricted by this policy. Quoted from Cross-Origin XMLHttpRequest:
Regular web pages can use the XMLHttpRequest object to send and receive data from remote servers, but they're limited by the same origin policy. Extensions aren't so limited. An extension can talk to remote servers outside of its origin, as long as it first requests cross-origin permissions.
To add the CORS authorization to the header using Apache, simply add the following line inside either the <Directory>, <Location>, <Files> or <VirtualHost> sections of your server config (usually located in a *.conf file, such as httpd.conf or apache.conf), or within a .htaccess file:
Header set Access-Control-Allow-Origin "*"
And then restart apache.
Altering headers requires the use of mod_headers. Mod_headers is enabled by default in Apache, however, you may want to ensure it's enabled.
I had the same problem in my Vue.js and SpringBoot projects. If somebody work with spring you can add this code:
#Bean
public FilterRegistrationBean simpleCorsFilter() {
UrlBasedCorsConfigurationSource source = new UrlBasedCorsConfigurationSource();
CorsConfiguration config = new CorsConfiguration();
config.setAllowCredentials(true);
// *** URL below needs to match the Vue client URL and port ***
config.setAllowedOrigins(Collections.singletonList("http://localhost:8080"));
config.setAllowedMethods(Collections.singletonList("*"));
config.setAllowedHeaders(Collections.singletonList("*"));
source.registerCorsConfiguration("/**", config);
FilterRegistrationBean bean = new FilterRegistrationBean<>(new CorsFilter(source));
bean.setOrder(Ordered.HIGHEST_PRECEDENCE);
return bean;
}
I found solution in this article Build a Simple CRUD App with Spring Boot and Vue.js
You are making a request to external domain 172.16.1.157:8002/ from your local development server that is why it is giving cross origin exception.
Either you have to allow headers Access-Control-Allow-Origin:* in both frontend and backend or alternatively use this extension cors header toggle - chrome extension unless you host backend and frontend on the same domain.
Try running this command in your terminal and then test it again.
curl -H "origin: originHost" -v "RequestedResource"
Eg:
If my originHost equals https://localhost:8081/ and my RequestedResource equals https://example.com/
My command would be as below:
curl -H "origin: https://localhost:8081/" -v "https://example.com/"
If you can notice the following line then it should work for you.
< access-control-allow-origin: *
Hope this helps.
Do specify #CrossOrigin(origins = "http://localhost:8081")
in Controller class.
You can solve this temporarily by using the Firefox add-on, CORS Everywhere. Just open Firefox, press Ctrl+Shift+A , search the add-on and add it!
You won't believe this,
Make sure to add "." at the end of the "url"
I got a similar error with this code:
fetch(https://itunes.apple.com/search?term=jack+johnson)
.then( response => {
return response.json();
})
.then(data => {
console.log(data.results);
}).catch(error => console.log('Request failed:', error))
The error I got:
Access to fetch at 'https://itunes.apple.com/search?term=jack+johnson'
from origin 'http://127.0.0.1:5500' has been blocked by CORS policy:
No 'Access-Control-Allow-Origin' header is present on the requested
resource. If an opaque response serves your needs, set the request's mode to 'no-cors' to fetch the resource with CORS disabled.
But I realized after a lot of research that the problem was that I did not copy the
right URL address from the iTunes API documentation.
It should have been
https://itunes.apple.com/search?term=jack+johnson.
not
https://itunes.apple.com/search?term=jack+johnson
Notice the dot at the end
There is a huge explanation about why the dot is important quoting issues about DNS and character encoding but the truth is you probably do not care. Try adding the dot it might work for you too.
When I added the "." everything worked like a charm.
I hope it works for you too.
install:
npm i cors
Then include cors():
app.get("/list",cors(),(req,res) =>{
});
In addition to the Berke Kaan Cetinkaya's answer.
If you have control over your server, you can do the following in ExpressJs:
app.use(function(req, res, next) {
// update to match the domain you will make the request from
res.header("Access-Control-Allow-Origin", "YOUR-DOMAIN.TLD");
res.header("Access-Control-Allow-Methods", "GET,HEAD,OPTIONS,POST,PUT");
res.header("Access-Control-Allow-Headers", "Origin, X-Requested-With, Content-Type, Accept");
next();
});
https://enable-cors.org/server_expressjs.html
I tried this code,and that works for me.You can see the documentation in this link
var io = require("socket.io")(http, {
cors: {
origin: "*",
methods: ["GET", "POST"]
}
})
The reason that I came across this error was that I hadn't updated the path for different environments.
you have to customize security for your browser or allow permission through customizing security. (it is impractical for your local testing)
to know more about please go through the link.
https://developer.mozilla.org/en-US/docs/Web/HTTP/CORS
These errors may be caused due to follow reasons, ensure the following steps are followed. To connect the local host with the local virtual machine(host). Here, I'am connecting http://localhost:3001/ to the http://abc.test Steps to be followed:
1.We have to allow CORS, placing Access-Control-Allow-Origin: in header of request
may not work. Install a google extension which enables a CORS request.*
2.Make sure the credentials you provide in the request are valid.
3.Make sure the vagrant has been provisioned. Try vagrant up --provision this make the localhost connect to db of the homestead.
Try changing the content type of the header. header:{ 'Content-Type' : 'application/x-www-form-urlencoded; charset=UTF-8;application/json' }
this point is very important.
Another solution to this problem in a specific scenario :
If
AWS APIGW is your backend with authentication enabled and
authentication fails,
your browser may end up complaining about CORS even if CORS is enabled in APIGW. You also need to enable CORS for 4XX as follows
API:YourAPI > Resources > /YourResource > Actions > Enable CORS > Gateway Responses for yourAPI check Default 4XX
Authentication will still fail but it won't look like CORS is the root cause
$.get('https://172.16.1.157:8002/firstcolumn/' + c1v + '/' + c1b, function (data) {
// some code...
});
Just put "https" .

Weird `/` behavior when Serving Flask with Apache2 + Gunicorn

I'm trying to build multiple endpoints and subendpoints within my application, part of it as a learning exercise, and part of it is that I have 2 domains.
For simplicity I'm going to refer to them as domain1 and domain2.
My Flask listening endpoints are on /api1 and /api2 for domains 1 & 2 respectively. Gunicorn is bound to listen on a unix socket at sock/domain1.sock and sock/domain2.sock. So far everything is working this way.
My Apache2 proxies the endpoints into the proper socket as follows:
for domain1 I have:
<Location /api>
ProxyPass unix:/var/www/socks/domain1.sock|http://127.0.0.1/api1
ProxyPassReverse unix:/var/www/socks/domain1.sock|http://127.0.0.1/api1
</Location>
for domain2 I have:
<Location /api>
ProxyPass unix:/var/www/socks/domain2.sock|http://127.0.0.1/api2
ProxyPassReverse unix:/var/www/socks/domain2.sock|http://127.0.0.1/api2
</Location>
I know that I don't need to have 2 sockets, but I'm doing so just for testing.
Now when I open domain1.com/api things are working perfectly. And so are for domain2.com/api
But when I open domain1.com/api/ (with a slash at the end) or domain2.com/api/ then it gives me a Site Not Found error. This is understandable since in my Flask I'm listening to the endpoint without a trailing slash. The fix for that is to implement / into my flask endpoint. So when I do that, the weird behavior occurs.
New Flask listening Endpoints are /api1/ and /api2/ (with trailing slash).
Now when I open domain.com/api/ it is working as intended. But when I'm on domain.com/api (without the slash) it's referring me to either domain.comapi or 127.0.01/api, where both are wrong scenarios. I tried to add a trailing slash in my Apache config, and tried multiple Flask approaches but they're all doing the same weird behavior and I can't understand why it's doing that. Now personally I don't mind using the endpoint without the slash, I just want to understand why this is happening. I also tried googling a lot but nothing came up related to my query.
Reproduceable Behavior:
I'm unable to link the 2nd domain as it is a protected IP for my company, so I created multiple endpoints so that you can click on to simulate the behavior.
https://thethiny.xyz/api1 -> sock|http://127.0.0.1/api1 -> internal /api1
https://thethiny.xyz/api2 -> sock|http://127.0.0.1/api2 -> internal /api2/
https://thethiny.xyz/api3 -> sock|http://127.0.0.1/api1/ -> internal /api1
https://thethiny.xyz/api4 -> sock|http://127.0.0.1/api2/ -> internal /api2
Working:
https://thethiny.xyz/api1
https://thethiny.xyz/api2/
https://thethiny.xyz/api4
Not Found:
https://thethiny.xyz/api1/
https://thethiny.xyz/api3
https://thethiny.xyz/api3/
Weird Redirect:
https://thethiny.xyz/api2
https://thethiny.xyz/api4/
Edit: I understand the problem and have came up with some solutions in the answer below. I'm not satisfied with the solutions but I'm taking this as a limitation of mapping endpoints to different underlying endpoints. For more information, read about Reverse Proxy Pass and Redirects and Rewriting Location Header in HTTPd
I now understand the problem. So in my Apache Proxy it is giving the request to Flask on the endpoint specified 127.0.0.1/api2, so when there's a redirect request from within Flask, it tries to redirect to 127.0.0.1/api2/, since Flask doesn't have any information about the original url source. Using ProxyPreserveHost solves this only when the endpoint resources match, as in mapping /api2/ to /api2 but not /api4/ to /api2/, since on the redirect, Flask receives a request for /api2 -> /api2/ and returns that having no information about /api4. Unfortunately I don't there's an actual solution to this from Apache2/Flask configurations other than manually handling the routes specifically to how you want them to be, as in do not allow Flask to redirect automatically since it will not know how, and instead either manually redirect (external redirect) to the correct endpoint, or handle each route separately (/api and /api/stuff but not /api/).
Example:
app.add_url_rule("/api2", view_func=StubFunction(), redirect_to="/api2/")
app.add_url_rule("/api2/", view_func=ActualFunction())
And add ProxyPreserveHost On to your Apache2 config or use the built in Proxy Fixer if you don't want to modify your Virtual Hosts:
from werkzeug.middleware.proxy_fix import ProxyFix
app.wsgi_app = ProxyFix(app.wsgi_app, x_proto=1, x_host=1)
What happens now is that 127.0.0.1 gets translated to yourdomain.tld when delivered to your Flask app. So when you're redirecting back using redirect_to, you're redirecting to your domain externally, no longer relatively. So in the case above, /api2 is redirecting to myDomain.tld/api2/ then /api2/ is called, which is functional.
You can also skip the preserve host and manually put in your domain name in the redirect as so:
app.add_url_rule("/api2", view_func=StubFunction(), redirect_to="https://yourDomain.tld/api2/")
But I don't like this approach in case you change your domain for some reason.
tl;dr, don't put a trailing slash in your ProxyPass Applications.

Get the host scheme and domain using Flask

I have a certain use case where a Flask app is not configured with SERVER_NAME and it resides in a subpath (say, http://example.com/subpath/subpath2), and, from within a request context, I need to extract just the http://example.com.
I noticed that, without SERVER_NAME being set in the Flask config, url_for with _external=True is still able to retrieve the full URL including the domain on which the Flask app is being served. So I figured there must be a way that Flask (or Werkzeug) retrieves the domain. I dug around inside the url_for function and what resulted was this function which I made to get the domain from within a request context.
from flask.globals import _request_ctx_stack
def get_domain():
fragments = [
_request_ctx_stack.top.url_adapter.url_scheme,
_request_ctx_stack.top.url_adapter.get_host(''),
]
return '://'.join(fragments)
I realise I'm accessing private members, but there is no other obvious way to do it that I could find.
Is this the correct way to do it?

How to detect which of the two virtual hosts is being used in python and flask

I have a website developed in flask running on an apache2 server that responds on port 80 to two URLs
Url-1 http://www.example.com
Url-2 http://oer.example.com
I want to detect which of the two urls the user is coming in from and adjust what the server does and store the variable in a config variable
app.config['SITE'] = 'OER'
or
app.config['SITE'] = 'WWW'
Looking around on the internet I can find lots of examples using urllib2 the issue is that you need to pass it the url you want to slice and I cant find a way to pull that out as it may change between the two with each request.
I could fork the code and put up two different versions but that's as ugly as a box of frogs.
Thoughts welcome.
Use the Flask request object (from flask import request) and one of the following in your request handler:
hostname = request.environ.get('HTTP_HOST', '')
url = urlparse(request.url)
hostname = url.netloc
This will get e.g. oer.example.com or www.example.com. If there is a port number that will be included too. Keep in mind that this ultimately comes from the client request so "bad" requests might have it set wrong, although hopefully apache wouldn't route those to your app.

How to get absolute url in Pylons?

How to get absolute url in Pylons ?
To generate a fully qualified URL with Routes, use qualified=True keyword in url() call.
Example:
print url("blog", id=123, qualified=True)
# depending on routing configuration,
# would print something like "http://somehost/blog/123"
If your web application is running behind load balancer or reverse proxy, you might get into issues where generated URLs point to backend appservers not the frontend proxy / load balancer. You can use host argument to correct for that:
print url("blog", id=123, qualified=True, host="example.com")
# ==> "http://example.com/blog/123"
Refer to Routes manual for more options and tweaks.

Categories