We Keep Coding

django.urls.reverse(): URL-encoding the slash

When one supplies URL args or kwargs to a django.urls.reverse() call,
Django will nicely URL-encode non-Ascii characters and URL-reserved characters.
For instance, given a declaration such as
path("prefix/<stuff>", view=MyView.as_view(), name="myurl")
we get
reverse('myurl', args=['aaa bbb']) == "/prefix/aaa%20bbb"
reverse('myurl', args=['aaa%bbb']) == "/prefix/aaa%25bbb"
reverse('myurl', args=['Ä']) == "/prefix/%C3%84"
and so on. So far, so good.
What Django will not encode, however, is the slash:
reverse('myurl', args=['aaa/bbb'])
will give us
django.urls.exceptions.NoReverseMatch:
Reverse for 'myurl' with arguments '('aaa/bbb',)' not found.
1 pattern(s) tried: ['prefix/(?P<stuff>[^/]+)$']
(The question what to encode and what not
has been discussed as a Django issue.
It's complicated.)
I found a remark in the code that may explain why
the slash is a special case:
_reverse_with_prefix in django/urls/resolvers.py contains a comment that says
# WSGI provides decoded URLs, without %xx escapes, and the URL
# resolver operates on such URLs. First substitute arguments
# without quoting to build a decoded URL and look for a match.
# Then, if we have a match, redo the substitution with quoted
# arguments in order to return a properly encoded URL.
Given that unencoded arguments are used in the matching initially,
it is no wonder that it does not work:
The slash looks like the end of the argument to Django and so there is
one more argument than expected.
My question:
I dearly want to use user-supplied data in natural-looking URLs,
so slashes occur occasionally. How can I make them work?
The URL structure I need is basically
/show_rooms/<organization>/<department>/<building>
I can think of these approaches:
Replace a slash in an argument with some exotic Unicode character
that will never occur otherwise. And back for received arguments.
This would sort of do the job, but is inconvenient,
non-standard, and therefore ugly.
Use slugs instead of the real names.
This would require extending my models to store the slugs (because
the ORM needs to find objects by them) and appears out of proportion to me.
URL-quote my arguments before passing them to reverse()
and unquote arguments when I receive them.
This is as inconvenient as (1).
It leads to URLs that are more difficult to read than
those from (1), because each % produced by quoting
will subsequently be encoded as %25.
But at least it is a standard-ish approach.
Sigh. Is this really the "right" way?
Any comments or fourth solutions are welcome!
Now that I've written it up, solution (1) does not look quite
so horrible to me. What replacement character would you use for a slash?

I suggest you try to pass the slash in as a regular URL and see if your view is able to match with it. If that's the case and the problem is in the reverse function itself not the view. How about passing the slash already encoded %2F?

It's me, the asker.
Here is the solution that I finally used:
I decided for solution (1).
It turned out less inconvenient than I had expected and
it works spectacularly well.
My Firefox browser shows the URL in text form, not urlencoded form,
and when you pick the right replacement character it looks almost natural.
Very nice.
Here is the code for the escaping (to be called in the template, hence a
custom templatetag)
and unescaping (to be called in the view):
import django.template as djt
register = djt.Library()
# see https://docs.djangoproject.com/en/stable/howto/custom-template-tags/
ALT_SLASH = '\N{DIVISION SLASH}'
#register.filter
def escape_slash(urlparam: str) -> str:
"""
Avoid having a slash in the urlparam URL part,
because it would not get URL-encoded.
See https://stackoverflow.com/questions/67849991/django-urls-reverse-url-encoding-the-slash
Possible replacement characters are
codepoint char utf8 name oldname
U+2044 ⁄ e2 81 84 FRACTION SLASH
U+2215 ∕ e2 88 95 DIVISION SLASH
U+FF0F ／ ef bc 8f FULLWIDTH SOLIDUS FULLWIDTH SLASH
None of them will look quite right if the browser shows the char rather than
the %-escape in the address line, but DIVISION SLASH comes close.
The normal slash is
U+002F / 2f SOLIDUS SLASH
To get back the urlparam after calling escape_slash,
the URL will be formed (via {% url ... } or reverse()) and URL-encoded,
sent to the browser,
received by Django in a request, URL-unencoded, split,
its param parts handed to a view as args or kwargs,
and finally unescape_slash will be called by the view.
"""
return urlparam.replace('/', ALT_SLASH)
def unescape_slash(urlparam_q: str) -> str:
return urlparam_q.replace(ALT_SLASH, '/')

Stylesheet does not work with special characters pyqt

Could not parse stylesheet of object 0x2d956381ba0
def change_btn_image(self, btn):
imagem = self.atv_imagens_bd[self.atividades.get_contador()]
nome_imagem = imagem[:-4]
print(nome_imagem)
btn.setStyleSheet(f"border-image: url(src/main/resources/base/{imagem});")
return nome_imagem

As explained in a slightly related answer, HTML (and, therefore, stylesheet) parsing capabilities of Qt are a bit limited if compared to web browser parsers.
While those syntax are somehow pretty "liberal", one shouldn't push the boundaries of their freedom too much: it's always good practice to use quotes for string based values (besides keywords, obviously), most importantly for URLs.
In this case, the problem is the utf character, which creates a problem as the parser is unable to correctly identify where the url ends.
Just add quotes around the url path:
btn.setStyleSheet(f"border-image: url('src/main/resources/base/{imagem}');")

How can I escape certain characters while using python's urllib.urlencode()?

I have a dictionary that I want to urlencode as query parameters.
The server that I am hitting expects the the query to look like this: http://www.example.com?A=B,C
But when I try to use urllib.urlencode to build the URL, I find that the comma gets turned into %2C:
>>> import urllib
>>> urllib.urlencode({"A":"B,C"})
'A=B%2CC'
Is there any way I can escape the comma so that urlencode treats it like a normal character?
If not, how can I work around this problem?

You can escape certain characters by specifying them explicitly as safe argument value
urllib.quote(str, safe='~()*!.\'')
More : https://docs.python.org/3.0/library/urllib.parse.html#urllib.parse.quote

You can do this by adding the query params as a string before hitting the endpoint.
I have used requests for making a request.
For example:
GET Request
import requests
url = "https://www.example.com/?"
query = "A=B,C"
url_final = url + query
url = requests.get(url_final)
print(url.url)
# https://www.example.com/?A=B,C
The comma (along with some other characters) is defined in RFC 3986 as a reserved character. This means the comma has defined meaning at various parts in a URL, and if it is not being used in that context it needs to be percent-encoded.
That said, the query parameter doesn't give the comma any special syntax, so in query parameters, we probably shouldn't be encoding it. That said, it's not entirely Requests' fault: the parameters are encoded using urllib.urlencode(), which is what is percent-encoding the query parameters.
This isn't easy to fix though, because some web services use , and some use %2C, and neither is wrong. You might just have to handle this encoding yourself.

flask url parameters with spaces generates urls with spaces

I'm trying to pass a user supplied string as a Flask URL parameter.
url_for(func_name, param="string with spaces") or similar generates a URL with spaces.
If the user enter a string with spaces the generated url has spaces it seems to work.
Also if I enter a URL with %20 it seems to redirect to a url with spaces.
I thought URLs with spaces were a bad idea.
How do I get it to work right (url_for and redirection)?
Or should I just accept it?
P.S.
Is passing a user supplied string as a parameter safe? If not how should I sanitize the user input string?

No, Flask generates URLs properly URL encoded; demoing with an existing application:
>>> with app.test_request_context('/'):
... print url_for('core.city', city='new york')
...
/new%20york
Your browser on the other hand may elect to show such URLs decoded for ease of reading.
url_for() quotes input to be URL-safe; encoded URLs cannot contain values that could be interpreted as HTML, so you are safe there as far as user-supplied values are concerned.

Flask route with URI encoded component

It seems Flask doesn't support routes with a URI encoded component. I'm curious if I'm doing something wrong, or if there is a special flag I need to include.
My route looks something like this:
#app.route('/foo/<encoded>/bar/')
def foo(encoded):
# ...
pass
The URL that this should match can look like these:
http://foobar.com/foo/xxx/bar/ # matched correctly, no URI component
http://foobar.com/foo/x%2Fx%2Fx%2F/bar/ # not matched correctly, URI component
Former URL works, latter spits out a lovely 404.
Thanks!

Add path to your url rule:
#app.route('/foo/<path:encoded>/bar/')
Update per comment: The route API docs are here: http://flask.pocoo.org/docs/api/#flask.Flask.route. The underlying classes that implement the path style route converter are here: http://werkzeug.pocoo.org/docs/routing/#custom-converters (this is one of the really nice parts of pocoostan.) As far as the trailing slashes, there are special rules that amount to:
If a rule ends with a slash and is requested without a slash by the
user, the user is automatically redirected to the same page with a
trailing slash attached.
If a rule does not end with a trailing slash and the user request the
page with a trailing slash, a 404 not found is raised.
Also keep in mind that if you are on Apache and are expecting a slash-trailed url, ie a bookmarklet that submits to http://ex.com/foo/<path:encoded>/bar and encoded gets something with double slashes, Apache will convert multiple slashes to a single one.