Python Request in unicode [duplicate] - python

This question already has answers here:
urllib.quote() throws KeyError
(3 answers)
Closed 7 years ago.
I am trying to make a program that requests to steam to get a the cheapest price for an item. For this I will be using StatTrak™ P250 | Supernova (Factory New) as an example.
The problem is that when requesting, you will make a url:
http://www.steamcommunity.com/market/priceoverview/?country=SG&currency=13&appid=730&market_hash_name=StatTrak™%20P250%20%7C%20Supernova%20%28Factory%20New%29
Afterwards, (I am using the requests module) I do this:
url = "http://www.steamcommunity.com/market/priceoverview/?country=SG&currency=13&appid=730&market_hash_name=StatTrak™%20P250%20%7C%20Supernova%20%28Factory%20New%29"
requests.get(url)
However, the server will return an error.
I can't seem to find solutions to replace ™. I have tried %2122. In python I tried using u'\u084a' but that didn't work too. The problem is that python sends literally \u084a in the request. Is there any way to solve this?


Just use URL encoding. You can't use unicode in urls.
>>> import urllib
>>> f = {'market_hash_name': 'StatTrak™'}
>>> urllib.urlencode(f)
'market_hash_name=StatTrak%E2%84%A2'
Also possible
>>> urllib.quote_plus('StatTrak™')

Related

Hex to plain ASCII? Python 2.7.13 [duplicate]

This question already has answers here:
Decode escaped characters in URL
(5 answers)
Closed 2 years ago.
Im trying to turn this:
%73%6c%61%70%72%69%73%65%40%6c%69%65%6e%6d%75%6c%74%69%6d%65%64%69%61%2e%63%6f%6d
into this:
slaprise#lienmultimedia.com
and my brain is exploding..
Any help would be appreciated.
Thank you

Python 2.7.17 (should work for Python 2.7.13)
import urllib2
url = urllib2.unquote("%73%6c%61%70%72%69%73%65%40%6c%69%65%6e%6d%75%6c%74%69%6d%65%64%69%61%2e%63%6f%6d")
print(url)
# slaprise#lienmultimedia.com

You are trying to URL-decode that string, use urllib:
from urllib import unquote
url = unquote("%73%6c%61%70%72%69%73%65%40%6c%69%65%6e%6d%75%6c%74%69%6d%65%64%69%61%2e%63%6f%6d")
# url = slaprise#lienmultimedia.com

regex getting url link [duplicate]

This question already has answers here:
Why can't Python parse this JSON data? [closed]
(3 answers)
Closed 5 years ago.
I am trying to extract the link I am getting from a curl command. Curl command throws back of type string.
{"success":true,"key":"Syv77d","link":"https://file.io/Syv77d","expiry":"14 days"}
In my below code this gets https://file.io/Syv77d","expiry":"14 days"}
link = re.search('https://.*$',fileIO)
What I wanted was just https://file.io/Syv77d
The link would vary so i would need the url without the double-qoutes. I think I am missing something in my regex.

Convert the string object to a JSON object.
Ex:
import json
jData = json.loads('{"success":true,"key":"Syv77d","link":"https://file.io/Syv77d","expiry":"14 days"}')
jData["link"]

Retrieving part of URL in python [duplicate]

This question already has answers here:
Python: Get URL path sections
(7 answers)
Closed 5 years ago.
I'm on Python 3.6.
How I can extract part of URL so I can use it as variable.
For example, the URL is http://example.com/comp/project.sec and I would like to get the project part without .sec as variable.
How can I achieve that?

url = "http://example.com/comp/project.sec"
project = url.split("/")[-1].split(".")[0]

Here's an alternative, which removes split requirement.
from os.path import splitext, basename
splitext(basename('http://example.com/comp/project.sec'))[0]

import urllib
urllib.parse.urlparse("http://example.com/comp/project.sec").path.split("/")[-1].split(".")[-1]

urllib.unquote not properly decoding url [duplicate]

This question already has an answer here:
URLDecoding requests
(1 answer)
Closed 7 years ago.
I am able to do the following in the python shell:
>>> import urllib
>>> s='https://www.microsoft.com/de-at/store/movies/american-pie-pr%C3%A4sentiert-nackte-tatsachen/8d6kgwzl63ql'
>>> print urllib.unquote(s)
https://www.microsoft.com/de-at/store/movies/american-pie-präsentiert-nackte-tatsachen/8d6kgwzl63ql
However, if I do this within a python program, it improperly decodes the url:
url = res.history[0].url if res.history else res.url
print '1111', url
print '2222', urllib.unquote(url)
111 https://www.microsoft.com/de-at/store/movies/american-pie-pr%C3%A4sentiert-nackte-tatsachen/8d6kgwzl63ql
222 https://www.microsoft.com/de-at/store/movies/american-pie-präsentiert-nackte-tatsachen/8d6kgwzl63ql
Why isn't this being properly decoded in the program but it is in my python shell?

The following worked to fix the issue:
url = urllib.unquote(str(res.url)).decode('utf-8', 'ignore')
res.url was a unicode string, but didn't seem to work well with urllib.unquote. So the solution was to first convert it to a string (like how it was in the python interpreter) and then decode it into Unicode.

Not receiving the parameters containing '?' properly. The part after '?' is getting trimmed. How do I avoid it? [duplicate]

This question already has answers here:
How to get URL of current page, including parameters, in a template?
(7 answers)
Closed 7 years ago.
I am writing a Django application. In my 'urls.py' I have written a URL pattern like this:
url(r'^rest/post/(.*)/$', rest_post),
Now when I am passing some URL like:
http://www.google.com/a?b
In my rest_post view I am getting only: http://www.google.com/a
I want to get the full URL. How do I do it?

simply
request.get_full_path()
in your views.
see https://docs.djangoproject.com/en/dev/ref/request-response/#django.http.HttpRequest.get_full_path

Categories