We Keep Coding

Issue with URL encoding

I have a query string that will return data successfully if I run it in Postman but if I pull the data via Python3 (http.client.HTTPSConnection), the target returns 'bad request'.
The problem:
/v4_6_release/apis/3.0/service/tickets?customFieldConditions=caption="Escalated To" AND value ="Cloud Operations"
I have tried putting +AND+ or %20and%20 or %20AND%20 instead of AND but no success. For example:
/v4_6_release/apis/3.0/service/tickets?customFieldConditions=caption="Escalated To"%20AND%20value="Cloud Operations"
^ This works fine in postman but returns <hr><p>HTTP Error 400. The request is badly formed.</p> when via Python3.
Is there anything else wrong with the url encoding here?

You have to encode all whitespaces in you URL. So the URL should be /v4_6_release/apis/3.0/service/tickets?customFieldConditions=caption="Escalated%20To"%20AND%20value="Cloud%20Operations"

cannot access URL due to SSL module not available Python

First time trying to use Python 3.6 requests library's get() function with data from quandl.com and load and dump json.
import json
import requests
request = requests.get("https://www.quandl.com/api/v3/datasets/CHRIS/MX_CGZ2.json?api_key=api_keyxxxxx", verify=False)
request_text=request.text()
data = json.loads(request_text)
data_serialized = json.dumps(data)
print(data_serialized)
I have an account at quandl.com to access the data. The error when python program is run in cmd line says "cannot connect to HTTPS URL because SSL mode not available."

import requests
import urllib3
urllib3.disable_warnings()
r = requests.get(
"https://www.quandl.com/api/v3/datasets/CHRIS/MX_CGZ2.json?api_key=api_keyxxxxx").json()
print(r)
Output will be the following since i don't have API Key
{'quandl_error': {'code': 'QEAx01', 'message': 'We could not recognize your API key. Please check your API key and try again.'}}
You don't need to import json module as requests already have it.

Although I have verified 'quandl_api_keys received the following error when trying to retrieve data with 'print' data 'json.dumps' function: "quandl_error": {"code": "QEAx01" ... discovered that the incorrect fonts on the quotation marks around key code in the .env file resulted in this error. Check language settings and fonts before making requests after the error mssg.

BrokenFilesystemWarning in Werkzeug [duplicate]

I have the following, very basic code that throws; TypeError: the JSON object must be str, not 'bytes'
import requests
import json
url = 'my url'
user = 'my user'
pwd = 'my password'
response = requests.get(url, auth=(user, pwd))
if(myResponse.ok):
Data = json.loads(myResponse.content)
I try to set decode to the Data variable, as follows but it throws the same error; jData = json.loads(myResponse.content).decode('utf-8')
Any suggestions?

json.loads(myResponse.content.decode('utf-8'))
You just put it in the wrong order, innocent mistake.
(In-depth answer). As courteously pointed out by wim, in some rare cases, they could opt for UTF-16 or UTF-32. These cases will be less common as the developers, in that scenario would be consciously deciding to throw away valuable bandwidth. So, if you run into encoding issues, you can change utf-8 to 16, 32, etc.
There are a couple of solutions for this. You could use request's built-in .json() function:
myResponse.json()
Or, you could opt for character detection via chardet. Chardet is a library developed based on a study. The library has one function: detect. Detect can detect most common encodings and then use them to encode your string with.
import chardet
json.loads(myResponse.content.decode(chardet.detect(myResponse.content)["encoding"]))

Let requests decode it for you:
data = response.json()
This will check headers (Content-Type) and response encoding, auto-detecting how to decode the content correctly.

python3.6+ does this automatically.so your code shouldn't return error in python3.6+
what's new in python3.6

making a simple GET/POST with url Encoding python

i have a custom url of the form
http://somekey:somemorekey#host.com/getthisfile.json
i tried all the way but getting errors :
method 1 :
from httplib2 import Http
ipdb> from urllib import urlencode
h=Http()
ipdb> resp, content = h.request("3b8138fedf8:1d697a75c7e50#abc.myshopify.com/admin/shop.json")
error :
No help on =Http()
Got this method from here
method 2 :
import urllib
urllib.urlopen(url).read()
Error :
*** IOError: [Errno url error] unknown url type: '3b8108519e5378'
I guess something wrong with the encoding ..
i tried ...
ipdb> url.encode('idna')
*** UnicodeError: label empty or too long
Is there any way to make this Complex url get call easy .

You are using a PDB-based debugger instead of a interactive Python prompt. h is a command in PDB. Use ! to prevent PDB from trying to interpret the line as a command:
!h = Http()
urllib requires that you pass it a fully qualified URL; your URL is lacking a scheme:
urllib.urlopen('http://' + url).read()
Your URL does not appear to use any international characters in the domain name, so you do not need to use IDNA encoding.
You may want to look into the 3rd-party requests library; it makes interacting with HTTP servers that much easier and straightforward:
import requests
r = requests.get('http://abc.myshopify.com/admin/shop.json', auth=("3b8138fedf8", "1d697a75c7e50"))
data = r.json() # interpret the response as JSON data.

The current de facto HTTP library for Python is Requests.
import requests
response = requests.get(
"http://abc.myshopify.com/admin/shop.json",
auth=("3b8138fedf8", "1d697a75c7e50")
)
response.raise_for_status() # Raise an exception if HTTP error occurs
print response.content # Do something with the content.

Decode Cookie variable extracted from a HTTP Stream - Python

I am using python to send a request to a server. I get a cookie from the server. I am trying to decode the encoding scheme used by the server - I suspect it's either utf-8 or base64.
So I create my header and connection objects.
resp, content = httpobj.request(server, 'POST', headers=HTTPheader, body=HTTPbody)
And then i extract the cookie from the HTTP Stream
cookie= resp['set-cookie']
I have tried str.decode() and unicode() but I am unable to get the unpacked content of the cookie.
Assume the cookie is
MjAyMTNiZWE4ZmYxYTMwOVPJ7Jh0B%2BMUcE4si5oDcH7nKo4kAI8CMYgKqn6yXpgtXOSGs8J9gm20bgSlYMUJC5rmiQ1Ch5nUUlQEQNmrsy5LDgAuuidQaZJE5z%2BFqAJPnlJaAqG2Fvvk5ishG%2FsH%2FA%3D%3D
The output I am expecting is
20213bea8ff1a309SÉì˜tLQÁ8².hÁûœª8<Æ
*©úÉzµs’Ïö¶Ñ¸•ƒ$.kš$5gQIPf®Ì¹,8�ºèA¦IœöZ€$ùå% *ao¾Nb²¶ÁöÃ

Try like this:
import urllib
import base64
cookie_val = """MjAyMTNiZWE4ZmYxYTMwOVPJ7Jh0B%2BMUcE4si5oDcH7nKo4kAI8CMYgKqn6yXpgtXOSGs8J9gm20bgSlYMUJC5rmiQ1Ch5nUUlQEQNmrsy5LDgAuuidQaZJE5z%2BFqAJPnlJaAqG2Fvvk5ishG%2FsH%2FA%3D%3D"""
res = base64.b64decode(urllib.unquote(cookie_val))
print repr(res)
Output:
"20213bea8ff1a309S\xc9\xec\x98t\x07\xe3\x14pN,\x8b\x9a\x03p~\xe7*\x8e$\x00\x8f\x021\x88\n\xaa~\xb2^\x98-\\\xe4\x86\xb3\xc2}\x82m\xb4n\x04\xa5`\xc5\t\x0b\x9a\xe6\x89\rB\x87\x99\xd4RT\x04#\xd9\xab\xb3.K\x0e\x00.\xba'Pi\x92D\xe7?\x85\xa8\x02O\x9eRZ\x02\xa1\xb6\x16\xfb\xe4\xe6+!\x1b\xfb\x07\xfc"
Of course the result here is a 8-bit string, so you have to decode it to get the the string that you want, i'm not sure which encoding to use, but there is the decoding result using the unicode-escape (unicode literal) :
>>> print unicode(res, 'unicode-escape')
20213bea8ff1a309SÉìtãpN,p~ç*$1ª~²^-\ä³Â}m´n¥`ÅBÔRT#Ù«³.K.º'PiDç?¨ORZ¡¶ûäæ+!ûü
Well Hope this can help .