I want to obscure a password in a URL for logging purposes. I was hoping to use urlparse, by parsing, replacing password with dummy password, and unparsing, but this is giving me:
>>> from urllib.parse import urlparse
>>> parts = urlparse('https://user:pass#66.66.66.66/aaa/bbb')
>>> parts.password = 'xxx'
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
AttributeError: can't set attribute
So the alternative seems to be this, which seems overkill.
Is there an easier way of replacing the password, using the standard library?
urlparse returns a (subclass of) named tuple. Use the namedtuple._replace() method to produce a new copy, and use geturl() on that to 'unparse'.
The password is part of the netloc attribute, which can be parsed further:
from urllib.parse import urlparse
def replace_password(url):
parts = urlparse(url)
if parts.password is not None:
# split out the host portion manually. We could use
# parts.hostname and parts.port, but then you'd have to check
# if either part is None. The hostname would also be lowercased.
host_info = parts.netloc.rpartition('#')[-1]
parts = parts._replace(netloc='{}:xxx#{}'.format(
parts.username, host_info))
url = parts.geturl()
return url
Demo:
>>> replace_password('https://user:pass#66.66.66.66/aaa/bbb')
'https://user:xxx#66.66.66.66/aaa/bbb'
Related
I'm writing a Python script that logs into a server and pulls router data through an api. The pulled data is then used to create a dictionary with router names as the key and a telnet url as the key. Here is an example of the url that's collected.
telnet://192.168.1.113:32769
The telnet port is the last 5 characters of the url and I'm trying to pull that information only. I know with a string I can use (-5) but I'm getting the following error.
Traceback (most recent call last):
File "C:\Users\b\Documents\Atom Test1 Project\test_wip.py", line 41, in <module>
test_value2=test_value.split(-5)
TypeError: must be str or None, not int
[Finished in 1.812s]
I think this means I need to convert it tonto a string. I tried converting and then retrieving the last 5 charcters but it's not working. Here is my code.
from __future__ import unicode_literals, print_function
import eve
import json
import time
from netmiko import ConnectHandler, redispatch
#from resteve import eve
import json
address = '192.168.1.113'
instance = m11.Server(address)
instance.login('admin', 'password', '0')
users = instance.get_all_nodes()
payload = json.loads(users.content)
data = payload['data']
users = instance.get_all_nodes()
payload = json.loads(users.content)
data = payload['data']
for item in payload["data"].values():
result[item["name"]] = item["url"]
test_value=item['url']
print(test_value)
test_value.format(str)
test_value2=test_value.split(-5)
print(test_value2)
I'm new at this and still putting it all together so any help is greatly appreciated. Thanks.
To get last 5 chars use indexing test_value[-5:], because .split() expects a string and here it will try to split on the first argument
I am trying to encode a string to url to search google scholar, soon to realize, urlencode is not provided in urllib3.
>>> import urllib3
>>> string = "https://scholar.google.com/scholar?" + urllib3.urlencode( {"q":"rudra banerjee"} )
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
AttributeError: 'module' object has no attribute 'urlencode'
So, I checked urllib3 doc and found, I possibly need request_encode_url. But I have no experience in using that and failed.
>>> string = "https://scholar.google.com/scholar?" +"rudra banerjee"
>>> url = urllib3.request_encode_url('POST',string)
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
AttributeError: 'module' object has no attribute 'request_encode_url'
So, how I can encode a string to url?
NB I don't have any particular fascination to urllib3. so, any other module will also do.
To simply encode fields in a URL, you can use urllib.urlencode.
In Python 2, this should do the trick:
import urllib
s = "https://scholar.google.com/scholar?" + urllib.urlencode({"q":"rudra banerjee"})
print(s)
# Prints: https://scholar.google.com/scholar?q=rudra+banerjee
In Python 3, it lives under urllib.parse.urlencode instead.
(Edit: I assumed you wanted to download the URL, not simply encode it. My mistake. I'll leave this answer as a reference for others, but see the other answer for encoding a URL.)
If you pass a dictionary into fields, urllib3 will take care of encoding it for you. First, you'll need to instantiate a pool for your connections. Here's a full example:
import urllib3
http = urllib3.PoolManager()
r = http.request('POST', 'https://scholar.google.com/scholar', fields={"q":"rudra banerjee"})
print(r.data)
Calling .request(...) will take care of figuring out the encoding for you based on the method.
Getting started examples are here: https://urllib3.readthedocs.org/en/latest/index.html#usage
I'm beginning to use CGI with Python.
After running the following piece of code:
#!c:\python34\python.exe
import cgi
print("Content-type: text/html\n\n") #important
def getData():
formData = cgi.FieldStorage()
InputUN = formData.getvalue('username')
InputPC = formData.getvalue('passcode')
TF = open("TempFile.txt", "w")
TF.write(InputUN)
TF.write(InputPC)
TF.close()
if __name__ =="__main__":
LoginInput = getData()
print("cgi worked")
The following error occurs:
Traceback (most recent call last):
File "C:\xampp\htdocs\actual\loginvalues.cgi", line 21, in <module>
LoginInput = getData()
File "C:\xampp\htdocs\actual\loginvalues.cgi", line 16, in getData
TF.write(InputUN)
TypeError: must be str, not None
>>>
I'm trying to write the values, inputted in html, to a text file.
Any help would be appreciated :)
Your calls to getValue() are returning None, meaning the form either didn't contain them, had them set to an empty string, or had them set by name only. Python's CGI module ignores inputs that aren't set to a non-null string.
Works for Python CGI:
mysite.com/loginvalues.cgi?username=myname&pass=mypass
Doesn't work for Python CGI:
mysite.com/loginvalues.cgi?username=&pass= (null value(s))
mysite.com/loginvalues.cgi?username&pass (Python requires the = part.)
To account for this, introduce a default value for when a form element is missing, or handle the None case manually:
TF.write('anonymous' if InputUN is None else InputUN)
TF.write('password' if InputPC is None else InputUN)
As a note, passwords and other private login credentials should never be used in a URL. URLs are not encrypted. Even in HTTPS, the URL is sent in plain text that anyone on the network(s) between you and your users can read.
The only time a URL is ever encrypted is over a tunneled SSH port or an encrypted VPN, but you can't control that, so never bank on it.
I know many people encountered this error before but I couldn't find the solution to my problem.
I have a URL that I want to normalize:
url = u"http://www.dgzfp.de/Dienste/Fachbeitr%C3%A4ge.aspx?EntryId=267&Page=5"
scheme, host_port, path, query, fragment = urlsplit(url)
path = urllib.unquote(path)
path = urllib.quote(path,safe="%/")
This gives an error message:
/usr/lib64/python2.6/urllib.py:1236: UnicodeWarning: Unicode equal comparison failed to convert both arguments to Unicode - interpreting them as being unequal
res = map(safe_map.__getitem__, s)
Traceback (most recent call last):
File "url_normalization.py", line 246, in <module>
logging.info(get_canonical_url(url))
File "url_normalization.py", line 102, in get_canonical_url
path = urllib.quote(path,safe="%/")
File "/usr/lib64/python2.6/urllib.py", line 1236, in quote
res = map(safe_map.__getitem__, s)
KeyError: u'\xc3'
I tried to remove the unicode indicator "u" from the URL string and I do not get the error message. But How can I get rid of the unicode automatically because I read it directly from a database.
urllib.quote() does not properly parse Unicode. To get around this, you can call the .encode() method on the url when reading it (or on the variable you read from the database). So run url = url.encode('utf-8'). With this you get:
import urllib
import urlparse
from urlparse import urlsplit
url = u"http://www.dgzfp.de/Dienste/Fachbeitr%C3%A4ge.aspx?EntryId=267&Page=5"
url = url.encode('utf-8')
scheme, host_port, path, query, fragment = urlsplit(url)
path = urllib.unquote(path)
path = urllib.quote(path,safe="%/")
and then your output for the path variable will be:
>>> path
'/Dienste/Fachbeitr%C3%A4ge.aspx'
Does this work?
I want to open a connection to a ldap directory using ldap url that will be given at run time. For example :
ldap://192.168.2.151/dc=directory,dc=example,dc=com
It is valid as far as I can tell. Python-ldap url parser ldapurl.LDAPUrl accepts it.
url = 'ldap://192.168.2.151/dc=directory,dc=example,dc=com'
parsed_url = ldapurl.LDAPUrl(url)
parsed_url.dn
'dc=directory,dc=example,dc=com'
But if I use it to initialize a LDAPObject, I get a ldap.LDAPError exception
ldap.initialize(url)
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
File "/usr/lib/python2.7/dist-packages/ldap/functions.py", line 91, in initialize
return LDAPObject(uri,trace_level,trace_file,trace_stack_limit)
File "/usr/lib/python2.7/dist-packages/ldap/ldapobject.py", line 70, in __init__
self._l = ldap.functions._ldap_function_call(ldap._ldap_module_lock,_ldap.initialize,uri)
File "/usr/lib/python2.7/dist-packages/ldap/functions.py", line 63, in _ldap_function_call
result = func(*args,**kwargs)
ldap.LDAPError: (0, 'Error')
I found that if I manually encode the dn part of the url, it works :
url = 'ldap://192.168.2.151/dc=directory%2cdc=example%2cdc=com'
#url still valid
parsed_url = ldapurl.LDAPUrl(url)
parsed_url.dn
'dc=directory,dc=example,dc=com'
#and will return a valid connection
ldap.initialize(url)
<ldap.ldapobject.SimpleLDAPObject instance at 0x1400098>
How can I ensure robust url handling in ldap.initialize without encoding parts of the url myself ? (which, I'm afraid, won't be that robust anyway).
You can programatically encode the last part of the URL:
from urllib import quote # works in Python 2.x
from urllib.parse import quote # works in Python 3.x
url = 'ldap://192.168.2.151/dc=directory,dc=paralint,dc=com'
idx = url.rindex('/') + 1
url[:idx] + quote(url[idx:], '=')
=> 'ldap://192.168.2.151/dc=directory%2Cdc=paralint%2Cdc=com'
One can use LDAPUrl.unparse() method to get a properly encoded version of the URI, like this :
>>> import ldapurl
>>> url = ldapurl.LDAPUrl('ldap://192.168.2.151/dc=directory,dc=example,dc=com')
>>> url.unparse()
'ldap://192.168.2.151/dc%3Ddirectory%2Cdc%3Dparalint%2Cdc%3Dcom???'
>>> ldap.initialize(url.unparse())
<ldap.ldapobject.SimpleLDAPObject instance at 0x103d998>
And LDAPUrl.unparse() will not reencode an already encoded url :
>>> url = ldapurl.LDAPUrl('ldap://example.com/dc%3Dusers%2Cdc%3Dexample%2Cdc%3Dcom%2F???')
>>> url.unparse()
'ldap://example.com/dc%3Dusers%2Cdc%3Dexample%2Cdc%3Dcom%2F???'
So you can use it blindly on any ldap uri your program must handle.