I am new to Python. I am trying to convert Python 2 code to Python 3. In my old code I have the following lines:
# Create a cookiejar to store cookie
cj = cookielib.CookieJar()
# Create opener
opener = urllib2.build_opener(urllib2.HTTPCookieProcessor(cj))
opener.addheaders = [('User-agent', 'Mozilla/5.0')]
I have converted these lines to:
# Create a cookiejar to store cookie
cj = cookielib.CookieJar()
# Create opener
opener = urllib.request.build_opener(urllib.request.HTTPCookieProcessor(cj))
opener.addheaders = [('User-agent', 'Mozilla/5.0')]
The issue I have is that I keep getting the following error:
NameError: global name 'cookielib' is not defined
I am not sure what I am doing wrong and how to fix this. Can someone please help me? Thank you very much.
Did you use the 2to3 tool? Also, using Python Docs,
Note The cookielib module has been renamed to http.cookiejar in Python
3. The 2to3 tool will automatically adapt imports when converting your sources to Python 3.
After seeing your comment, the problem is that it'shttp.cookiejar and not http.cookieJar.
Notice the un-captitalised J.
I think I have the solution. The following seems to work:
cj = http.cookiejar.CookieJar()
The library cookielib module has been renamed to http.cookiejar in Python 3, see https://docs.python.org/2/library/cookielib.html.
You may use the the 2to3 tool to convert your source code to Python 3, see https://docs.python.org/2/library/2to3.html
Related
I want to get the value of the 'latest' version tag from here: https://papermc.io/repo/repository/maven-public/com/destroystokyo/paper/paper-api/maven-metadata.xml
I tried using this python:
import urllib.request
from xml.etree import ElementTree
opener = urllib.request.build_opener()
opener.addheaders = [('User-agent', 'Mozilla/5.0')]
data = opener.open('https://papermc.io/repo/repository/maven-public/com/destroystokyo/paper/paper-api/maven-metadata.xml').
root = ElementTree.fromstring(data)
versioning = root.find("versioning")
latest = versioning.find("latest")
snip.rv = latest.text
The problem is, using this inside of vim (I'm trying to make UltiSnips snippets with it) makes the whole of vim extremely slow after the code has finished running.
What's causing my program to slow down just when I add that ^^ code?
I don't know if this will solve the performance issue in vim, but the code was not running for me due to errors in it.
opener.open returns a file-like object, so you should read it using
ElementTree.parse instead of ElementTree.fromstring (actually there is a trailing dot after opener.open(...), so I don't know if you missed a read() thereafter. In that case the return value is indeed a string).
Apart from that, you could try to close the opener to see if that frees up some resources (or use the with).
I attach an example of the improved code:
import urllib.request
from xml.etree import ElementTree
opener = urllib.request.build_opener()
opener.addheaders = [('User-agent', 'Mozilla/5.0')]
with opener.open('https://papermc.io/repo/repository/maven-public/com/destroystokyo/paper/paper-api/maven-metadata.xml') as data:
root = ElementTree.parse(data)
latest = root.find("./versioning/latest")
snip.rv = latest.text
I have created an opener with urllib2.build_opener() that contains a cookielib.CookieJar(), and now I wish to manually add a cookie to the opener.
How can I achieve this?
Like the second example of the cookielib documentation suggests:
import os, cookielib, urllib2
cj = cookielib.MozillaCookieJar()
cj.load(os.path.join(os.path.expanduser("~"), ".netscape", "cookies.txt"))
opener = urllib2.build_opener(urllib2.HTTPCookieProcessor(cj))
r = opener.open("http://example.com/")
Here's the link:
Cookies examples
Above example applies to Mozilla cookies, but generic algorithm is the same.
If adding by hand is required, reading the documentation further, you can use:
http://docs.python.org/library/cookie.html#module-Cookie Cookie object, which you fill up the way you see fit and further on add it to a CookieJar with
CookieJar.set_cookie(cookie)
Set a Cookie, without checking with policy to see whether or not it should be set.
I need to use Python to download a large number of URLS, but they require a password to access them (similar to systems like cpanel, for example).
Is there a way I can do this, storing the cookie?
I'd like to use urllib2 if possible.
EDIT: To clarify, it's my website and I have the login details.
UPDATE:
OK I'm using this:
cj = cookielib.CookieJar()
opener = urllib2.build_opener(urllib2.HTTPCookieProcessor(cj))
login_data = urllib.urlencode({'login_name' : username, 'password' : password})
opener.open(loginURL, login_data)
productlist = opener.open(productURL)
print productlist.read()
But it just spits out the login page again. What isn't working?
(Variables are there, I just didn't show you what they are for security)
You have to use the urllib2.HTTPCookieProcessor, like this:
import urllib2
from cookielib import CookieJar
cookiejar = CookieJar()
opener = urllib2.build_opener()
cookieproc = urllib2.HTTPCookieProcessor(cookiejar)
opener.add_handler(cookieproc)
Then you just use opener.open() to access URLs, and cookies will automatically be saved and reused in future requests.
I am using CentOS 5.5 and Python.
The following code works perfectly,
cj = cookielib.CookieJar()
opener = urllib2.build_opener(urllib2.HTTPCookieProcessor(cj))
login_data = urllib.urlencode({'username' : 'myname', 'password' : '123456'})
resp = opener.open('http://www.example.com/signin.html', login_data)
However, I want to save the cookies to /var/www/html/cookies and load it when I need, so I don't have to sign in every time, just like the browsers do.
How can I save and load the cookies? Thanks a lot!
cookielib defines a separate FileCookieJar class that has the functionality you're looking for.
can anyone help me with loop i want loop that code
login_form_data = urllib.urlencode(login_form_seq)
opener = urllib2.build_opener()
site = opener.open(B, login_form_data).read()
the code allow me to login to site but site have problem and the problem is: you can't login from first time
that mean I have to press submit then when page reload press submit again... so i think loop will do that but How!?
You need to handle cookies. Look at the cookielib module.
If it is a cookie handling problem, use the "HTTPCookieProcessor" in urllib2.
By applying it to your opener.
cookieHandler = urllib2.HTTPCookieProcessor() # Needed for cookie handling
# Apply the handler to an opener
opener = urllib2.build_opener(cookieHandler)
It seems that you are not accepting and saving the cookie(s) required by the page you are trying to access. This is not surprising given that urllib2 does not automatically do this for you. As others have said you'll have to explicitly write code to accept cookies. Something like this:
import urllib2, cookielib
cj = cookielib.CookieJar()
opener = urllib2.build_opener(urllib2.HTTPCookieProcessor(cj))
urllib2.install_opener(opener)
login_form_data = urllib.urlencode(login_form_seq)
site = opener.open(B, login_form_data).read()
This would be a good time to read up about cookielib and HTTP state management in Python.