Hi i am newish to python, and working on modeling of the CV-19 outbreak in the UK.
Currently I am writting a program to automatically download the UK governments latest death statistics.
Here is my code so far:
import wget
url = "https://coronavirus.data.gov.uk/downloads/csv/coronavirus-cases_latest.csv"#
wget.download(url, 'C:/Users/Moshe/Downloads/Covid_19_uk_timeseries.csv')
Running this in pycharm I get:
C:\Users\Moshe\PycharmProjects\HelloWorld\venv\Scripts\python.exe C:/Users/Moshe/.PyCharmCE2019.1/config/scratches/Get_CV19_UK_stats.py
Traceback (most recent call last):
File "C:/Users/Moshe/.PyCharmCE2019.1/config/scratches/Get_CV19_UK_stats.py", line 6, in <module>
wget.download(url, 'C:/Users/Moshe/Downloads/Covid_19_uk_timeseries.csv')
File "C:\Users\Moshe\PycharmProjects\HelloWorld\venv\lib\site-packages\wget.py", line 526, in download
(tmpfile, headers) = ulib.urlretrieve(binurl, tmpfile, callback)
File "C:\Users\Moshe\AppData\Local\Programs\Python\Python37\lib\urllib\request.py", line 247, in urlretrieve
with contextlib.closing(urlopen(url, data)) as fp:
File "C:\Users\Moshe\AppData\Local\Programs\Python\Python37\lib\urllib\request.py", line 222, in urlopen
return opener.open(url, data, timeout)
File "C:\Users\Moshe\AppData\Local\Programs\Python\Python37\lib\urllib\request.py", line 531, in open
response = meth(req, response)
File "C:\Users\Moshe\AppData\Local\Programs\Python\Python37\lib\urllib\request.py", line 641, in http_response
'http', request, response, code, msg, hdrs)
File "C:\Users\Moshe\AppData\Local\Programs\Python\Python37\lib\urllib\request.py", line 569, in error
return self._call_chain(*args)
File "C:\Users\Moshe\AppData\Local\Programs\Python\Python37\lib\urllib\request.py", line 503, in _call_chain
result = func(*args)
File "C:\Users\Moshe\AppData\Local\Programs\Python\Python37\lib\urllib\request.py", line 649, in http_error_default
raise HTTPError(req.full_url, code, msg, hdrs, fp)
urllib.error.HTTPError: HTTP Error 308: Permanent Redirect
Process finished with exit code 1
The file I am trying to get can be found on this page:
https://coronavirus.data.gov.uk/?_ga=2.157835066.1251021075.1589887735-1783596499.1585566366
Why won't this work? and what do i need to change?
Can you verify that your output path is valid and writeable?
try
import wget
import os
output_dir = 'C:/Users/Moshe/Downloads'
output_file = 'Covid_19_uk_timeseries.csv'
assert os.path.exists(output_dir)
url = "https://coronavirus.data.gov.uk/downloads/csv/coronavirus-cases_latest.csv"#
wget.download(url, out = os.path.join(output_dir,output_file))
Related
i was just trying to read a file from the internet..
then an error came..
Traceback (most recent call last):
File "E:\Sapphire\Programming\download_file.py", line 46, in <module>
download(arg[1])
File "E:\Sapphire\Programming\download_file.py", line 19, in download
file_info = req.urlopen("http://proget.whirlpool.repl.co/{}/{}.txt".format(arch, name))
File "E:\Programming\Python38-32\lib\urllib\request.py", line 222, in urlopen
return opener.open(url, data, timeout)
File "E:\Programming\Python38-32\lib\urllib\request.py", line 531, in open
response = meth(req, response)
File "E:\Programming\Python38-32\lib\urllib\request.py", line 640, in http_response
response = self.parent.error(
File "E:\Programming\Python38-32\lib\urllib\request.py", line 569, in error
return self._call_chain(*args)
File "E:\Programming\Python38-32\lib\urllib\request.py", line 502, in _call_chain
result = func(*args)
File "E:\Programming\Python38-32\lib\urllib\request.py", line 649, in http_error_default
raise HTTPError(req.full_url, code, msg, hdrs, fp)
urllib.error.HTTPError: HTTP Error 308: Permanent Redirect
the line that brought the error (line 20):
file_info = req.urlopen("http://proget.whirlpool.repl.co/{}/{}.txt".format(arch, name))
(the link that it should use is correct..)
any ideas how it came?
SPECS:
OS: Windows 7 SP1
Python: Python 3.8.10
Arch: x86 (32-bit)
Http is outdated, so most sites would redirect to https.
So use
file_info = req.urlopen("https://proget.whirlpool.repl.co/{}/{}.txt".format(arch, name))
I get an error when i try to download a video from youtube with pytube.
from pytube import YouTube
yt = YouTube('https://www.youtube.com/watch?v=9bZkp7q19f0')
stream = yt.streams.first()
stream
stream.download()
The error says "Forbidden", but fetching e.g. with curl from the same URL seems to work just fine. What could be wrong here?
Traceback (most recent call last):
File "C:/Users/Mr. jarvis/Desktop/youtube/youtube.py", line 7, in <module>
stream.download()
File "C:\Users\Mr. jarvis\AppData\Local\Programs\Python\Python37-32\lib\site-packages\pytube\streams.py", line 217, in download
bytes_remaining = self.filesize
File "C:\Users\Mr. jarvis\AppData\Local\Programs\Python\Python37-32\lib\site-packages\pytube\streams.py", line 164, in filesize
headers = request.get(self.url, headers=True)
File "C:\Users\Mr. jarvis\AppData\Local\Programs\Python\Python37-32\lib\site-packages\pytube\request.py", line 21, in get
response = urlopen(url)
File "C:\Users\Mr. jarvis\AppData\Local\Programs\Python\Python37-32\lib\urllib\request.py", line 222, in urlopen
return opener.open(url, data, timeout)
File "C:\Users\Mr. jarvis\AppData\Local\Programs\Python\Python37-32\lib\urllib\request.py", line 531, in open
response = meth(req, response)
File "C:\Users\Mr. jarvis\AppData\Local\Programs\Python\Python37-32\lib\urllib\request.py", line 641, in http_response
'http', request, response, code, msg, hdrs)
File "C:\Users\Mr. jarvis\AppData\Local\Programs\Python\Python37-32\lib\urllib\request.py", line 569, in error
return self._call_chain(*args)
File "C:\Users\Mr. jarvis\AppData\Local\Programs\Python\Python37-32\lib\urllib\request.py", line 503, in _call_chain
result = func(*args)
File "C:\Users\Mr. jarvis\AppData\Local\Programs\Python\Python37-32\lib\urllib\request.py", line 649, in http_error_default
raise HTTPError(req.full_url, code, msg, hdrs, fp)
urllib.error.HTTPError: HTTP Error 403: Forbidden
Use pytube3 instead, it fixes this error (but is Python 3 only): https://github.com/hbmartin/pytube3
Experienced the same problem. I found unfinished downloaded file in the project folder. After deleting the unfinished downloaded file and running stream.download() again, it worked. So maybe worth checking.
pytube version 12.0.0
I am trying to download error log files such as IIS logs, HTTP API error logs from the server and save it to my disk. These logs contain the .log extension.
Have tried below code to download, it works well for text files but it didn't work for the files which are of type .log:
from urllib.request import urlopen
response = urlopen('https://servername/path/errorlog.txt') #doesn't work for response = urlopen('https://servername/path/errorlog.log')
data = response.read()
# Write data to file
filename = "test.txt"
file_ = open(filename, 'w')
file_.write(data)
file_.close()
This is the error messages i am getting:
Traceback (most recent call last):
File "<pyshell#26>", line 1, in <module>
response = urlopen(url)
File "C:\Python\lib\urllib\request.py", line 223, in urlopen
return opener.open(url, data, timeout)
File "C:\Python\lib\urllib\request.py", line 532, in open
response = meth(req, response)
File "C:\Python\lib\urllib\request.py", line 642, in http_response
'http', request, response, code, msg, hdrs)
File "C:\Python\lib\urllib\request.py", line 570, in error
return self._call_chain(*args)
File "C:\Python\lib\urllib\request.py", line 504, in _call_chain
result = func(*args)
File "C:\Python\lib\urllib\request.py", line 650, in http_error_default
raise HTTPError(req.full_url, code, msg, hdrs, fp)
urllib.error.HTTPError: HTTP Error 404: Not Found
Do you have a solution to this problem?
I'm setting my jenkins environment and, as part of this process, I need to create a slave node.
Bellow follow the script that is crashing:
import jenkins
server = jenkins.Jenkins('http://localhost:9090')
server.create_node('slave')
Follows the output:
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
File "/usr/local/lib/python2.7/dist-packages/jenkins/__init__.py", line 1156, in create_node
self._build_url(CREATE_NODE, params), b''))
File "/usr/local/lib/python2.7/dist-packages/jenkins/__init__.py", line 341, in jenkins_open
response = urlopen(req, timeout=self.timeout).read()
File "/usr/lib/python2.7/urllib2.py", line 127, in urlopen
return _opener.open(url, data, timeout)
File "/usr/lib/python2.7/urllib2.py", line 410, in open
response = meth(req, response)
File "/usr/lib/python2.7/urllib2.py", line 523, in http_response
'http', request, response, code, msg, hdrs)
File "/usr/lib/python2.7/urllib2.py", line 448, in error
return self._call_chain(*args)
File "/usr/lib/python2.7/urllib2.py", line 382, in _call_chain
result = func(*args)
File "/usr/lib/python2.7/urllib2.py", line 531, in http_error_default
raise HTTPError(req.get_full_url(), code, msg, hdrs, fp)
urllib2.HTTPError: HTTP Error 400: Bad Request
Use jenkinsapi module,
from jenkinsapi.jenkins import Jenkins
server = Jenkins('http://localhost:9090')
server.create_node('slave')
It appears to be a bug in python-jenkins 0.4.11, which was fixed by commit
302eb5dad6439842e076fead5bd76f20f615e6a8.
I got a similar error, upgrading python-jenkins fixed it.
I try to learn python network programming from book "Foundation of Python Network Programming" Apres,
first we learn about 'virtualenv' and I install virtualenv , try to get 'googlemap package' and try using simple program call 'search1' :
#!/usr/bin/env python
# Foundations of Python Network Programming - Chapter 1 - search1.py
from googlemaps import GoogleMaps
address = '207 N. Defiance St, Archbold, OH'
print GoogleMaps().address_to_latlng(address)
after that I try running it,
the result from the book is
$ python search1.py
(41.5228242, -84.3063479)
the result I get is error liket this
Traceback (most recent call last):
File "search1.py", line 6, in
print GoogleMaps().address_to_latlng(address)
File "/home/administrator/gmapenv/local/lib/python2.7/site-packages/googlemaps.py", line 310, in address_to_latlng
return tuple(self.geocode(address)['Placemark'][0]['Point']['coordinates'][1::-1])
File "/home/administrator/gmapenv/local/lib/python2.7/site-packages/googlemaps.py", line 259, in geocode
url, response = fetch_json(self._GEOCODE_QUERY_URL, params=params)
File "/home/administrator/gmapenv/local/lib/python2.7/site-packages/googlemaps.py", line 50, in fetch_json
response = urllib2.urlopen(request)
File "/usr/lib/python2.7/urllib2.py", line 127, in urlopen
return _opener.open(url, data, timeout)
File "/usr/lib/python2.7/urllib2.py", line 407, in open
response = meth(req, response)
File "/usr/lib/python2.7/urllib2.py", line 520, in http_response
'http', request, response, code, msg, hdrs)
File "/usr/lib/python2.7/urllib2.py", line 445, in error
return self._call_chain(*args)
File "/usr/lib/python2.7/urllib2.py", line 379, in _call_chain
result = func(*args)
File "/usr/lib/python2.7/urllib2.py", line 528, in http_error_default
raise HTTPError(req.get_full_url(), code, msg, hdrs, fp)
urllib2.HTTPError: HTTP Error 403: Forbidden
What make this error , because it I run down step by step like in the book