I'm trying to download files from a site using the wget module.
The code is really simple:
image = 'linkoftheimage'
wget.download(image)
This works fine, but it saves the file in the folder with the python script. My goal is to download it in a different folder, but I can't find a way to specify it.
I tried a different approach with os module .
os.system(f'wget -O {directory} {image}')
This metod gives me an error: sh: -c: line 0: syntax error near unexpected token `('
So I tried another method:
with open(f'{directory}/photo %s.jpg' %a,'wb') as handler:
handler.write(image)
This also didn't worked out.
Does anyone have an idea on how could I solve this?
the package you specified has not been updated since 2015, it's repository is gone and so should probably be avoided. you can download files using the built-in requests module like so:
import requests
image_url = 'https://www.fillmurray.com/200/300'
file_destination = 'desired/destination/file.jpg'
res = requests.get(image_url)
if res.status_code == 200: # http 200 means success
with open(file_destination, 'wb') as file_handle: # wb means Write Binary
file_handle.write(res.content)
Related
My python progam is creating wrong file path.
The file path formed is wrong : '/autocameratest2\data\TestImages/7_vw_test.png'
The correct file path should be: '/autocameratest2/data/TestImages/7_vw_test.png'
The file path is fp = builtins.open(filename, "rb")
FileNotFoundError: [Errno 2] No such file or directory: '/autocameratest2\\data\\TestImages/7_vw_test.png'
172.17.0.1 - - [05/Feb/2022 17:34:22] "POST /places?camid=1&image1test=7_vw_test.png&image2perfect=5_vw_master.png HTTP/1.1" 500 -
In the python api program, the below URL
http://127.0.0.1:5000/places?camid=1&image1test=7_vw_test.png&image2perfect=5_vw_master.png
returns a json file. The code works fine in VisualStudio code or outside docker. It python code give a path problem inside docker.
image1test_path = os.path.join(IMAGE_FOLD_PATH,'autocameratest2\data\TestImages',image1test)
image2perfect_path = os.path.join(IMAGE_FOLD_PATH,'autocameratest2\data\TestImages',image2perfect)
test_results = generate_report(camid, image1test_path, image2perfect_path)
test_names = ['CamId','Blur','check_scale','noise','scrolled','allign','mirror','blackspots','ssim_score','staticlines','rotation_deg']
Use pathlib, python's path parsing library, it has linux, windows type paths. You can freely manipulate between them.
Your problem is that docker runs on linux unless told otherwise, linux uses / for path and windows \, thus the issue.
pathlib.Path will help you.
>>> import pathlib
>>> pathlib.Path("xxx/yyy\iii.z")
WindowsPath('xxx/yyy/iii.z')
The following approach helped my create a file path that I wanted. like this "/autocameratest2/data/TestImages"
image1test_path = os.path.join('data','TestImages',image1test)
image2perfect_path = os.path.join('data','TestImages',image2perfect)
Other techniques which I tried for building the path is below. These are good though it did not solve my problme.
import pathlib
#image1test_path = pathlib.Path.cwd().joinpath('data', 'TestImages', args.image1test)
#image2perfect_path = pathlib.Path.cwd().joinpath('data', 'TestImages', args.image2perfect)
#image1test_path = pathlib.Path('data', 'TestImages', args.image1test)
#image2perfect_path = pathlib.Path('data', 'TestImages', args.image2perfect)
I have a URL which is a download link to a software .exe file.
The intended operation is to use Python 3 to download the said file and then do a silent installation.
import ssl
from urllib.parse import urlparse
import requests
#convert to string
url = str(url)
#convert ftp download path to http and remove chars to make it downloadable
httpurl = re.sub("ftp://","https://",url)
print("url is: " + url)
#function to break file name from the full path
def split(downloadurl):
p,downloadexename = os.path.split(downloadurl)
return [downloadexename]
#Remove all the unnecessary stuff you don't need
downloadurl = httpurl.replace("'","").replace(',','').replace(":2100/FTP Folders/Software","").replace(" ","%20").strip("(").strip(")")
print("Download URL is: "+downloadurl)
down_name = os.path.basename(downloadurl)
down_dir = r"C:\Desktop"
#Create folder if it doesn't exist for download as required
if not os.path.exists(down_dir):
os.makedirs(down_dir)
full_path = os.path.join(down_dir, down_name)
# Silent Install
subprocess.call([full_path, '/Silent'], shell=True)
It depends on what kind of installer the .exe is, but there is a high chance it already has the ability to install silently. For example, InnoSetup installers (a very common type) use the parameter /SILENT. If you don't know the installer type, you may try running the .exe with different variants of /s, /S, /SILENT etc. -- or some variants of /?, --help to show command line usage.
I'm new to Python. This is my first Ansible module in order to delete the SimpleDB domain from ChaosMonkey deletion.
When tested in my local venv with my Mac OS X, it keeps saying
Module unable to decode valid JSON on stdin. Unable to figure out
what parameters were passed.
Here is the code:
#!/usr/bin/python
# Delete SimpleDB Domain
from ansible.module_utils.basic import *
import boto3
def delete_sdb_domain():
fields = dict(
sdb_domain_name=dict(required=True, type='str')
)
module = AnsibleModule(argument_spec=fields)
client = boto3.client('sdb')
response = client.delete_domain(DomainName='module.params['sdb_domain_name']')
module.exit_json(changed = False, meta = response)
def main():
delete_sdb_domain()
if __name__ == '__main__':
main()
And I'm trying to pass in parameters from this file: /tmp/args.json.
and run the following command to make the local test:
$ python ./delete_sdb_domain.py /tmp/args.json
please note I'm using venv test environment on my Mac.
If you find any syntax error in my module, please also point it out.
This is not how you should test your modules.
AnsibleModule expects to have specific JSON as stdin data.
So the closest thing you can try is:
python ./delete_sdb_domain.py < /tmp/args.json
But I bet you have your json file in wrong format (no ANSIBLE_MODULE_ARGS, etc.).
To debug your modules you can use test-module script from Ansible hacking pack:
./hacking/test-module -m delete_sdb_domain.py -a "sdb_domain_name=zzz"
In my python script i am getting the links of the images from the webpage. But few of the images links are like this
image.php?u=155594&dateline=1182409179
The terminal says like this
HTTP request sent, awaiting response... 200 OK
Length: 4159 (4.1K) [image/png]
But image gets saved like image.php?blabla
is there any to save it in proper format with extension
For using wget, you need to use the -O option to specify output file. For example:
wget -O img.png http://example.com/image.php?foo=bar
Here's a little python script for when you don't know the type:
import os
import urllib2
import sys
d = urllib2.urlopen(sys.argv[0])
o = open('image.%s' % d.info().gettype(), 'w')
o.write(d.read())
I've looked all around Google and its archives. There are several good articles, but none seem to help me out. So I thought I'd come here for a more specific answer.
The Objective: I want to run this code on a website to get all the picture files at once. It'll save a lot of pointing and clicking.
I've got Python 2.3.5 on a Windows 7 x64 machine. It's installed in C:\Python23.
How do I get this script to "go", so to speak?
=====================================
WOW. 35k views. Seeing as how this is top result on Google, here's a useful link I found over the years:
http://learnpythonthehardway.org/book/ex1.html
For setup, see exercise 0.
=====================================
FYI: I've got zero experience with Python. Any advice would be appreciated.
As requested, here's the code I'm using:
"""
dumpimages.py
Downloads all the images on the supplied URL, and saves them to the
specified output file ("/test/" by default)
Usage:
python dumpimages.py http://example.com/ [output]
"""
from BeautifulSoup import BeautifulSoup as bs
import urlparse
from urllib2 import urlopen
from urllib import urlretrieve
import os
import sys
def main(url, out_folder="C:\asdf\"):
"""Downloads all the images at 'url' to /test/"""
soup = bs(urlopen(url))
parsed = list(urlparse.urlparse(url))
for image in soup.findAll("img"):
print "Image: %(src)s" % image
filename = image["src"].split("/")[-1]
parsed[2] = image["src"]
outpath = os.path.join(out_folder, filename)
if image["src"].lower().startswith("http"):
urlretrieve(image["src"], outpath)
else:
urlretrieve(urlparse.urlunparse(parsed), outpath)
def _usage():
print "usage: python dumpimages.py http://example.com [outpath]"
if __name__ == "__main__":
url = sys.argv[-1]
out_folder = "/test/"
if not url.lower().startswith("http"):
out_folder = sys.argv[-1]
url = sys.argv[-2]
if not url.lower().startswith("http"):
_usage()
sys.exit(-1)
main(url, out_folder)
On windows platform, you have 2 choices:
In a command line terminal, type
c:\python23\python xxxx.py
Open the python editor IDLE from the menu, and open xxxx.py, then press F5 to run it.
For your posted code, the error is at this line:
def main(url, out_folder="C:\asdf\"):
It should be:
def main(url, out_folder="C:\\asdf\\"):
Usually you can double click the .py file in Windows explorer to run it. If this doesn't work, you can create a batch file in the same directory with the following contents:
C:\python23\python YOURSCRIPTNAME.py
Then double click that batch file. Or, you can simply run that line in the command prompt while your working directory is the location of your script.
Since you seem to be on windows you can do this so python <filename.py>. Check that python's bin folder is in your PATH, or you can do c:\python23\bin\python <filename.py>. Python is an interpretive language and so you need the interpretor to run your file, much like you need java runtime to run a jar file.
use IDLE Editor {You may already have it} it has interactive shell for python and it will show you execution and result.
Your command should include the url parameter as stated in the script usage comments.
The main function has 2 parameters, url and out (which is set to a default value)
C:\python23\python "C:\PathToYourScript\SCRIPT.py" http://yoururl.com "C:\OptionalOutput\"
If you want to run .py files in Windows, Try installing Git bash
Then download python(Required Version) from python.org and install in the main c drive folder
For me, its :
"C:\Python38"
then open Git Bash and go to the respective folder where your .py file is stored :
For me, its :
File Location : "Downloads"
File Name : Train.py
So i changed my Current working Directory From "C:/User/(username)/" to "C:/User/(username)/Downloads"
then i will run the below command
" /c/Python38/python Train.py "
and it will run successfully.
But if it give the below error :
from sklearn.model_selection import train_test_split
ModuleNotFoundError: No module named 'sklearn'
Then Do not panic :
and use this command :
" /c/Python38/Scripts/pip install sklearn "
and after it has installed sklearn go back and run the previous command :
" /c/Python38/python Train.py "
and it will run successfully.
!!!!HAPPY LEARNING !!!!