Scenario: Website x has a form in which a file can be uploaded from your local machine.
Question: Is it possible to pass in a remote url (e.g. http://blah.whatever/file.ext) instead of a local file, thus causing the remote server to download the file found at http://blah.whatever/file.ext instead of having to download the file to my local machine and then upload it?
I'm most familiar with Python. So, if this is possible through something like the requests module, that would be fantastic.
Related
I have a script that scans a local folder and uploads some of the files to an SQL Server through a POST request. I would love to modify it to take files from a GCP bucket instead of from local storage. However I have no experience with GCP and I am having difficulty finding documentation supporting what I am trying to do. I have a few questions for anyone who has tried anything like this before:
Is a GET request the best way to copy GCP bucket files into a different location? I.e Is there a way to put my script directly into GCP and just use the POST request I already have, referencing the bucket instead of a folder?
If a GET request is the best way, does anyone know of a good resource to learn about HTTPS requests with GCP? (Not sure how to create the GET request/ what information Google would need).
After my GET request (if this is the best way), do the files necessarily have to download to my computer before the POST request to the SQL server OR is there any way to send the files to upload without having to download them?
If you want to replace your local storage by Cloud Storage, you have several things to know
The most transparent, and if you use a linux compliant OS, is to use GCSFuse. You will be able to mount a Cloud Storage bucket in a local directory and work as it was a local storage. HOWEVER, GCSFuse is a wrapper that transform system call to HTTP calls, latency, feature and performance are absolutely not the same.
When you search a file on Cloud Storage, you can only search with a prefix, not with a suffix (if you look for special extension such as .sql or .csv, it's not possible).
You must download the file content locally before sending it to your database. Except if you have a module/extension in your database able to read data from an URL or directly from Cloud Storage (that shouldn't exist).
gsutil is the best tool to handle the Cloud Storage file
I am working on a Python tool to synchronize files between my local machines and a remote server. When I upload a file to the server the modification time property of that file on the server is set to the time of the upload process and not to the mtime of the source file, which I want to preserve. I am using FTP.storbinary() from the Python ftplib to perform the upload. My question: Is there a simple way to preserve the mtime when uploading or to set it after the upload? Thanks.
Short answer: no. The Python ftplib module offers no option to transport the time of the file. Furthermore, the FTP protocol as defined by rfc-959 has no provision to directly get not set the mtime of a file. It may be possible on some servers through SITE commands, but this is server dependant.
If it is possible for you, you should be able to pass a site command with the sendcmd method of a connection object. For example if the server accepts a special SITE SETDATE filename iso-8601-date-string you could use:
resp = ftp.sendcmd(f'SITE SETDATE {file_name} {date-string}')
I am working on a system (python program) that runs on local machine but it needs to fetch data hosted somewhere on web (images in my case).
What it does is:
Send a SQL query to webhost (localhost currently)
The response sends back the names of images (it is stored in an array called fetchedImages lets assume).
Now once I have all the names of required images all I want to do is access the file directly from localhost and copy it to local machine. But this is what my problem is:
I am trying to access it as:
source = "localhost/my-site/images"
localDir = "../images"
for image in fetchedImages:
copy(source+image,localDir)
but the problem is that, the localhost is created using XAMPP and I cannot access localhost since python doesn't accept it as path. How can I access localhost if it isn't created via SimpleHTTPServer but XAMPP?
It can be solved using requests as:
import requests as req
from StringIO import StringIO
from PIL import Image
source = "http://localhost/my-site/images/"
localDir = "../images"
for image in fetchedImages:
remoteImage = req.get(source+image)
imgToCopy = Image.open(StringIO(remoteImage.content))
imgToCopy.save(localDir+image)
the requests will access the web resource thus making system easy to work with dynamic paths (localhost/my-site or www.my-site.com) and then copy those resources to local machine for processing.
I have a server, which is running perfectly in this address (example): http://stackoverflow.cloud.com
And I have some files in this address which I would like to be download from there to my computer. I installed in the server the IIS and I can access to the files this way:
http://stackoverflow.cloud.com/files/nameOfFile.png
With this last URL I would like to download the file to my computer.
import urllib
urllib.urlretrieve('http://stackoverflow.cloud.com/files/nameOfFile.png', 'C:\\myFile.png')
With this code instead of saving the file to my computer, it saves the file in the harddisk of the server.
I tried the module Webbrowser, but I'm working with python version 2.6 and Webbrowser is for 3.2 or something.
Any other idea of how I can accomplish this?
Solved.
Seems that openERP has is own way of popup links:
return { 'type': 'ir.actions.act_url', 'url': 'http://stackoverflow.cloud.com/files/nameOfFile.png'}
Thank you all.
I never tried using urllib before but I can guide you to read the documentation for the urllib.urlretrieve function in the Python standard library:
urllib.urlretrieve(url[, filename[, reporthook[, data]]])
Copy a network object denoted by a URL to a local file, if necessary. If the URL points to a local file, or a valid cached copy of the object exists, the object is not copied. Return a tuple (filename, headers) where filename is the local file name under which the object can be found, and headers is whatever the info() method of the object returned by urlopen() returned (for a remote object, possibly cached). Exceptions are the same as for urlopen().
How can I download files from a website using wildacrds in Python? I have a site that I need to download file from periodically. The problem is the filenames change each time. A portion of the file stays the same though. How can I use a wildcard to specify the unknown portion of the file in a URL?
If the filename changes, there must still be a link to the file somewhere (otherwise nobody would ever guess the filename). A typical approach is to get the HTML page that contains a link to the file, search through that looking for the link target, and then send a second request to get the actual file you're after.
Web servers do not generally implement such a "wildcard" facility as you describe, so you must use other techniques.
You could try logging into the ftp server using ftplib.
From the python docs:
from ftplib import FTP
ftp = FTP('ftp.cwi.nl') # connect to host, default port
ftp.login() # user anonymous, passwd anonymous#
The ftp object has a dir method that lists the contents of a directory.
You could use this listing to find the name of the file you want.