I am trying to submit an image URL from Reddit.com to a vision API using requests.get() in Python but I am running into difficulties in what could be a simple error on my part. The requests.get() request is successful when the link points to an explicit *.jpg, e.g., https://upload.wikimedia.org/wikipedia/commons/thumb/2/2b/Beef_fillet_steak_with_mushrooms.jpg/800px-Beef_fillet_steak_with_mushrooms.jpg, but unsuccessful when the link points to what I perceive to be a soft link, e.g., https://preview.redd.it/9xu97c5snpr51.jpg?width=640&crop=smart&auto=webp&s=e68c02166f6fd21a47a957b187b98b92608f54a9. Note that when pasted into a browser, both links work fine.
Does anyone have a suggestion for how I might preprocess the second link so it is handled like the first link? I would like to eventually have this code run remotely, so avoiding having to download the file locally is preferred.
From the documentation on: https://requests.readthedocs.io/en/master/user/quickstart/
You can access the response body as bytes, for non-text requests:
from PIL import Image
from io import BytesIO
i = Image.open(BytesIO(r.content))
Related
This is my first question I've posted here so let me know if I need to add more information. I have set up a python code which utilizes requests.post to send an HTTP request to the website (the code shown below). I am trying to post the data that is sent from python to the weebly website I have created. I believe the easiest option for this would be to embed HTML code into the website, however I have never used HTML before and cannot find a good source to learn it.
Python code:
import requests
DataSent = {"somekey":"somevalue"}
url = "http://www.greeniethegenie123.weebly.com"
r = requests.post(url, data = DataSent)
print(r.text)
Edit: The question is how can I set up an HTML code to receive the request and post it on the website. Or if there is any other way to send the data that would work too. I just have a sensor recording numbers that I would like to post to the weebly website.
Edit: It looks like HTML is not possible to do this, does anyone have other advice for how to send data from a raspberry pi to a website? The main problem is the website needs to update the data every minute to be useful in what I am trying to do.
You would have to use Javascript instead of HTML to accomplish this.
HTML is used for the structure of a webpage, while javascript can be used for requests, updating content, and lots of other stuff.
Here are some links to help you out on HTML and Javascript:
HTML Intro
Javascript Intro
For requests with Javascript, I would recommend using Axios:
Axios NPM
Here's a link explaining how to use Axios as well:
Axios Tutorial
import urllib.request
url = 'https://cdn.discordapp.com/avatars/305196810048110603/f31411d41b42b65a0b6eca686dd67b08.webp?size=1024.jpg'
pic = open('abc.jpg', 'wb+')
pic.write(urllib.request.urlopen(urllib.request.Request(url, headers={'User-Agent': 'Mozilla/5.0'})).read())
pic.close()
So, I've browsed some questions on here and this is my current code which fakes a browser since discord doesn't like people downloading avatar pics.
The problem I'm having is that the image seems to be blank, which means that there is something wrong with my usage of urllib. I would appreciate any help I can get
Try opening the image in Chrome. It seems to work fine there. It seems the image is not actually a jpeg, but WEBPVP8/RIFF. I haven't encountered that before.
Here is a page explaining how to convert riff files to jpg/png
http://www.freewaregenius.com/convert-webp-image-format-jpg-png-format/
I know there is a python-imagemagick library, that might be useful for automating / coding the conversion.
I have been trying to download an image from website (no username and password required) but every time I am getting an empty file. I have used conventional urllib .retrieve and requests methodologies but getting the same result. One thing more is that if I try to open the same image manually by copy pasting the URL after 15-20 min then that image itself does not open. I am assuming that some sort of session handling is required in this case . Below is my code which returns me empty image.
import os
import urllib
def savePic(url):
uri="C:\Python27\Scripts\Photosurl2.jpg"
if url!="":
urllib.urlretrieve(url, uri)
savePic("http://www-nass.nhtsa.dot.gov/nass/cds/GetBinary.aspx?ImageView&ImageID=491410290&Desc=Lookback+from+final+rest&Title=Scene+Photos+-+image1&Version=1&Extend=jpg")
Any help is appreciated.
When you try to implement some HTTP code in Python do not forget to validate that you can use curl or wget to perform these HTTP requests. This will save you a lot of time trying to debug a problem that is not in your code.
The also have very good verbose modes which will give you some hints regarding what you are missing.
Also, most senior Python developers are using the requests library instead of the urllib ones.
PS. Requests library is easier to use than urllib.
I am trying to fetch some information from Workflowy using Python Requests Library. Basically I am trying to programmatically get the content under this URL: https://workflowy.com/s/XCL9FCaH1b
The problem is Workflowy goes through a 'loading phase' before the actual content is displayed when I visit this website so I end up getting the content of 'loading' page when I get the request. Basically I need a way to defer getting the content so I can bypass the loading phase.
It seemed like Requests library is talking about this problem here: http://www.python-requests.org/en/latest/user/advanced/#body-content-workflow but I couldn't get this example work for my purposes.
Here is the super simple block of code that ends up getting the 'loading page':
import requests
path = "https://workflowy.com/s/XCL9FCaH1b"
r = requests.get(path, stream=True)
print(r.content)
Note that I don't have to use Requests just picked it up because it looked like it might offer a solution to my problem. Also currently using Python 2.7.
Thanks a lot for your time!
I am trying to grab a PNG image which is being dynamically generated with JSP in a web service.
I have tried visiting the web page it is contained in and grabbing the image src attribute; but the link leads to a .jsp file. Reading the response with urllib2 just shows a lot of gibberish.
I also need to do this while logged into the web service in question, using mechanize. This seems to exclude the option of grabbing a screenshot with webkit2png or similar.
Thanks for any suggestions.
If you use urllib correctly (for example, making sure your User-Agent resembles a browser etc), the "gibberish" you get back is the actual file, so you just need to write it out to disk (open the file with "wb" for writing in binary mode) and re-read it with some image-manipulation library if you need to play with it. Or you can use urlretrieve to save it directly on the filesystem.
If that's a jsp, chances are that it takes parameters, which might be appended by the browser via javascript before the request is done; you should look at the real request your browser makes, before trying to reproduce it. You can do that with the Chrome Developer Tools, Firefox LiveHTTPHeaders, etc etc.
I do hope you're not trying to break a captcha.