We Keep Coding

How do I get the list of all images on a page?

In Firefox, I can get a list of all images from the "Media" tab of the Page Info window:
How can I obtain such a list using Python Selenium? In addition to getting such a list of image URLs, I would also like to be able to get each image's data (i.e. the image itself) without needing to make additional network requests.
Please DO NOT suggest that I parse the HTML to look for <img ... /> tags. That is clearly not what I'm looking for. I am looking for image responses. Not all image responses are present in the DOM. Example: some image responses from AJAX requests.

Understanding google's HTML

first time poster here.
I am just getting into python and coding in general and I am looking into the requests and BeutifulSoup libraries. I am trying to grab image url’s from google images. When inspecting the site in chrome i can find the “div” and the correct img src url. But when I open the HTML that “requests” gives me I can find the same “div” but the img src url is something completely different and only leads to a black page if used.
Img of the HTML requests get
Img of the HTML found in chrome's inspect tool
What I wonder, and want to understand is:
why are these HTML's different
How do I get the img src that is found with the inspect tool with requests?
Hope the question makes sense and thank you in advance for any help!

Maybe differences between the the response HTML and the code in chrome inspector stems for updates to the page when JS changes it . for example when you use innerHTML() to edit div element so the code you add will add to DOM stack so as the code in the inspector but it would have no influence on the response.
You may search the http:// in the begging and the .png or .jpg or any other image format in the end.

Simply put, your code retrieves a single HTML page, and lets you access it, as it was retrieved. The browser, on the other hand, retrieves that HTML, but then lets the scripts embedded in (or linked from) it run, and these scripts often make significant modifications to the HTML (also known as DOM - Document Object Model). The browser's inspector inspects the fully modified DOM.

how to extract youtube thumbnail from youtube link in python

I want to extract and display Youtube search results for a query to the user.
In that process, I have completed fetching the Youtube link and also extracted the title from the link.
Nevertheless I also want the thumbnail of that link displayed, same as that displayed in Youtube suggestions section.

For a question like this, I'd recommend using the site:youtube.com Google Images search, and just have a look at one or two thumbnails. I believe the below should work in all cases, though you'd need to test on different types of videos.
If the video URL is https://www.youtube.com/watch?v=xxxxxxxxxxxx
The thumbnail URL is https://i.ytimg.com/vi/xxxxxxxxxxxx/maxresdefault.jpg

Can I serve generated images without storing them on the server side?

I have a form where the user uploads two images, and based on them I generate about 10 other images ( using PIL ). The thing is, I want to show an HTML page that contains all the generated images, but I would like not to have to store them on server side. Is this possible?

You could use the Data URI scheme. The examples section on that Wikipedia article has some nice things to start with. What you need then, is to convert the binary image data to base64 so you can include it on your page. Fortunately, there are scripts available for this already.
Browser support seems okay, all major browser have no problem with it. For IE, it is supported from IE8 upwards (with IE8 having a limitation of 32KiB for the URI size).

You need to design your URLs correctly and have a view for an URL pointing to an image. In that view then you send the generated image. The Django website has an example for a PDF.
https://docs.djangoproject.com/en/1.3/howto/outputting-pdf/
In your (static?) HTML page you have a reference then like
<img src="/dyn_images/foo.png"/>
and an URL rule watching for that.

Including a dynamic image in a web page using POST?

I have written a CGI script that creates an image dynamically using GET data. To include this image in my webpage, I am using the following code:
<img src="image.py?text=xxxxxxxxxxxxxx">
The problem is that I expect in the future the "text" field will get very long and the URL will become too large. From Googling around there doesn't seem to be a fixed limit on URL length (ie. depends on the browser, server, proxy, etc.) Is there a better way to do this?
If it matters, I am working with Django and Python and I cannot use any client-side scripting (ie. JavaScript).
Cheers,
Ben

Store the text somewhere (e.g. a database) and then pass through the primary key.

This will get you an Image as the result of a POST -- you may not like it
Put an iFrame where you want the image and size it and remove scrollbars
Set the src to a form with hidden inputs set to your post parameters and the action set to the URL that will generate the image
submit the form automatically with JavaScript in the body.onload of the iFrame's HTML
Then, either:
Serve back an content-type set to an image and stream the image bytes
or:
store the post parameters somewhere and generate a small id
serve back HTML with an img tag using the id in the url -- on the server look up the post parameters
or:
generate a page with an image tag with an embedded image
http://danielmclaren.net/2008/03/embedding-base64-image-data-into-a-webpage

Putting together what has already been said, how about creating two pages. First page sends a POST request when the form is submitted (lets say to create_img.py) with a text=xxxxxxx... parameter. Then create_img.py takes the text parameter and creates an image with it and inserts it (or a filesystem reference) into the db, then when rendering the second page, generate img tags like <img src="render_img.py?row_id=0122">. At this point, render_img.py simply queries the db for the given image. Before creating the image you can check to see if its already in the database therefore reusing/recycling previous images with the same text parameter.

img's use GET. You'll have to come up with another mechanism. How about calling the same functionality in image.py and saving the file as a temp file which you ref in the img tag? Or how about saving the value of text in a db row during the rendering of this img tag and using the row_id as what you pass into the image.py script?

You may be able to mitigate the problem by compressing the text in the get parameter.

From the link below it looks like you'll be fine for a while ;)
http://www.boutell.com/newfaq/misc/urllength.html

If you're using django, maybe you can do this via a template tag instead?
Something like:
<img src="{% create_image "This is the text that will be displayed" %}">
The create_image function would create the image with a dummy/random/generated filename, and return the path.
This avoids having to GET or POST to the script, and the images will have manageable filenames.
I can see some potential issues with this approach, I'm just tossing the idea out there ;)

OK, I'm a bit late to the party, but you could use a mix of MHTML (for IE7 and below) and the data URI scheme (for all other modern browsers). It does require a bit of work on both client and server but you can ultimately end up with
newimg.src = 'blah';
The write-up on how to do this is at http://gingerbbm.com/?p=127.