I have troubles with creating screenshots using imgkit. I can easily make a screenshot full site, but I don't know how to screenshot only specific div block in html. I'm totally noob in html, I don't think it is good idea to learn it for one python script...
For example, make a screenshot only for this google block
Related
I have a problem getting javascript content into HTML to use it for scripting. I used multiple methods as phantomjs or python QT library and they all get most of the content in nicely but the problem is that there are javascript buttons inside the page like this:
Pls see screenshot here
Now when I load this page from a script these buttons won't default to any value so I am getting back 0 for all SELL/NEUTRAL/BUY values below. Is there a way to set these values when you load the page from a script?
Example page with all the values is: https://www.tradingview.com/symbols/NEBLBTC/technicals/
Any help would be greatly appreciated.
If you are trying to achieve this with scrapy or with derivation of cURL or urrlib I am afraid that you can't do this. Python has another external packages such selenium that allow you to interact with the javascript of the page, but the problem with selenium is too slow, if you want something similar to scrapy you could check how the site works (as i can see it works through ajax or websockets) and fetch the info that you want through urllib, like you would do with an API.
Please let me know if you understand me or i misunderstood your question
I used seleneum which was perfect for this job, it is indeed slow but fits my purpose. I also used the seleneum firefox plugin to generate the python script as it was very challenging to find where exactly in the code as the button I had to press.
I'm trying to learn how to use selenium. I'm trying to work on creating script to like instagram photos; however, i'm running into a problem where xpath won't detect the image i want to click. I think this is probably due to the fact it's a javascript button.
This is a picture of the element i am inspecting. There's multiple pictures on the site and i am given the line
<a class="thumb-shadow" href="javascript:void(0);"></a>
https://gyazo.com/558df373e6ac426f098759665fd8f918
I've tried clicking the xpath of image wrapper, but it doesn't work either. How can i click the javascript item? Are there any resources you can point me to?
Thanks
Try driver.find_element_by_xpath("//a[#class='thumb-shadow']/img").click()
i need to write a python script , the script should access a webpage , which has a "upload" button , normally when you upload a photo with that button a new page opens . and once that page opens i need to look for a string there
so the script should upload there a photo , which i provide to the script and then check the output page for a string
i have no background in that sort of coding (i know basic python ) .
can i get a reference or some pointers on what reading should i do to perform that task? thank you very much
While this question is not specific enough to give you a good answer, I can make a couple of suggestions. I would look into using a library for sending requests to pages, such as requests. I would also look into libraries for parsing html, such as Beautiful Soup. Essentially you will need to use requests to get the page's html, and then you'll need to parse that html using Beautiful Soup to find what you're looking for on the page.
You should do some reading about these libraries and/or other similar ones and try to get a better understanding of your problem. Afterward, come back to Stack Overflow once you have more specific questions or problems you've run into.
This is my first StackOverflow post so please bear with me.
What I'm trying to accomplish is a simple program written in python which will change all of a certain html tag's content (ex. all <h1> or all <p> tags) to something else. This should be done on an existing web page which is currently open in a web browser.
In other words, I want to be able to automate the inspect element function in a browser which will then let me change elements however I wish. I know these changes will just be on my side, but that will serve my larger purpose.
I looked at Beautiful Soup and couldn't find anything in the documentation which will let me change the website as seen in a browser. If someone could point me in the right direction, I would be greatly appreciative!
What you are talking about seems to be much more of the job of a browser extension. Javascript will be much more appropriate, as #brbcoding said. Beautiful Soup is for scraping web pages, not for modifying them on the client side in a browser. To be honest, I don't think you can use Python for that.
I want to save a web page. I use python urllib to parse the web page. But I
find the saved file, where some content is missing. The missing part
is block from the source web page, such as this part <div
style="display: block;" id="GeneInts">...</div>.
I don't know how to parse a whole page without something block in it. Could you help me
figure it out? Thank you!
This is my program
url = 'http://receptome.stanford.edu/hpmr/SearchDB/getGenePage.asp?Param=4502931&ProtId=1&ProtType=Receptor'
f = urllib.urlretrieve(url,'test.html')
Whenever I need to let Javascript operate on a page before I can scrape it, the first thing I always turn to is SeleniumRC -- while it's mainly designed for purposes of testing, I've never found a better tool for this challenging task. For the "using it from Python" part, see here and links therefrom.
That page generates a great deal of its content with JavaScript executed at load-time, including, I think, the part you're trying to extract. You need a screen-scraper that can run JavaScript and then save out the modified DOM. I don't know where you get one of those.