With the "Inspect Element" option in a common browser it is possible to access the "Sources" tab and, not only see the files which the website uses, but also mark a line of code (as shown in the image below at line 463 with a .js file), which will make the browser pause when that line of code is executed (essentially a debugger). In this sense, it seems possible to check if a certain line of code is executed, which is what I need to finish an automation with Python, preferably with Selenium, but which I also don't know how to do.
Selenium tests are intended to be "black box". That is, you load a page and only access the things that are available in the browser window to verify that behavior is correct. You should NOT try to verify that specific parts of code were written. Even if you can figure out a way to do this, it will make your tests extremely brittle.
If you are writing the JavaScript code, then I suggest using a framework such as Jest or Mocha to test it directly. You still shouldn't verify that a specific line of code is executed, but you can test a function by calling it directly and checking the return value or side effects to make sure they are correct.
Related
So, as many have encountered Chrome's memory usage can be debilitating so I want to PROGRAMMATICALLY access "Chrome's Task Manager" or something equivalent which displays Tabs, their PIDs, AND SUBFRAMES (if possible) so I can create a script to kill them. After a ton of reading, and a lot of answers suggesting that it might not be possible and some where the posters didn't answer the question, I'm just not sure that the answer to THIS question has truly definitively been given yet.
The closest I've gotten is using "chrome://system/" in the omnibox, but when you expand "mem_usage" it doesn't give you useful memory usage numbers per tab at all and lacks the subframes, but with this method I'd planned to use the tab titles to kill the tabs if I could have actually gotten their memory footprint.
I tried using the win32gui's FindWindowEx method to get the text, but that requires you to know the child element of the text displayed so I downloaded "Window Detective" to see the name of the element(s) the "Chrome Task Manager" displays, the PIDs, and Tab names for, but "Window Detective" only came back with the "EmbeddedMenuWindowClass" which is the entire Window, not related to the text displayed.
I made a fleeting effort to use Selenium, but it seems as though it's just not capable of grabbing Special Menus inside of Chrome over just grabbing tab names and elements within a webpage.
I also tried "pywinauto" but that thing's documentation is really, really bad. I've tried using the "connect" method, "hwndwrapper", etc. but grasping what parameters to use, how their being passed, and what to expect along with how to then use the output you get (such as a list of PIDs or Handles) is seriously bungled. I would have loved if the documentation included the OUTPUT so I could actually see what I'm supposed to get so I don't end up rabbit-holing only to find out that everything I've tested doesn't deliver what I'm looking for. Sorry if that sounds whiny, but I spent hours reading the PDF and the official docs and got badly frustrated.
I also tried launching chrome via the Command Line following this document for logging (http://www.chromium.org/for-testers/enable-logging), but it may be a Chromium only thing, but cannot find the logging document that's created and when I redirect its output to a file, I get nothing useful.
But yes, does anyone know how to programmatically access "Chrome Task Manager's" text, or something else that will deliver similar output? Thank you.
Why to just to use the python os module something like this:
import subprocess as sub
p = sub.Popen(['your command', 'arg1', 'arg2', ...],stdout=sub.PIPE,stderr=sub.PIPE)
output, errors = p.communicate()
print output
You can use the command ps to list all the proceses running in chrome however that only will give you the pid not the name of tab to get that I think the only way is to get the data directly from chorme by creating an extension.
https://developer.chrome.com/extensions/tabs
I am trying to find a way that I can have a program step through Python code line by line and do something with the results of each line. In effect a debugger that could be controlled programmatically rather than manually. pdb would be exactly what I am looking for if it returned its output after each step as a string and I could then call pdb again to pickup where I left off. However, instead it outputs to stdout and I have to manually input "step" via the keyboard.
Things I have tried:
I am able to redirect pdb's stdout. I could redirect it to a second
Python program which would then process it. However, I cannot
figure out how to have the second Python program tell pdb to
step.
Related to the previous one, if I could get pdb to step all the way
through to the end (perhaps I could figure out something to spoof a
keyboard repeatedly entering "step"?) and redirect the output to a
file, I could then write another program that acted like it was
stepping through the program when it was actually just reading the
file line by line.
I could use exec to manually run lines of Python code. However,
since I would be looking at one line at a time, I would need to
manually detect and handle things like conditionals, loops, and
function calls which quickly gets very complicated.
I read some posts that say that pdb is implemented using
sys.settrace. If nothing else works I should be able to recreate
the behavior I need using this.
Is there any established/straight forward way to implement the behavior that I am looking for?
sys.settrace() is the fundamental building block for stepping through Python code. pdb is implemented entirely in Python, so you can just look at the module to see how it does things. It also has various public functions/methods for stepping under program control, read the library reference for your version of Python for details.
I read some posts that say that pdb is implemented using sys.settrace.
If nothing else works I should be able to recreate the behavior I need
using this.
Don't view this as a last resort. I think it's the best approach for what you want to accomplish.
I have a couple of functions written in a single Python file. They perform a sequence of steps on a file-based dataset.
My workflow:
After I finished coding a part of the function's body, I run the function to see how it goes.
It may break at a certain point.
I fix the code and re-run the function.
The problem is that when I re-run the function, it will execute the lines that were already completed successfully. Yet I want to be able to start not from the beginning but rather from an arbitrary point. This is because the whole function runs for several minutes and it would be a waste time to wait for it to complete.
I could implement "checks" to see whether this operation is required (e.g., don't create a file if it already exists), but this would imply adding a lot of new validation code (e.g., make sure that the existing file does contain the content needed); in reality, my function will be run on a dataset in known format and the whole function should be executed.
The most obvious solution is to comment out the parts that were executed successfully, but it's a hustle and I got tired of commenting and uncommenting parts as I move forwards and the function gets larger.
Are there are any better approaches than commenting out lines for ignoring certain part of the function's body when executing?
I am on Wing IDE if this has something to do with the debugging tricks in an IDE itself.
Wing can move the program counter to a different line in the function via the right-click popup menu, but you'd need to do this every time you run the function. I think a better approach is to refactor the function into smaller functions -- then you can comment out or conditionalize only the function calls. You could also write tests that call some of the functions and not others.
As programmers we read more than we write. I've started working at a company that uses a couple of "big" Python packages; packages or package-families that have a high KLOC. Case in point: Zope.
My problem is that I have trouble navigating this codebase fast/easily. My current strategy is
I start reading a module I need to change/understand
I hit an import which I need to know more of
I find out where the source code for that import is by placing a Python debug (pdb) statement after the imports and echoing the module, which tells me it's source file
I navigate to it, in shell or the Vim file explorer.
most of the time the module itself imports more modules and before I know it I've got 10KLOC "on my plate"
Alternatively:
I see a method/class I need to know more of
I do a search (ack-grep) for the definition of that method/class across the whole codebase (which can be a pain because the codebase is partly in ~/.buildout-eggs)
I find one or more pieces of code that define that method/class
I have to deduce which one of them is the one I need to read
This costs a lot of time, which is understandable for a big codebase. But I get the feeling that navigating a large and unknown Python codebase is a common enough problem.
So I'm looking for technical tools or strategic solutions for this problem.
...
I just can't imagine hardcore Python programmers using the strategies outlined above.
on Vim, I like NERDTree (a file browser) and taglist.vim (source code browser --> http://www.vim.org/scripts/script.php?script_id=273)
also in Vim, you can use CTRL-] to jump to a definition (:h CTRL-]):
download exuberant ctags http://ctags.sourceforge.net/
follow the install directions and put it somewhere on your PATH
from the 'root' directory of your source code, make a tags file from the shell: "ctags -R"
(make sure you have :set noautochdir, and make sure :pwd is the root directory from step 3)
go into Vim, cursor over some function or class name, hit CTRL-]
by default, if there's multiple matches for the tag, it shows you everywhere it was imported, and where it was declared
if the tag only has one match, it immediately jumps to it
...then use Ctrl+O and Ctrl+I to move back and forth from where you were
(repeat above steps for the source code of particular libraries you use, i usually keep a separate Vim window open to study stuff)
I use ipython's ?? command
You just need to figure out how to import the things you want to look for, then add ?? to the end of the module or class or function or method name to view their source code. And the command completion helps on figuring out long names as well.
Try red pill: https://github.com/klen/python-mode
I am new to selenium and need some clarification on some stuff. I've tried click and clickAndWait in the IDE, and while the references stated clearly what each meant, when I export the test case to Python, it seems like both are doing the same thing here
driver.find_element_by_xpath("//li[#id='pa-u_8298348-bd']/a/span[2]").click() #click and wait
driver.find_element_by_link_text("IMVironments").click() #click
Can someone tell me what is the difference here then?
From Selenium docs:
Many Actions can be called with the "AndWait" suffix, e.g.
"clickAndWait". This suffix tells Selenium that the action will cause
the browser to make a call to the server, and that Selenium should
wait for a new page to load.
2 years later, I know. But I was looking for the same answer and found it.
From the Selenium Wiki
The AndWait alternative is always used when the action causes the browser to navigate to another page or reload the present one.
In other words, Selenium's webdriver, when utilizing the Click() command, will inherently use the "AndWait" modifier if it recognizes a page load.
See also Interface WebElement
void click()
Click this element. If this causes a new page to load, this method will attempt to block until the page has loaded.
I'm not sure if this is a python thing, but it appears you are only calling 'click' and are not calling 'clickAndWait'. I think this is what Trott is pointing out too.
With this said I have noticed that IE often requires AndWait a bit more often than Firefox or Chrome, which seem to deal with just click a bit better (due to speed?). You can also create your own 'wait' in Selenium, which is what I usually do in critical situations (as in you need to interact with a specific element so you 'wait' until it exists).
EDIT:
After your comment, I now understand what you are saying (was mildly confused about the 'clickandwait' comment :P). From what I can tell AndWait is not always available for the Python binding, I found that here, but I'll admit it's not 100% clear as it suggests otherwise in other places. Perhaps this would be useful instead?
wait_for_page_to_load(timeout)[source]
Waits for a new page to load.
You can use this command instead of the “AndWait” suffixes,
“clickAndWait”, “selectAndWait”, “typeAndWait” etc. (which are only
available in the JS API).
Placed this as an edit, because was too long for a comment, sorry!