We Keep Coding

Selenium Web driver, python: Test passes in local but fails in GitHub CI due to AssertionError

I´ve faced a problem that my one test passes in local but not in CI.
I have a progress bar with: positions 0%, 50%, 100% of starting a server, status of starting the server - "Server requested", "Spawning server...", "Server ready at" and status log.
I rewrote this case in few times but the result is the same - I got AssertionError, it cannot be catch one of values in CI.
I have no so big experience in Selenium and I´m stuck in which way I need to move to fix such problem. I´d appreciate if anyone helps me!!!
while '/spawn-pending/' in browser.current_url or is_displayed(
browser, SpawningPageLocators.PROGRESS_BAR
):
# checking text messages that the server is starting to up
texts = browser.find_elements(*SpawningPageLocators.TEXT_SERVER)
texts_list = []
for text in texts:
text.text
texts_list.append(text.text)
assert str(texts_list[0]) == SpawningPageLocators.TEXT_SERVER_STARTING
assert str(texts_list[1]) == SpawningPageLocators.TEXT_SERVER_REDIRECT
# checking progress of the servers starting (messages, % progress and events log))
logs_list = []
try:
progress_bar = browser.find_element(*SpawningPageLocators.PROGRESS_BAR)
progress = browser.find_elements(*SpawningPageLocators.PROGRESS_MESSAGE)
logs = browser.find_elements(*SpawningPageLocators.PROGRESS_LOG)
for status_message in progress:
progress_messages = status_message.text
percent = (
progress_bar.get_attribute('style')
.split(';')[0]
.split(':')[1]
.strip()
)
for i in range(len(logs)):
progress_log = logs[i].text
logs_list.append(progress_log)
if progress_messages == "":
assert percent == "0%"
assert int(len(logs_list)) == 0
if progress_messages == "Server requested":
assert percent == "0%"
assert int(len(logs_list)) == 1
> assert str(logs_list[0]) == "Server requested"
E AssertionError: assert '' == 'Server requested'
E - Server requested
After rewrote it to
while '/spawn-pending/' in browser.current_url:
progress_message = browser.find_element(
*SpawningPageLocators.PROGRESS_MESSAGE
).text
# checking text messages that the server is starting to up
texts = browser.find_elements(*SpawningPageLocators.TEXT_SERVER)
texts_list = []
for text in texts:
text.text
texts_list.append(text.text)
assert str(texts_list[0]) == SpawningPageLocators.TEXT_SERVER_STARTING
assert str(texts_list[1]) == SpawningPageLocators.TEXT_SERVER_REDIRECT
# checking progress of the servers starting (messages, % progress and events log))
logs_list = []
try:
progress_bar = browser.find_element(*SpawningPageLocators.PROGRESS_BAR)
progress = browser.find_element(*SpawningPageLocators.PROGRESS_MESSAGE)
logs = browser.find_elements(*SpawningPageLocators.PROGRESS_LOG)
percent = (
progress_bar.get_attribute('style').split(';')[0].split(':')[1].strip()
)
for log in logs:
# only include non-empty log messages
# avoid partially-created elements
if log.text:
logs_list.append(log.text)
except (NoSuchElementException, StaleElementReferenceException):
break
expected_messages = [
"Server requested",
"Spawning server...",
f"Server ready at {app.base_url}user/{user.name}/",
]
if progress_message:
assert progress_message in expected_messages
if logs_list:
# race condition: progress_message _should_
# be the last log message, but it _may_ be the next one
assert progress_message
if len(logs_list) < 2:
> assert percent == "0%"
Got : AssertionError: assert '50%' == '0%' on the line "assert percent == "0%"

PYTHON - Itertools.combinations results in memory error

I am creating a brute force script (for my studies) on python which connects to my login.php form and tries a combinaison of every characters of the alphabet to find the password.
The thing is that, after something like 110 000 combinaison, I am getting a memory error.
I already looked up on the net to find solutions which seems to be :
--> gc.collect() & del var
I tried to add the gc.collect() on the blank field of my function bruteforce so everytime i % 100 == 0, I clear the memory space but It dosen't work.
I can't find the way to add it in my script to free memory.
I think the problem is that I can't clear the memory space when my function is running. Maybe I should play with many threads like :
start 1
stop 1
clear space
start 2
stop 2
clear space
etc...
Do you guys have any suggestions ?
Here is my code :
import itertools, mechanize, threading, gc, os, psutil
charset='abcdefghijklmnopqrstuvwxyz'
br=mechanize.Browser()
combi=itertools.combinations(charset, 2)
br.open('http://192.168.10.105/login.php')
def check_memory():
process = psutil.Process(os.getpid())
mem_info = process.memory_info()
return mem_info.rss
def bruteforce():
i = 0
for x in combi:
br.select_form(nr=0)
br.form ['username'] = 'myusername'
tried = br.form ['password'] = ''.join(x)
i = i+1
print "Checking : ", tried, " Number of try : ", i
response=br.submit()
if response.geturl()=="http://192.168.10.105/index.php":
print("The password is : ", ''.join(x))
break
if i % 100 == 0:
mem_before = check_memory():
print "Memory before : ", mem_before
print "Free memory "
mem_after = check_memory()
print "Memory after : ", mem_after
x = threading.Thread(target = bruteforce)
x.start()
Thank you !

Thanks to #chepner, I found that the issue was with mechanize, not with itertools.
A good way to fix it is by clearing the history : "Out of Memory" error with mechanize

Python script to check bond status for Linux

I have the Below python code which I'm using to determine the Linux bond/team status. This code works just fine. I am not good at aligning the output formatting thus getting little hiccup.
I wanted the Printing Format into a Certain format, would appreciate any help on the same.
Below is the code exercise:
#!/usr/bin/python
# Using below file to process the data
# cat /proc/net/bonding/bond0
import sys
import re
def usage():
print '''USAGE: %s [options] [bond_interface]
Options:
--help, -h This usage document
Arguments:
bond_interface The bonding interface to query, eg. 'bond0'. Default is 'bond0'.
''' % (sys.argv[0])
sys.exit(1)
# Parse arguments
try:
iface = sys.argv[1]
if iface in ('--help', '-h'):
usage()
except IndexError:
iface = 'bond0'
# Grab the inf0z from /proc
try:
bond = open('/proc/net/bonding/%s' % iface).read()
except IOError:
print "ERROR: Invalid interface %s\n" % iface
usage()
# Parse and output
active = 'NONE'
Link = 'NONE'
slaves = ''
state = 'OK'
links = ''
bond_status = ''
for line in bond.splitlines():
m = re.match('^Currently Active Slave: (.*)', line)
if m:
active = m.groups()[0]
m = re.match('^Slave Interface: (.*)', line)
if m:
s = m.groups()[0]
slaves += ', %s' % s
m = re.match('^Link Failure Count: (.*)', line)
if m:
l = m.groups()[0]
links += ', %s' % l
m = re.match('^MII Status: (.*)', line)
if m:
s = m.groups()[0]
if slaves == '':
bond_status = s
else:
slaves += ' %s' % s
if s != 'up':
state = 'FAULT'
print "%s %s (%s) %s %s %s" % (iface, state, bond_status, active, slaves, links)
Result:
$ ./bondCheck.py
bond0 OK (up) ens3f0 , ens3f0 up, ens3f1 up , 0, 0
Expected:
bond0: OK (up), Active Slave: ens3f0 , PriSlave: ens3f0(up), SecSlave: ens3f1(up) , LinkFailCountOnPriInt: 0, LinkFailCountOnSecInt: 0

I tried to format in a very basic way as shown below :
print "%s: %s (%s), Active Slave: %s, PriSlave: %s (%s), SecSlave: %s (%s), LinkFailCountOnPriInt: %s, LinkFailCountOnSecInt: %s" % (iface, state, bond_status, active, slaves.split(',')[1].split()[0], slaves.split(',')[1].split()[1], slaves.split(',')[2].split()[0], slaves.split(',')[2].split()[1], links.split(',')[1], links.split(',')[2])
RESULT:
bond0: OK (up), Active Slave: ens3f0, PriSlave: ens3f0 (up), SecSlave: ens3f1 (up), LinkFailCountOnPriInt: 1, LinkFailCountOnSecInt: 1
However, I would suggest to get the values into variables prior and then use them in the print statement so as to avoid "out of index" issues during print() , as in rare cases like bond with only one interface will report indexing error while splitting hence good to get the values in variable and suppress the out of index into exception for those cases.

Do not use the way with "/proc/net/bonding/%s' for querying bond status. It could trigger system panic. Try to use "/sys/class/net/bondX/bonding", it is more safe.

Using sys.exit in for loops

I am writing a small python script that iterates through a large json output and grabs the information I need and puts it into small dictionaries. It then iterates through the dictionaries to look for an key called restartcount. If the count is more than more than 3 but less than 5 it prints warning. If greater than 5 it prints critical. However this script is set to be a nagios plugin which requires exit codes to be placed with warning sys.exit(1), and sys.exit(2) for critical. If you look at my script I use my function to grab the info I need into a small dictionary, then run a for loop. If I place a sys.exit after inside any if statement I iterate only through the first dictionary and the rest are not checked. Any help will be appreciated as to how to incorporate the exit codes without losing skipping or missing any information.
Code:
import urllib2
import json
import argparse
from sys import exit
def get_content(pod):
kube = {}
kube['name'] = pod["metadata"]["name"]
kube['phase'] = pod["status"]["phase"]
kube['restartcount'] = pod["status"]["containerStatuses"][0]["restartCount"]
return kube
if __name__ == '__main__':
parser = argparse.ArgumentParser( description='Monitor Kubernetes Pods' )
parser.add_argument('-w', '--warning', type=int, help='levels we should look into',default=3)
parser.add_argument('-c', '--critical', type=int, help='its gonna explode',default=5)
parser.add_argument('-p', '--port', type=int, help='port to access api server',default=8080)
args = parser.parse_args()
try:
api_call = "http://localhost:{}/api/v1/namespaces/default/pods/".format(args.port)
req = urllib2.urlopen(api_call).read()
content = json.loads(req)
except urllib2.URLError:
print 'URL Error. Please re-check the API call'
exit(2)
for pods in content.get("items"):
try:
block = get_content(pods)
print block
except KeyError:
print 'Container Failed'
exit(2)
if block["restartcount"] >= args.warning and block["restartcount"] < args.critical:
print "WARNING | {} restart count is {}".format(block["name"], block["restartcount"])
if block["restartcount"] >= args.critical:
print "CRITICAL | {} restart count is {}".format(block["name"], block["restartcount"])
what the block variable looks like:
{'phase': u'Running', 'restartcount': 0, 'name': u'pixels-1.0.9-k1v5u'}

Create a variable called something like exit_status. Initialize it to 0, and set it as needed in your code (e.g. where you are currently calling exit). At the end of program execution, call sys.exit(exit_status) (and no where else).
Rewriting the last section of your code:
exit_status = 0
for pods in content.get("items"):
try:
block = get_content(pods)
print block
except KeyError:
print 'Container Failed'
exit(2)
if block["restartcount"] >= args.warning and block["restartcount"] < args.critical:
print "WARNING | {} restart count is {}".format(block["name"], block["restartcount"])
if exit_status < 1: exit_status = 1
if block["restartcount"] >= args.critical:
print "CRITICAL | {} restart count is {}".format(block["name"], block["restartcount"])
exit_status = 2
sys.exit(exit_status)

The variable approach is correct
Problem is that as you check further you probably set it to 1 when it was already 2 so I would suggest add here a condition not to set it to 1 if it is already 2

benchmarking python script reveals mysterious time delays

i have two modules: moduleParent and moduleChild
i'm doing something like this in moduleParent:
import moduleChild
#a bunch of code
start = time.time()
moduleChild.childFunction()
finish = time.time()
print "calling child function takes:", finish-start, "total seconds"
#a bunch of code
i'm doing something like this in moduleChild:
def childFunction():
start = time.time()
#a bunch of code
finish = time.time()
print "child function says it takes:", finish-start, "total seconds"
the output looks like this:
calling child function takes: .24 total seconds
child function says it takes: 0.0 total seconds
so my question is, where are these .24 extra seconds coming from?
thank you for your expertise.
#
here is the actual code for "childFuntion". it really shouldn't take .24 seconds.
def getResources(show, resourceName='', resourceType=''):
'''
get a list of resources with the given name
#show: show name
#resourceName: name of resource
#resourceType: type of resource
#return: list of resource dictionaries
'''
t1 = time.time()
cmd = r'C:\tester.exe -cmdFile "C:\%s\info.txt" -user root -pwd root'%show
cmd += " -cmd findResources -machineFormatted "
if resourceName:
cmd += '-name %s'%resourceName
if resourceType:
cmd += '_' + resourceType.replace(".", "_") + "_"
proc=subprocess.Popen(cmd, stdout=subprocess.PIPE, stderr=subprocess.PIPE)
output = proc.stdout.read()
output = output.strip()
resourceData = output.split("\r\n")
resourceData = resourceData[1:]
resourceList = []
for data in resourceData:
resourceId, resourceName, resourceType = data.split("|")
rTyp = "_" + resourceType.replace(".", "_") + "_"
shot, assetName = resourceName.split(rTyp)
resourceName = assetName
path = '//projects/%s/scenes/%s/%s/%s'%(show, shot, resourceType.replace(".", "/"), assetName)
resourceDict = {'id':resourceId, 'name':resourceName, 'type':resourceType, 'path':path }
resourceList.append(resourceDict)
t2 = time.time()
print (" ", t2 - t2, "seconds")
return resourceList

Edit 2: I just noticed a typo in child function, you have t2 - t2 in the print statement
ignore below:
Calling the function itself has overhead (setting up stack space, saving local variables, returning, etc). The result suggests that your function is so trivial that setting up for a function call took longer than running the code itself.
Edit: also, calling the timers as well as print ads overhead. Now that I think about it, calling print could account for a lot of that .24 seconds. IO is slow.

You can't measure the time of a function by running it once, especially one which runs so short. There are a myriad of factors which could affect the timing, not the least of which is what other processes you have running on the system.
Something like this I would probably run at least a few hundred times. Check out the timeit module.