I have built a multiprocessing password cracker (using a wordlist) for a specific function, it halved the time needed compared to using a single process.
The original problem being that it would show you the cracked password and terminate the worker, but the remaining workers would carry on until they ran out of words to hash! not ideal.
My new step forward is to use Manager.Event() to terminate the remaining workers, this works as I had hoped (after some trial and error), but the application now takes far longer that it would take as a single process, I'm sure this must be due to the if function inside pwd_find() but I thought I would seek some advice.
#!/usr/bin/env python
import hashlib, os, time, math
from hashlib import md5
from multiprocessing import Pool, cpu_count, Manager
def screen_clear(): # Small function for clearing the screen on Unix or Windows
if os.name == 'nt':
return os.system('cls')
else:
return os.system('clear')
cores = cpu_count() # Var containing number of cores (Threads)
screen_clear()
print ""
print "Welcome to the Technicolor md5 cracker"
print ""
user = raw_input("Username: ")
print ""
nonce = raw_input("Nonce: ")
print ""
hash = raw_input("Hash: ")
print ""
file = raw_input("Wordlist: ")
screen_clear()
print "Cracking the password for \"" + user + "\" using "
time1 = time.time() # Begins the 'Clock' for timing
realm = "Technicolor Gateway" # These 3 variables dont appear to change
qop = "auth"
uri = "/login.lp"
HA2 = md5("GET" + ":" + uri).hexdigest() # This hash doesn't contain any changing variables so doesn't need to be recalculated
file = open(file, 'r') # Opens the wordlist file
wordlist = file.readlines() # This enables us to use len()
length = len(wordlist)
screen_clear()
print "Cracking the password for \"" + user + "\" using " + str(length) + " words"
break_points = [] # List that will have start and stopping points
for i in range(cores): # Creates start and stopping points based on length of word list
break_points.append({"start":int(math.ceil((length+0.0)/cores * i)), "stop":int(math.ceil((length+0.0)/cores * (i + 1)))})
def pwd_find(start, stop, event):
for number in range(start, stop):
if not event.is_set():
word = (wordlist[number])
pwd = word.replace("\n","") # Removes newline character
HA1 = md5(user + ":" + realm + ":" + pwd).hexdigest()
hidepw = md5(HA1 + ":" + nonce +":" + "00000001" + ":" + "xyz" + ":" + qop + ":" + HA2).hexdigest()
if hidepw == hash:
screen_clear()
time2 = time.time() # stops the 'Clock'
timetotal = math.ceil(time2 - time1) # Calculates the time taken
print "\"" + pwd + "\"" + " = " + hidepw + " (in " + str(timetotal) + " seconds)"
print ""
event.set()
p.terminate
p.join
else:
p.terminate
p.join
if __name__ == '__main__': # Added this because the multiprocessor module sometimes acts funny without it.
p = Pool(cores) # Number of processes to create.
m = Manager()
event = m.Event()
for i in break_points: # Cycles though the breakpoints list created above.
i['event'] = event
a = p.apply_async(pwd_find, kwds=i, args=tuple()) # This will start the separate processes.
p.close() # Prevents any more processes being started
p.join() # Waits for worker process to end
if event.is_set():
end = raw_input("hit enter to exit")
file.close() # Closes the wordlist file
screen_clear()
exit()
else:
screen_clear()
time2 = time.time() # Stops the 'Clock'
totaltime = math.ceil(time2 - time1) # Calculates the time taken
print "Sorry your password was not found (in " + str(totaltime) + " seconds) out of " + str(length) + " words"
print ""
end = raw_input("hit enter to exit")
file.close() # Closes the wordlist file
screen_clear()
exit()
Edit (for #noxdafox):
def finisher(answer):
if answer:
p.terminate()
p.join()
end = raw_input("hit enter to exit")
file.close() # Closes the wordlist file
screen_clear()
exit()
def pwd_find(start, stop):
for number in range(start, stop):
word = (wordlist[number])
pwd = word.replace("\n","") # Removes newline character
HA1 = md5(user + ":" + realm + ":" + pwd).hexdigest()
hidepw = md5(HA1 + ":" + nonce +":" + "00000001" + ":" + "xyz" + ":" + qop + ":" + HA2).hexdigest()
if hidepw == hash:
screen_clear()
time2 = time.time() # stops the 'Clock'
timetotal = math.ceil(time2 - time1) # Calculates the time taken
print "\"" + pwd + "\"" + " = " + hidepw + " (in " + str(timetotal) + " seconds)"
print ""
return True
elif hidepw != hash:
return False
if __name__ == '__main__': # Added this because the multiprocessor module sometimes acts funny without it.
p = Pool(cores) # Number of processes to create.
for i in break_points: # Cycles though the breakpoints list created above.
a = p.apply_async(pwd_find, kwds=i, args=tuple(), callback=finisher) # This will start the separate processes.
p.close() # Prevents any more processes being started
p.join() # Waits for worker process to end
You can use the Pool primitives to solve your problem. You don't need to share an Event object which access is synchronised and slow.
Here I give an example on how to terminate a Pool given the desired result from a worker.
You can simply signal the Pool by returning a specific value and terminate the pool within a callback.
I think your hunch is correct. You are checking a synchronization primitive inside a fast loop. I would maybe only check if the event is set every so often. You can experiment to find the sweet spot where you check it enough to not do too much work but not so often that you slow the program down.
Related
I am working on a program that can search wikipedia given an input like: "Who is Elon Musk" - however, with his specific example (and others), when the variable is placed into wikipedia.summary(string), it removes the "l" in "Elon Musk". Attached is my function:
import wikipedia
import spacy
def search_wiki(self, search):
nlp = spacy.load('en_core_web_sm')
doc = nlp(search)
subject_phrase = get_subject_phrase(doc)
try:
results = wikipedia.page(str(subject_phrase)) #removes here
final = results
print(wikipedia.summary(final, sentences=1))
except wikipedia.exceptions.PageError:
results = wikipedia.search(str(subject_phrase))
print("\nDid you mean " + results[0] + "?\n")
possible = results[0]
if input() == "yes":
final = possible
print(final) #prints "Elon Musk"
print(wikipedia.summary(final, sentences=1)) #removes here too
else:
print("\nThese are the other searches that came up for " + str(subject_phrase) + ":\n")
for r in results:
print(r)
print("\nPlease type the result you want me to search for:\n")
final = input()
print(wikipedia.summary(final, sentences=1))
except wikipedia.exceptions.DisambiguationError as e:
print("I couldn't find anything for " + str(subject_phrase) + ". Here are some related results:")
for o in e.options:
print(o)
print("\nWhich of these did you mean?")
final = input()
print(wikipedia.summary(final, sentences=2))
print("\nWould you like to hear more about " + final + "?\n")
if input() == "yes":
print("\nYou can read more about " + final + " at " + wikipedia.page(final).url + "\n")
entry = wikipedia.summary(final, sentences=3)
return lambda: entry, True
else:
entry = wikipedia.summary(final, sentences=1)
return lambda: entry, False`
I am pretty stuck, I have the variable printing right before the call to wikipedia.summary(string), so I know that the variable is not changing on my end (or at least I think).
hey i've been working on a timer using python time class. the time should start after the user presses F7 but for some reason it starts counting as soon as I run the program.
from pynput import keyboard
from time import perf_counter
from time_sig import signature
print("Welcome to your smart timer")
datafile = open('data', 'a')
ex_name = input("Please type in the exercise type: ")
datafile.write("\nStarted exercise: " + ex_name.upper())
print("Click F7 to start, F8 every time you finished a question and F9 to stop")
count = 0
def execute_F7():
print("starting timer")
t = perf_counter()
def execute_F8():
print("finished question")
tic = perf_counter
transfer = signature(tic)
global count
count += 1
datafile.write("\nfinished question number " + str(count) + " on time: " + str(signature.convert(transfer)) + " in (" + str(signature.q_time(transfer)) + ")")
print(str(signature.convert(transfer)))
def execute_F4():
tic = perf_counter
transfer = signature(tic)
print(str(signature.convert(transfer)))
datafile.write("\nfinished question number " + str(count) + " on time: " + str(signature.convert(transfer)) + " in (" + str(signature.q_time(transfer)) + ")")
datafile.write("\nWorkout done!")
print("Workout done!")
exit()
def on_press(key):
if key == keyboard.Key.f7:
execute_F7()
elif key == keyboard.Key.f8:
execute_F8()
elif key == keyboard.Key.f4:
execute_F4()
with keyboard.Listener(on_press=on_press) as listener:
listener.join()
love any help thanks!
The perf_counter is always running in the background; its absolute value is undefined, but based on what you're seeing I expect it starts at 0. To time a particular event, you need to subtract perf_counter() from an earlier reference value. Something like:
from time import perf_counter
from typing import Optional
class Stopwatch:
"""Measures elapsed time in seconds."""
def __init__(self):
self._start: Optional[float] = None
def start(self) -> None:
"""Starts the stopwatch."""
self._start = perf_counter()
def stop(self) -> float:
"""Stops the stopwatch, returning the elapsed time."""
assert self._start is not None, "Missing call to start()"
elapsed = perf_counter() - self._start
self._start = None
return elapsed
Could someone help with my current code. I would like to add task numbers to my tasks that get saved in my output text document. I would need to loop it so each task will be assigned the next task number. If possible I would like to be able to call on these task numbers later.
My code so far is:
def add_task():
if menu == "a" or menu == "A":
with open( 'user.txt' ) as fin :
usernames = [i.split(',')[0] for i in fin.readlines() if len(i) > 3]
task = input ("Please enter the username of the person the task is assigned to.\n")
while task not in usernames :
task = input("Username not registered. Please enter a valid username.\n")
else:
task_title = input("Please enter the title of the task.\n")
task_description = input("Please enter the task description.\n")
task_due = input("Please input the due date of the task. (yyyy-mm-dd)\n")
date = datetime.date.today()
task_completed = False
if task_completed == False:
task_completed = "No"
else:
task_completed = ("Yes")
with open('tasks.txt', 'a') as task1:
task1.write("\nUser assigned to task:\n" + task + "\nTask Title :" + "\n" + task_title + "\n" + "Task Description:\n" + task_description + "\n" + "Task Due Date:\n" + task_due + "\n" + "Date Assigned:\n" + str(date) + "\n" + "Task Completed:\n" + task_completed + "\n")
print("The new assigned task has been saved")
add_task()
Firstly, I don't really want to go into detail but the way you are storing your output is highly inefficient and difficult to access the larger your text file gets. Why not use some free DB system to store your data.
Secondly. Assuming that you want to write many tasks at a time but only "save" once so to speak, consider using a dict of dicts.
def write_task_to_txt(task):
### break down the dict to from your lines to write to your text
def add_task(task_list,task_id):
new_tasks[task_id] = {}
new_tasks[task_id]["username"] = "username"
new_tasks[task_id]["title"] = "Task 1"
### How you fill up a task
return task_list
new_tasks = {}
for i in range(10):
new_tasks = add_task(new_tasks,i+1)
write_task_to_txt(new_tasks)
With this you can always access the task using new_tasks[task_id] to pull all the data of that task. Note the for loop is using an iterator. If you want to avoid this you could use a global and a while loop instead. BUT if you want to do that, i recommend converting your application into a class and use class variables instead.
Here is a skeleton of how I would try that:
class yourclass():
def __init__(self):
self.task_num = 1 #use 1 if no values
self.tasks_towrite = {}
self.mode_select()
def mode_select(self):
self.menu = input("choose mode")
while(1):
if self.menu == "a" or self.menu == "A":
self.add_task()
if self.menu == "s".casefold() #Cool function that does the same as your menu thingy
self.write_to_text()
else:
print("exit")
self.close_program()
def close_program(self): # Exit function
print("exiting")
sys.exit(1)
def add_task(self): #Add task
with open( 'user.txt' ) as fin :
self.usernames = [i.split(',')[0] for i in fin.readlines() if len(i) > 3]
task = input ("Please enter the username of the person the task is assigned to.\n")
while task not in self.usernames :
task = input("Username not registered. Please enter a valid username.\n")
else:
new_task = {}
new_task["username"] = task
new_task["title"] = input("Please enter the title of the task.\n")
new_task["description"] = input("Please enter the task description.\n")
new_task["due"] = input("Please input the due date of the task. (yyyy-mm-dd)\n")
date = datetime.date.today()
task_completed = False
if task_completed == False:
new_task["completed"] = "No"
else:
new_task["completed"] = "Yes"
new_task["assigned"] = date
self.tasks_towrite[self.task_num] = new_task
sefl.task_num +=1 #New test number
return None #returns to mode_select
def write_to_text(self):
with open('tasks.txt', 'a') as task1:
for i in self.tasks_towrite:
task1.write(str(i) + "\n") #Writes it all at once You can reformat however you want
self.tasks_towrite = {}
print("The new assigned tasks has been saved")
return None #returns to menu or you could go to self.close_program to exit
if __name__== '__main__':
x = yourclass()
I am trying to access a variable within a function in a class and print it. Whenever I try I keep getting the error: AttributeError: 'NoneType' object has no attribute 'job_ID'.
def driver():
q = my_queue.Queue_()
for line in df:
if 'received' in line:
q.enqueue(line)
print("Adding job " + q.new_item.job_ID + " to the queue with the timestamp: " + q.new_item.time_stamp + ".")
print("The prority of the job is: " + q.new_item.job_priority)
print("The job type is: " + q.new_item.job_type)
if 'respond' in line:
q.dequeue()
print("Completed job " + q.current.job_ID + " in " + str(int(q.time_elapsed)) + " seconds.")
if 'active' in line:
q.active_jobs()
print("Total number of jobs: " + str(len(q.temp)))
print("Average priority: " + str(q.average))
if 'modify' in line:
q.modify(line)
print("Modified job " + q.current.job_ID)
The error is coming from the last print statement in this code.
This is the function within the class that is being used here:
def modify(self, x): # need to fix printing bug
self.current = self.head
while self.current != None:
if x[1] in self.current.get_data():
self.current.data[2] = x[2]
self.current.data[3] = x[3]
break
# print("Modified job " + current.job_ID)
else:
# print('The job details cannot be modified.')
pass
self.current = self.current.get_next()
The exit condition for the loop in the modify function that you have provided is self.current == None.
When you call modify() in this last conditional statement:
if 'modify' in line:
q.modify(line) // here
print("Modified job " + q.current.job_ID)
You are making q.current evaluate to None. Therefore, the reason why you are getting an AttributeError is because q.current is None, which has no such attribute called job_ID.
To fix your problem, you must ensure that q.current is not None before printing q.current.job_ID. I can't give you any help beyond this, since I don't know what the purpose of your program is.
I currently have the following code: You enter a string, the computer then pulls random letters and tries to match it to the letters in your string. This repeates and with each iteration the computer gets closer to guessing your string. I would like to output the initial string entered or the 'target' and the string format of the number of iterations it took to get the correct match. I want to output this to a text document. So far the script produces a text document but does not output to it. I would like it to save the data after each iteration from the main loop. I have the working program i just need assitance with the output, any ideas on how that could be done?
Here is the progress i made:
import string
import random
possibleCharacters = string.ascii_lowercase + string.digits + string.ascii_uppercase + ' .,!?;:£$^%&*|'
file = open('out.txt', 'w')
again = 'Y'
while again == 'Y' or again == 'y':
target = input("Enter your target text: ")
attemptThis = ''.join(random.choice(possibleCharacters) for i in range(len(target)))
attemptNext = ''
completed = False
generation = 0
while completed == False:
print(attemptThis)
attemptNext = ''
completed = True
for i in range(len(target)):
if attemptThis[i] != target[i]:
completed = False
attemptNext += random.choice(possibleCharacters)
else:
attemptNext += target[i]
generation += 1
attemptThis = attemptNext
genstr = str(generation)
print("Target matched! That took " + genstr + " generation(s)")
file.write(target)
file.write(genstr)
again = input("please enter Y to try again: ")
file.close()
Addressing both the original question and the one in the comments:
How to write to file after each iteration of the loop: call file.flush() after file.write(...) :
file.write(target)
file.write(genstr)
file.flush() # flushes the output buffer to the file
To add a newline after each "target" and "genstring" that you write, well, add a newline to the string (or whatever other output formatting you want) :)
file.write(target + '\n')
file.write(genstr + '\n')