I built a code in python with multi classes and threads, after I start the code and because of While True loop, the Code keeps running and writing in each step a Report. I need to link an outside function to the running Code to print the report.
example :
Running code:
import threading as TH
def WriteTXT():
file = open('File.TXT','a')
file.write('Test_WriteTXT')
file.close()
runLoop()
def runLoop():
th.start()
th = TH.thread(target = WriteTXT)
th.start()
Outside function :
def report():
file = open('File.TXT','w')
Txt_file = file.read()
print(Txt_file)
How to call report function and link it to the Running code to print Txt_file?
All you would do is call runLoop(), WriteTXT(), and report() functions at the end of the code. You also have to import threading "as" TH instead of "ad", and add the missing colon (:) when defining the WriteTxt function. The new code might look like this below:
import threading as TH
def WriteTXT():
file = open('File.TXT','a')
file.write('Test_WriteTXT')
file.close()
runLoop()
def runLoop():
th.start()
def report():
file = open('File.TXT','w')
Txt_file = file.read()
print(Txt_file)
WriteTxt()
th = TH.thred(target = WriteTXT)
th.start()
runLoop()
Related
I am use threading to do parallel download now i have a url_list img_list i want to download it in 2 thread so def two download.
i put half into download1 and half in download2 so it will speed up to complete, but finally when i run the scripts my download still in serial,i don't know why, how can i modify my script?
here's the code:
import requests,threading
img_list=[...]
num=len(img_list)
def download_1(img_list):
n=0
for i in img_list:
n+=1
with open('./img/'+str(n)+'.jpg','wb')as f:
f.write(requests.get(i).content)
print(str(n)+"download1 complete")
def download_2(img_list):
n=len(img_list)
for i in img_list:
n+=1
with open('./img/'+str(n)+'.jpg','wb')as f:
f.write(requests.get(i).content)
print(str(n)+"download2 complete")
thread_1 = threading.Thread(target=download_1(img_list[:int(num/2)]))
thread_2 = threading.Thread(target=download_2(img_list[int(num/2):]))
thread_1.start()
thread_2.start()
In this line
threading.Thread(target=download_1(img_list[:int(num/2)]))
you call download_1(...) and pass the result (null) to thread. That's why it runs serially. Instead you want to pass download_1 function itself (not the result of calling it) to the thread. Like this:
threading.Thread(target=download_1, args=(img_list[:int(num/2)],))
Do it in both places.
Side note: you should t.join() both threads at the end.
You are calling both functions at the time of creating threads. So, threads are passed null and, therefore, do nothing. You should change the code like this:
thread_1 = threading.Thread(target=download_1, args=(img_list[:int(num/2)]))
thread_2 = threading.Thread(target=download_2, args=(img_list[int(num/2):]))
thread_1.start()
thread_2.start()
I have two classes in Python script. One of is Main() and the second is Loading()
class Main:
pass
class Loading:
pass
At first, works Main() that return was filled dictionary
Then is created instance of Loading() that iterates all images and downloads them:
## LOAD IMAGES ##
imageLoader = Loading()
imageLoader.save()
So, the problem is when I call this script it creates one major thread that is waiting for the end of imageLoader = Loading().
As a result, a major thread works so long, it invokes 502 Server error.
How to run imageLoader = Loading() in a separate background thread that to release major thread?
What will be launched first in this code:
LOADED_IMAGES = {}
IMAGES_ERRORS = []
IMAGES = {"A": "https://images.aif.ru/009/299/3378e1a1ab2d1c6e6be6d38253dd3632.jpg", "B": "http://static1.repo.aif.ru/1/77/623957/b99ee5f894f38261e4d3778350ffbaae.jpg"}
excel = Excel()
excel.readExcel(file_path, 'Main')
imageLoader = ImageLoader()
Thread(target=imageLoader.run().save()).start()
Does it work line by line or Thread will be created immediately?
**This is full code:**
class ImageLoader:
def run(self):
for article, image in IMAGES.items():
if image is None or image == '':
continue
LOADED_IMAGES[article] = self.loadImage(self.replaceHttpsProtocol(image), '/home/o/oliwin4/jara/public_html/image/catalog/s/')
def replaceHttpsProtocol(self, url):
return url.replace("https:", "http:")
def nameNameGenerate(self):
return int(round(time.time() * 1000))
def extention(self, path):
ext = path.split(".")[-1]
return '.' + ext if ext else "jpg"
def save(self):
for article, image in LOADED_IMAGES.items():
self.add(article, image)
def add(self, article, image):
Products.update(image=image).where(Products.sku == article).execute()
def loadImage(self, path, path_folder):
try:
filename = str(self.nameNameGenerate()) + str(self.extention(path))
wget.download(url=path, out=path_folder + filename)
return 'catalog/s/' + filename
except BaseException as e:
IMAGES_ERRORS.append(str(e))
Using:
def runOnThread():
imageLoader = ImageLoader()
imageLoader.run()
imageLoader.save()
if __name__ == "__main__":
Thread(target=runOnThread, daemon=True).start()
You need to look for which line is blocking your code to run it in a separated thread, usually the blocking line is some kind of I/O or expensive computation.
To do it you can use the threading module.
So, assuming that your blocking line is the
imageLoader.save()
Try to run it in a separated thread with this code.
from threading import Thread
Thread(target=imageLoader.save()).start()
As mentioned, you can use Python's threading module for this. Though, a thread takes a reference to a function (passing target a function call is useless / wrong).
In your case, if you want to both instantiate then run a function on an object in a separate thread, you should put these two in a function:
def runOnThread():
imageLoader = Loading()
imageLoader.save()
Then pass a reference of this function to a new thread, like so (notice no ()):
from threading import Thread
Thread(target=runOnThread).start()
If you don't want the main thread to wait for the new thread to finish, you could make it a Daemon thread, like so:
Thread(target=runOnThread, daemon=True).start()
Shorter version of all the above:
from threading import Thread
Thread(target=lambda: Loading().save(), daemon=True).start()
I have a program that's running a separate thread, which contains three execfile() statements running external Python scripts. Without changing these scripts, is there a way to have the print() statements within them print their commands out to a log file? From my code below, I need the print commands from within File1, File2, and File3 to go into a log file, without being able to change those files. Is this possible?
Code:
MyThread.py
import threading
class MyThread(threading.Thread):
def run(self):
execfile('File1.py')
execfile('File2.py')
execfile('File3.py')
Program.py
from MyThread import *
MyThread().start()
I've seen the Q/A posted here (redirect prints to log file) and tried this solution, but the print() statements from the external files aren't added to the log file:
import threading, sys
class MyThread(threading.Thread):
def run(self):
old_stdout = sys.stdout
output_file = open('output.log', 'w')
sys.stdout = output_file
execfile('File1.py')
execfile('File2.py')
execfile('File3.py')
sys.stdout = old_stdout
output_file.close()
OK, so this was...fun. What I did is I took the print->file method frm here and prepended it to the file I want to run.
my_thread.py
import threading
def prepend_stdout(filename):
with open(filename, 'r+') as f:
std_out = 'import sys\nold_stdout = sys.stdout\nlog_file = open("message.log","w")\nsys.stdout = log_file\n'
content = f.read()
return std_out + content
class MyThread(threading.Thread):
def run(self):
content = prepend_stdout("test.py")
exec(content)
MyThread().start()
test.py
print("whatever, man")
I then ran python my_thread.py and the output in "message.log" was
whatever, man
I tried the solution posted my Cory Madden, which worked until the exec() call (nothing printed after this, from print() calls within the thread itself). Then I went back to the suggested answer originally given (redirect prints to log file) and did a few things differently, solving the problem. By adding the line output_file.flush() after each execfile() statement, the print() commands from within the execfile() scripts now print out to the external log file. Thus:
...
def run(self):
old_stdout = sys.stdout
output_file = open('path/to/file.log', 'w')
sys.stdout = output_file
execfile('File1.py')
output_file.flush()
execfile('File2.py')
output_file.flush()
execfile('File3.py')
output_file.flush()
output_file.close()
sys.stdout = old_stdout
now works for my instance.
I have a few scripts that i want to run simultaneously, they read a CSV file, im trying the following;
import sys
import csv
out = open("C:\PYDUMP\PYDUMPINST.csv","r")
dataf=csv.reader(out)
for row in dataf:
take = row[0]
give = row[1]
def example():
try:
lfo = int(take)
if lfo > 0:
#code
except Exception, e:
pass
example()
This is saved as takefile1.py. I have 20 scripts with similar structures that i want to run simultaneously. So im using(which i have been using for running other batches of scripts trouble free) the following;
import csv
import sys
from threading import Thread
def firstsend_lot():
execfile("C:\Users\takefile1.py")
execfile("C:\Users\takefile2.py")
def secondsend_lot():
execfile("C:\Users\takefile3.py")
execfile("C:\Users\takefile4.py")
if __name__ == '__main__':
Thread(target = firstsend_lot).start()
Thread(target = secondsend_lot).start()
So i am getting the error "global name 'take' is not defined". Anyone got any suggestions? Im pretty hopeless at Python so pretend you are talking to an idiot.
Your function example(), do not have access to take. Try adding a line in it:
def example():
global take
try:
lfo = int(take)
if lfo > 0:
#code
except Exception, e:
pass
I have a loop in Tkinter:
def main():
#Global Variables
windows = []
buttons = []
labels = []
messageboxes = []
global theme
theme = 0
listboxes = []
global register
register = []
global path
path = ""
# Lotsa' Code
Tkinter.mainloop()
if __name__ == "__main__":
main()
def save_f():
global register
outFile = open('FobbySave.txt', 'wb')
pickle.dump(register, outFile)
outFile.close()
global register
#At Quit
atexit.register(save_f)
atexit fails. But when I try to print register it has no problem. save_f worked when I put it in the Tkinter loop, but atexit didn't. So can somebody tell me what am I doing wrong?
P.S.
Sorry forgot to write atexit the first time. But it's in my code.
Edit: Orginal code down here
import pickle
import atexit
def save_f():
global register
outFile = open('Something.txt', 'wb')
pickle.dump(register, outFile)
outFile.close()
atexit.register(save_f)
OK turns out the problem was that I needed atexit.register(save_f) instead of atexit.register(save_f()).
You're not supposed to make a function call!
Looking at your code I would suggest to try this instead:
def main():
# ... everything in main ...
Tkinter.mainloop()
def save_f():
outFile = open('FobbySave.txt', 'wb')
pickle.dump(register, outFile)
outFile.close()
#At Quit
atexit.register(save_f)
if __name__ == "__main__":
main()
The problem might have been that you initialize your atexit after you run the main method. So after the code gets killed (and stops executing) you try to add the atexit method.
Your basic script works for me, provided I import atexit and set register to something. e.g.:
import pickle
import atexit
def save_f():
outFile = open('Something.txt', 'wb')
pickle.dump(register, outFile)
outFile.close()
register = 1
atexit.register(save_f)
(note that global isn't necessary either). In cases such as this, you should make sure that you don't have another file named atexit.py in the current directory or somewhere else on your PYTHONPATH ...