We Keep Coding

Optimize the process of importing multiple images from the internet, tkinter

I'm writing a recipe program and as part of it, I want to make a series of buttons with different images (for each recipe). I am using the Edamam recipe search api (here) which gives me urls to all the images I need.
The way I my code operates is as follows:
for each recipe:
data = urlopen(image_url)
image = ImageTk.PhotoImage(data=data.read()) # Convert to a tkinter PhotoImage object
b = Button(root, command = lambda i=recipe_url:callback(i), image = image) # callback() opens the webpage for a given url
b.image = image # To prevent garbage collection from removing the image
b.pack()
Obviously this is a simplified version and I've removed or renamed things for it to make sense without looking at the entire program. However, it addresses the performance issue which is the process of downloading and converting images.
The relevant information I can think of is:
20 images need to be converted each time
It takes around 18 seconds for the entire process to run
I am using the smallest sized images available (sizes of large, medium, small, and thumbnail are available)
The relevant modules I'm using are PIL, urllib, and tkinter
My question is how can I increase the speed of this component without compromising on functionality.
Let me know if there's any other information I need to add and thank you in advance for the help.

Thanks to #Alexander for the tip off on multiprocessing, I tried implementing it but I couldn't really wrap my head around it, so I used multithreading instead and now it is far more efficient. Each thread contains the code for one button and then the loop simply defines and then starts the threads:
x = threading.Thread(target=image_conversion, args=(item,))
x.start()

OpenCv Video Display PyQt5

I'm working with opencv3, python 3 and pyqt5. I want to make a simple GUI in which I want open up a new window to play a video along with some other widgets when a button is clicked on the main window. I've used QPixmap for displaying images in the past so I create a label and try to set the frames in the pixmap in a loop. The loop works fine but I am unable to get a display of the video/new window.
The loop I want to execute in the new window looks something like this:
def setupUi():
vid=cv2.VideoCapture('file')
ret, frame=vid.read()
while ret:
Qimg=convert(frame)
self.label.setpixmap(Qimg)
self.label.update()
ret,frame=vid.read()
convert() is a function I've written myself that converts the cv frame to QImage type to be set into the pixmap.
I'm only a beginner with pyQt so don't know what I am doing wrong. I've read about using signals, threads for the new window and QtApplication.processEvents() but don't know how these work and how they'll fit into my problem.
It would be helpful if someone could set me in the right direction and also point out some resources to create good interfaces for my apps using OpenCV and python.

The reason that this isn't running is that your while loop is blocking Qt's event loop. Basically, you're stuck in the while loop and you never give control back to Qt to redraw the screen.
Your update() call isn't doing what you think it is; it's updating the data stored by the object, but this change does not show up until the program reenters the eventloop.
There are probably multiple ways of handling this, but I see two good options, the first being easier to implement:
1) Call QApplication.processEvents() in every iteration of your while loop. This forces Qt to update the GUI. This will be much more simple to implement than 2).
2) Move the function to a separate class and use QThread combined with moveToThread() to update the data, and communicate with the GUI thread using signals/slots. This will require restructuring your code a bit, but this might be good for your code overall. Right now the code that is generating the data is in your MainWindow class presumably, while the two should be kept separate according to Qt's Model-View design pattern. Not very important for a small one-off app, but will help keep your code base intelligible as your app grows in size.

More Efficient Alternatives to plt.draw() in MatPlotLib

I'm trying to make an interactive GUI using MatPlotLib, and the thing users notice most is the latency between, say, changing a slider, and the GUI responding.
Currently, I am using plt.draw() at the end of every event, which works well, except for the fact that it causes 256ms of the function's 259ms runtime.
In researching alternatives to plt.draw(), I came across this post: why is plotting with Matplotlib so slow? , in which it was recommended to use fig.canvas.blit(ax1.bbox) as an alternative which reloads just the graph, and not the entire figure.
When I use this, the event handler runs in 3ms, however the GUI updates during the following event, rather than at the end of the current event as if the event was caught in a buffer: User input 1>pause>User input 2>GUI responds to input 1>pause>User input 3>GUI responds to input 2, making the program unusable.
So what alternatives do I have? Fixing either one of these problems would allow me to run a very fast GUI. I'll post code if you need it.
Note 1: When using fig.canvas.blit(ax1.bbox), the sliders, buttons, etc will change immediately, and only the subplots will behave as described above.
Note 2: fig.canvas.draw_idle(), as used in the widgets example here http://matplotlib.org/examples/widgets/slider_demo.html, and fig.canvas.draw(), perform identically to plt.draw()

Python PyQt Progress Bar Busy

I have found several questions that looks similar to this such as:
Busy indication with PyQt progress bar
Progress bar with pyqt
Display installation progress in PyQt
But most of the examples and answers were on 1 GUI thread and 1 data processing thread.
Let me explain my scenario.
I have 3 classes:
class MainWindow (Inherited from QMainWindow class and act as main GUI)
class LogInWidget (Inherited from QWidget class.This class consists of all the username/password field + a progress bar, pls see the image below)
3.class DataTableWidget (Inherited from QWidget class.consist of a table that display a few hundreds thousands of data)
The program is designed so that once a user logged in to the program via LogInWidget, he will be able to see all the data displayed by the DataTableWidget.
I use .setCentralWidget method to set both LogInWidget and DataTableWidget.(ie.if the user password is correct, i use
table = DataTableWidget ()
self.setCentralWidget(table)
to display the table.
My problem is: When user logged in, there is a big GUI freeze before he can see the data table as there are so many data entry to display via the table.
So I would like to show a busy progress bar (on my LogInWidget) before the program finish loading the table.
I thought of using Qthread to accomplish it but since both tasks, displaying progress bar and creating DataTableWidget and setting as centralwidget is GUI tasks, how should i approach it?

You would have to connect your progress bar value to a signal from the DataTableWidget. But this in turn assumes that the DataTableWidget generates this signal many times while rendering it, which is not likely. Perhaps all you need is a way to show that the GUI is busy, like bar with a line on it that goes back and forth between the two ends of the bar. Then you don't need to exchange data between the the two widgets.

There is no straight forward way to do it. However, here is an idea:
As #schollii says, you need a signal to be emitted by DataTableWidget while it is rendering. Such a signal does not exist. And even if it did, it wouldn't be processed until the widget was rendered
However, if DataTableWidget has a model, you could override the data() method with your own which just calls the data() method of the base class and also updates a progress bar (though only update the progress bar when first rendering the table, not during subsequent calls to data()). As the view renders, the view will be calling the data() method many times. This way you get access to points in time during the rendering process.
Of course the progress bar will not redraw until control returns to the event loop (the rendering finishes), but you could probably work around that by calling QApplication.processEvents().
Note that this is quite hacky, in general bad practice, and completely destroys the model-view paradigm. I don't recommend doing it. But I can't think of anyway to achieve the same outcome. Perhaps you could just speed up the rendering of the DataTableWidget. For instance, QTableView.resizeColumnsToContents() is known to be slow. Are you calling that anywhere?

Python Tkinter Label redrawing every 10 seconds

I have this following Python Tkinter code which redraw the label every 10 second. My question is , to me it seems like it is drawing the new label over and over again over the old label. So, eventually, after a few hours there will be hundreds of drawing overlapping (at least from what i understand). Will this use more memory or cause problem?
import Tkinter as tk
import threading
def Draw():
frame=tk.Frame(root,width=100,height=100,relief='solid',bd=1)
frame.place(x=10,y=10)
text=tk.Label(frame,text='HELLO')
text.pack()
def Refresher():
print 'refreshing'
Draw()
threading.Timer(10, Refresher).start()
root=tk.Tk()
Refresher()
root.mainloop()
Here in my example, i am just using a single label.I am aware that i can use textvariable to update the text of the label or even text.config. But what am actually doing is to refresh a grid of label(like a table)+buttons and stuffs to match with the latest data available.
From my beginner understanding, if i wrote this Draw() function as class, i can destroy the frame by using frame.destroy whenever i execute Refresher function. But the code i currently have is written in functions without class ( i don't wish to rewrite the whole code into class).
The other option i can think of is to declare frame in the Draw() as global and use frame.destroy() ( which i reluctant to do as this could cause name conflict if i have too many frames (which i do))
If overdrawing over the old drawing doesn't cause any problem (except that i can't see the old drawing), i can live with that.
These are all just my beginner thoughts. Should i destroy the frame before redraw the updated one? if so, in what way should i destroy it if the code structure is just like in my sample code? Or overdrawing the old label is fine?
EDIT
Someone mentioned that python tkinter is not thread safe and my code will likely to fail randomly.
I actually took this link as a reference to use threading as my solution and i didn't find anything about thread safety in that post.
I am wondering what are the general cases that i should not use threading and what are the general cases i could use threading?

The correct way to run a function or update a label in tkinter is to use the after method. This puts an event on the event queue to be executed at some time in the future. If you have a function that does some work, then puts itself back on the event queue, you have created something that will run forever.
Here's a quick example based on your example:
import Tkinter as tk
import time
def Draw():
global text
frame=tk.Frame(root,width=100,height=100,relief='solid',bd=1)
frame.place(x=10,y=10)
text=tk.Label(frame,text='HELLO')
text.pack()
def Refresher():
global text
text.configure(text=time.asctime())
root.after(1000, Refresher) # every second...
root=tk.Tk()
Draw()
Refresher()
root.mainloop()
There are a lot of things I would change about that program from a coding style point of view, but I wanted to keep it as close to your original question as possible. The point is, you can use after to call a function that updates the label without having to create new labels. Plus, that function can arrange for itself to be called again at some interval. In this example I picked one second just so that the effect is easier to see.
You also asked "I am wondering what are the general cases that i should not use threading and what are the general cases i could use threading?"
To put a blunt point on it, you should never use threading if you have to ask a question about when to use threading. Threading is an advanced technique, it is complicated, and it easy to get wrong. It's quite possible for threading to make your program slower rather than faster. It has subtle consequences, such as your programs failing mysteriously if you do things that aren't thread safe.
To be more specific to your situation: you should avoid using threads with tkinter. You can use them, but you can't access widgets from these other threads. If you need to do something with a widget, you must put an instruction in a thread-safe queue, and then in the main thread you need to periodically check that queue to see if there's an instruction to be processed. There are examples of that on this website if you search for them.
If all that sounds complicated, it is. For most of the GUIs you write, you won't need to worry about that. If you can take large processes and break them down into chunks that execute in 100 ms or less, you only need after, and never need threads.

To allow the cleanup with minimal code changes, you could pass previous frames explicitly:
import Tkinter as tk
def Draw(oldframe=None):
frame = tk.Frame(root,width=100,height=100,relief='solid',bd=1)
frame.place(x=10,y=10)
tk.Label(frame, text='HELLO').pack()
frame.pack()
if oldframe is not None:
oldframe.destroy() # cleanup
return frame
def Refresher(frame=None):
print 'refreshing'
frame = Draw(frame)
frame.after(10000, Refresher, frame) # refresh in 10 seconds
root = tk.Tk()
Refresher()
root.mainloop()