We Keep Coding

Understanding a key difference between multiprocessing and threading in Python

I have written a program to use threads and I create an instance of a custom object with a run(p, q) method. I pass this run() method as the target for the thread, like this:
class MyClass(object):
def run(p, q):
# code here
obj = MyClass()
thrd = threading.Thread(target=obj.run, args=(a, b))
My thread starts by executing the run() method with the passed arguments, a and b. In my case, one of them is an Event that is eventually used to stop the thread. Also, the run() method has access to all the object's instance variables, which include other objects.
As I understand it, this works because a thread shares memory with the program that creates it.
My question is how this differs with multiprocessing. e.g.
proc = multiprocessing.Process(target=obj.run, args=(a, b))
I believe a process does not share memory, so am I able to do the same? How is the process able to access the whole obj object when it is just being given a reference to one method? Can I pass an Event? If the created process gets a copy of the whole creating program's memory, what happens with things like open database connections? How does it connect with the original Event?
And a final question (thanks for bearing with me). Is it necessary that the whole program is duplicated (with all its imported modules etc.) in a created process? What if I want a minimal process that doesn't need as much as the main program?
Happy to receive any hints at answers, or a pointer to somewhere that describes multiprocessing in this amount of detail.
Thanks so much.
Julian

each process has its own memory map! two process don't share thier memory each other.
Inside a process thread are declared inside the memory of the process they come from.
it doesn't, "obj.run" 's code is copied from this process to the new spawned.

Can global variables be accessed from a thread within a new process?

In the script's main thread, I setup a variable called queue and fill it with URLs. Then I create 8 processes using multiprocessing.Process and then those processes spawn 10 threads each using the threading library.
Within the thread worker (which was spawned by another process as per above), I have global queue.
Then will queue.get() act as expected? I've tried it and it seems to be okay during some tests and others not.
Question is, can global variables be accessed from another process and thread?

It is hard to understand what you are exactly asking. But there are two main questions here:
Can global variables be accessed from another process?
No, not without some form of inter-process communication, and even then you would be passing a copy of that variable to the other process. Each process has its own global state.
Can global variables be accessed from another thread?
Yes, a thread who lives in the same process can access a global variable, but you must ensure the safety of any memory that is accessed by multiple threads. Meaning, threads should not access writable memory at the same time as other threads or you run the risk of one thread writing to the memory while another thread attempts to read it.
Answering Question Above
If I understand the setup correctly, each of your child processes has their own global variable queue. Each of those queues should be accessible only to the threads that are spawned within that process.

Python multiprocessing avoid global variables

My handbrake compressor uses a multiprocessing queue to store movies to compress. In my main class I call a process which gets an queue element if exists. These process spawn another process which calls then a subprocess that calls handbrake. After that in the first process a second process will be called which reads the actual stdout of handbrake. That works very well.
Now I want to write the state (processing, waiting, finished) and the progress to a global variable. I need this, because later, I will implement an xml-rpc interface. Over this interface it should be possible to get the status of each compress thread.
Global variables are bad style, and I read, that there is something I do wrong, when I have to use it. But I don't know how I can make it right?
I have also a sqlite3 db, where I can store the progress. If I implement a web gui later I don't like to call the db file directly, I want my application to do this.

Does python identify critical section automatically?

I am new to Python multithreading coding. I consult the manual for Lock Object usage and find the normal case is
g_mutex = Lock()
g_mutex.acquire()
#some code
g_mutex.release()
But the lock does not specify what variable or function it is going to lock? So does python automatically find all critical variables to be locked? What if I call some function to modify some variables?

The purpose of a Lock is that at most one thread can hold it at any given time. If you acquire the lock, you can be sure no other process holds it. It's up to you to define the semantics of the lock -- if you want to use it to guard the access to some variable, just do so: acquire the lock before messing around with that variable, release it when you are done. There is no need for an explicit relation between the lock object and the variable it protects -- this is defined by the way you use it. (Also note that there is nothing Python-specific to this concept.)

Lock will not found variables or functions to lock. Lock is a mecanism that you can use to do so. For example, if you want to protect the variable foo of any modification, you must:
create a lock for that variable
acquire before any modification or usage in your whole code
release after

What variables are shared and not between processes in python?

GLOBAL_VAR = 1
class Worker:
class_var = 2
Worker's instances are created by multiple processes. Will they have their own copies of above variables? If not, can I use them to safely lock access to a resource accessed by multiple class instances (creating and accessing them in a thread-safe manner, of course)? I want to do it transparently for a class user. What is the right way to do this?

Are you talking about multithreading or multiprocessing? These are very different in Python.
Threads can access variables like you are use to in Python, without restriction.
Process on the other hand can't access variables from another process (aside some exception of shared variables). Process will copy the current state of the local variables on creation, but it will only be a copy.