I have an application I'm working on in Python 2.7 which has several classes that need to interact with each other before returning everything back to the main program for output.
A brief example of the code would be:
class foo_network():
"""Handles all network functionality"""
def net_connect(self, network):
"""Connects to the network destination"""
pass
class foo_fileparsing():
"""Handles all sanitation, formatting, and parsing on received file"""
def file_check(self, file):
"""checks file for potential problems"""
pass
Currently I have a main file/function which instantiates all the classes and then handles passing data back and forth, as necessary, between them and their methods. However this seems a bit clumsy.
As such I'm wondering two things:
What would be the most 'Pythonic' way to handle this?
What is the best way to handle this for performance and memory usage?
I'm wondering if I should just instantiate objects of one class inside another (from the example, say, creating a foo_fileparsing object within the foo_network class if that is the only class which will be calling it, rather than my current approach of returning everything to the main function and passing it between objects that way.
Unfortunately I can't find a PEP or other resource that seems to address this type of situation.
You can use modules. And have every class in one module.
and then you can use import to import only a particular method from that class.
And all you need to do for that is create a directory with the name same as you class name and put a __init__.py file in that directory which tells python to consider that directory as a module.
Then for example the foo_network folder contains a file named foo_network.py and a file __init__.py and in foo_network.py the code is
class foo_network():
"""Handles all network functionality"""
def net_connect(self, network):
"""Connects to the network destination"""
pass
and in any other file you can simply use
import net_connect from foo_network
it will only import that particular method. This way your code will not look messy and you can will be importing only what is required.
You can also do
from foo_network import *
to import all methods at once.
Related
I'm trying to implement a facade pattern to access two different file types, but I keep on running into circular dependencies. Is this common and if so what is the standard way to avoid it?
I have two file-types (A and B) in separate modules which are accessed by a facade in another separate interface.py module. The facade module needs to import the FileType classes from each module in order to return the objects, and also implements a method determine_file_type(path) and a custom error class IncorrectFileType.
I now wish to add an add_data_from_another_file method to FileTypeA. The first thing it needs to do is determine what type of file it's adding data from, but it can't access the interface.determine_file_type method without creating a circular dependency. I'm also unable to raise the IncorrectFileType error from within either file_type_a,b module for the same reason.
## Module: interface.py
from file_type_a import FileTypeA
from file_type_b import FileTypeB
def get_file(path):
if determine_type(path) == "typeA":
return FileTypeA(path)
elif determine_type(path) == "typeB":
return FileTypeB(path)
def determine_file_type(path):
...
return file_type
class IncorrectFileTypeError(ValueError):
pass
## Module: file_type_a.py
class FileTypeA():
def add_data_from_another_file(self, path):
file_type = determine_file_type(path) ### CAN'T IMPORT THIS
if file_type == "typeB":
raise IncorrectFileTypeError() ### CAN'T IMPORT THIS
## Module: file_type_b.py
class FileTypeB():
pass
One solution could be to implement the determine_file_type as a staticmethod on a AbstractFileType class but this doesn't help me if I need to raise an exception in one of the concrete classes. (It also feels like it might be messy in my real example, but that's probably a separate issue!)
This feels like a classic use of the Facade pattern, so what is the obvious trick I'm missing here?
I have a program written in my python using the PyPDF2 package to scrape a batch of pdf files. These PDF's aren't in the greatest shape so in order for my program to run, I need to modify the pdf.py file located within the package library as recommended by this website:
https://cheonhyangzhang.wordpress.com/2015/03/31/python-pdffilereader-pdfreaderror-eof-marker-not-found/
Is there a way I can implement this change to the file while keeping the original file intact? I've tried creating a child class of PdfFileReader class and modifying the 'read' method as prescribed by my link above, however, I've found that that leads to several import dependency issues that I'd like to avoid.
Is there an easier way to do this?
I would recommend to copy the pdf.py file into our script directory and rename it to mypdf.py. You can then modify the copy as you please without affecting the original. You can import the package using
import mypdf
I have done something similar for shutil.py as the default buffer size is too small in Windows for transferring large files.
You can add (or redefine) a method of class using setattr() like this (where the class has been defined inline rather than being imported only for purposes of illustration):
class Class(object):
pass
def func(self, some_other_argument):
return some_other_argument
setattr(Class, 'func', func)
if __name__ == '__main__':
c = Class()
print(c.func(42)) # -> 42
I have a situation where there's a complex object that can be referenced by unique name like package.subpackage.MYOBJECT. While it's possible to pickle this object using standard pickle algorithm, resulting data string will be very big.
I'm looking for some way to get same pickling semantic for an object that is already here for classes and functions: Python's pickle just dumps their fully qualified names, not code. This way just string like package.subpackage.MYOBJECT will be dumped and upon unpickling object will be imported, just like it happens for functions or classes.
It seems that this task boils down to making object aware of variable name it's bound to, but I have no clues how to do it.
Here's short example to explain myself clearly (obvious imports are skipped).
File bigpackage/bigclasses/models.py:
class SomeInterface():
__meta__ = ABCMeta
#abstractmethod
def operation():
pass
class ImplementationA(SomeInterface):
def operation():
print "ImplementationA"
class ImplementationB(SomeInterface):
def operation():
print "ImplementationB"
IMPL_A = ImplementationA()
IMPL_B = ImplementationB()
File bigpackage/bigclasses/tasks.py:
#celery.task
def background_task(impl, somearg):
assert isinstance(impl, SomeInterface)
impl.operation()
print somearg
File bigpackage/bigclasses/work.py:
from bigpackage.bigclasses.models import IMPL_A, IMPL_B
from bigpackage.bigclasses.tasks import background_task
background_task.submit(IMPL_A, "arg1")
background_task.submit(IMPL_B, "arg2")
Here I have trivial background Celery task that accept one of two available implementations of SomeInterface as an argument. Task's arguments are pickled by Celery, passed to a queue and executed on some worker server, that runs exactly the same code base. My idea is to avoid deep pickling of IMPL_A and IMPL_B and instead pass them as bigpackage.bigclasses.models.IMPL_A and bigpackage.bigclasses.models.IMPL_B correspondingly. That will help with performance and total traffic for queue server and also provide some safety against changes in IMPL_A and IMPL_B that will make them non-pickleable (for example lambda anywhere in object attributes hierarchy).
I'm kinda new to Python, so i'm still lost in the whole namespace thing.
I've created a package, with the init file in it and also a classname.py file, with the class, obviously.
For instance:
from parachute import container, emitter
I tried to instance the class Container directly, but it gave me an error, so i had to instance it as container.Container(). How can i avoid doing this?
Basically, what i want to do is to import a class from a package and avoid typing the package name and/or the file name.
Thanks in advance, and please let me know if the question isn't clear enough.
UPDATE
The structure i have is:
- parachute
-- init.py
-- container.py
Serves as a controller, i'd say, instancing, calling and glueing all the other parts together.
-- sparkles.py
Has two classes: Sparkle and Sparkles. Sparkle is a single element, with only one property so far, and Sparkles serves as a collection. Sparkles() belongs to the Emitter, and Sparkle() belongs to Sparkles().
-- emitter.py
Emitter could be seen as the user entity. It has a name and an uid, it belongs to a Container and the Container belongs to it.
Now, from outside the package i'm calling Container and passing some arguments, and the Container instances and distributes the arguments as it needs.
I have the impression that this isn't the best way to do what i need to do, which is: Create a collection of sparkles, owned by the emitter.
Don't put the class in it's own file. Put Container and Emitter directly in parachute.py.
You can then do
from parachute import Container, Emitter
or
import parachute
container = parachute.Container()
This essentially boils down to "Python isn't Java so for best results, don't treat it like it is" ;)
from module import Class
classInst = Class()
This will work if your class is in module.py
Is it possible to overload the from/import statement in Python?
For example, assuming jvm_object is an instance of class JVM, is it possible to write this code:
class JVM(object):
def import_func(self, cls):
return something...
jvm = JVM()
# would invoke JVM.import_func
from jvm import Foo
This post demonstrates how to use functionality introduced in PEP-302 to import modules over the web. I post it as an example of how to customize the import statement rather than as suggested usage ;)
It's hard to find something which isn't possible in a dynamic language like Python, but do we really need to abuse everything? Anyway, here it is:
from types import ModuleType
import sys
class JVM(ModuleType):
Foo = 3
sys.modules['JVM'] = JVM
from JVM import Foo
print Foo
But one pattern I've seen in several libraries/projects is some kind of a _make_module() function, which creates a ModuleType dynamically and initializes everything in it. After that, the current Module is replaced by the new module (using the assignment to sys.modules) and the _make_module() function gets deleted. The advantage of that, is that you can loop over the module and even add objects to the module inside that loop, which is quite useful sometimes (but use it with caution!).