I have a homegrown python library. As it is a library, it should be initialized with parameters every time it is used, based on different projects using it. For example, here is the sample pseudo code:
import myownlibrary
myownlibrary.init('path_to_config_file_containing_details_to_process_data')
Any idea how this can be achieved?
A sample reference code will be very helpful.
Thanks
I added it to the __init__.py as below
global configpath
def setpath(self, pathpassedin):
self.configpath = pathpassedin
print("Value passed in: ", pathpassedin)
I thought self was required, but having self also requires that the method be called as below:
import myownlibrary as mylib
myownlibrary.setpath(mylib, 'path_to_config_file')
Not sure if this is the way to set this up. Would appreciate feedback
Related
After dozens of research on the subject and a lot of thinking, I leave it to you in this new question:
Is it possible to mock an entire library with Python? I would like the import of this library and all its packages / modules / etc to be done without having to define each element by hand, with mock and sys.module ... :(
In my case, I use a library specific to the job and I would like to be able to work on my code at home, without having to recode my imports, on code which is not dependent on this library.
Example:
"""Main file.
I define the mock here.
"""
mocked = MagicLibraryMock("mylib") # the dream
"""File with lib imports.
I can import anything and use it as a mock.
"""
import mylib
from mylib.a import b
from mylib.z import c
from mylib.a.e.r import x
foo = x()
bar = c.a.e.r.t.d()
bar.side_effect = [1, 2, 3]
bar()
I tried to integrate a class inherited from a dictionary to overload the __getitem__ method of sys.modules. But the problem is that the import method also uses __iter__, and there it becomes much more complicated to return a MagicMock according to the result, knowing that it is not recommended to directly modify the import source code - source.
Finally I lose less time extracting imports from my application to sub-modules which will take care of solving them. I can thus intercept these imports more easily without dirtying my code.
The design is more interesting.
Thanks for your help.
I'm working with Tensorflow in Python. In a custom written function I found #tf_export() before the function definition like below, the function of which I don't understand. Could somebody explain?
#tf_export("signal.ifftshift")
def ifftshift(x, axes=None, name=None):
As I understand, it allows Tensorflow to expose a function or class under a different name. For example, the Server class within the distribute module actually lives in the training/server_lib.py file within the repo, but, since it is exported as distribute.Server, you can use it like tf.distribute.Server().
# training/server_lib.py
#tf_export("distribute.Server", v1=["distribute.Server", "train.Server"])
#deprecation.deprecated_endpoints("train.Server")
class Server(object):
...
It makes it confusing to find the code, but I imagine it's a more flexible way to create these "logical" modules.
It is a convenient way to output dot delimited symbols directly to the tf API. Namely, a user can access ifftshift() with tf.signal.ifftshift(), without caring about the true path (here tf.python.ops.signal.fft_ops.ifftshif()).
Basically I have 3 modules that all communicate with eachother and import eachother's functions. I'm trying to import a function from my shigui.py module that creates a gui for the program. Now I have a function that gets the values of user entries in the gui and I want to pass them to the other module. I'm trying to pass the function below:
def valueget():
keywords = kw.get()
delay = dlay.get()
category = catg.get()
All imports go fine, up until I try to import this function with
from shigui import valueget to another module that would use the values. In fact, I can't import any function to any module from this file. Also I should add that they are in the same directory. I'm appreciative of any help on this matter.
Well, I am not entirely sure of what imports what, but here is what I can tell you. Python can sometimes allow for circular dependencies. However, it depends on what the layout of your dependencies is. First and foremost, I would say see if there is any way you can avoid this happening (restructuring your code, etc.). If it is unavoidable then there is one thing you can try. When Python imports modules, it does so in order of code execution. This means that if you have a definition before an import, you can sometimes access the definition in the first module by importing that first module in the second module. Let me give an example. Consider you have two modules, A and B.
A:
def someFunc():
# use B's functionality from before B's import of A
pass
import B
B:
def otherFunc():
# use A's functionality from before A's import of B
pass
import A
In a situation like that, Python will allow this. However, everything after the imports is not always fair game so be careful. You can read up on Python's module system more if you want to know why this works.
Helpful, but not complete link: https://docs.python.org/3/tutorial/modules.html
I spent last months rewriting from scratch a new version of my Python algorithm. One of my goals was to write a perfectly documented code, easy to read and understand for "anyone".
In the same project folder I put a lot of different modules and each module contain a class. I used classes as functions and related variables container, in that way a class contain all the functions with a specific task, for example wrinting on Excel files all the output results of the algorithm.
Here an example:
Algorithm.py
import os
import pandas as pd
import numpy as np
from Observer import Observer
def main(hdf_path):
for hdf_file in os.listdir(hdf_path):
filename = str(hdf_file.replace('.hdf', '.xlsx'))
Observer.create_workbook(filename)
dataframe = pd.read_hdf(hdf_file)
years_array = dataframe.index.levels[0].values
for year in years_array:
year_mean = np.mean(dataframe.loc[year].values)
Observer.mean_values = np.append(Observer.mean_values, dataframe_mean)
Observer.export_result()
if __name__ == "main":
hdf_path = 'bla/bla/bla/'
main(hdf_path)
Observer.py
import numpy as np
import openpyxl
class Observer:
workbook = None
workbookname = None
mean_values = np.array([])
def create_workbook(filename):
Observer.workbook = openpyxl.Workbook()
Observer.workbookname = filename
# do other things
def save_workbook():
Observer.workbook.save('results_path' + Observer.workbookname)
def export_results():
# print Observer.mean_values values in different workbook cells
# export result on a specific sheet
I hope that you can understand from this simple example how do I use class on my project. For every class I define a lot of variables (workbook for example) and I call them from other modules as if they were global variables. In that way I can easily access them from anywhere and I dont need to pass them to functions explicitly, cause I can simply write Classname.varname.
My question is: is it bad design? Will it create some problems or performance slowdown?
Thanks for your help.
My question is: is it bad design?
Yes.
I can simply write Classname.varname.
You are creating a very strong coupling between classes when you enforce calling Classname.varname. The class that access this variable is now strongly coupled with Classname. This prevent you from changing the behavior in OOP way by passing different parameters, and will complicate testing of the class - since you will be unable to mock Classname and use its mock instead of the "real" class.
This will result in code duplication when you try to run 2 pieces of very similar code in two parts of your app, which only vary in these parameters. You will end up creating two almost identical classes, one using Workbook and the other using Notepad classes.
And remember the vicious cycle:
Hard to test code -> Fear of refactor -> Sloppy code
^ |
| |
---------------------------------------
Using proper objects, with ability to mock them (and dependency injection) is going to guarantee your code is easily testable, and the rest will will follow.
I'm creating a class to extend a package, and prior to class instantiation I don't know which subset of the package's namespace I need. I've been careful about avoiding namespace conflicts in my code, so, does
from package import *
create problems besides name conflicts?
Is it better to examine the class's input and import only the names I need (at runtime) in the __init__ ??
Can python import from a set [] ?
does
for name in [namespace,namespace]:
from package import name
make any sense?
I hope this question doesn't seem like unnecessary hand-ringing, i'm just super new to python and don't want to do the one thing every 'beginnger's guide' says not to do (from pkg import * ) unless I'm sure there's no alternative.
thoughts, advice welcome.
In order:
It does not create other problems - however, name conflicts can be much more of a problem than you'd expect.
Definitely defer your imports if you can. Even though Python variable scoping is simplistic, you also gain the benefit of not having to import the module if the functionality that needs it never gets called.
I don't know what you mean. Square brackets are used to make lists, not sets. You can import multiple names from a module in one line - just use a comma-delimited list:
from awesome_module import spam, ham, eggs, baked_beans
# awesome_module defines lots of other names, but they aren't pulled in.
No, that won't do what you want - name is an identifier, and as such, each time through the loop the code will attempt to import the name name, and not the name that corresponds to the string referred to by the name variable.
However, you can get this kind of "dynamic import" effect, using the __import__ function. Consult the documentation for more information, and make sure you have a real reason for using it first. We get into some pretty advanced uses of the language here pretty quickly, and it usually isn't as necessary as it first appears. Don't get too clever. We hates them tricksy hobbitses.
When importing * you get everything in the module dumped straight into your namespace. This is not always a good thing as you could accentually overwrite something like;
from time import *
sleep = None
This would render the time.sleep function useless...
The other way of taking functions, variables and classes from a module would be saying
from time import sleep
This is a nicer way but often the best way is to just import the module and reference the module directly like
import time
time.sleep(3)
you can import like from PIL import Image, ImageDraw
what is imported by from x import * is limited to the list __all__ in x if it exists
importing at runtime if the module name isn't know or fixed in the code must be done with __import__ but you shouldn't have to do that
This syntax constructions help you to avoid any name collision:
from package import somename as another_name
import package as another_package_name