I'm working on a python project that has some of duplicated code.
I'm trying to remove the duplicated files, and keep only one file, however, my problem is that others projects already use the removes files, and I don't want to have to change their imports.
Example:
src/
-----a.py
-----b.py
Lets assume both a and b implement the same function f, and there are pieces of code in the other project who uses from src.a import f and others use from src.b import f
Right now my solution is to keep the implementation in a and have b only contain from src.a import f.
Is there a different way where I could completely remove b but still have the ability to use from src.b import f ?
Without a horrible import hook hack that emulates a virtual b.py, no.
However, you can just make b.py a "re-export" module like
from a import foo, baz, quux
assuming those are the duplicate names.
Related
I have written a bunch of functions stored in different python files over the past years that I would like to dump into a folder and just use 'import folder' to call the functions from those python files.
So far, all the solutions I have read needs to either:
from folder.pyfilename import func1
func1()
or
from folder import *
pyfilename.func1()
Does anyone know if it's possible to do something like this?
You could add a file folder/__init__.py and have it do the subfolder imports.
__init__.py
from .prog1 import foo, bar
from .prog2 import baz
__all__ = ["foo", "bar", "baz"]
__all__ lists the variables that are imported by from foo import * so its not really all of the names, but it is all of the names the implementer thinks you should care about.
Now you can write programs that do
from folder import *
foo()
Your question doesn't make too much sense to be honest, however you could do something like this:
import folder.program as prog
from folder import program2 as prog2
prog.function()
prog2.function()
In case you don't know, the as keyword is used to create an alias.
As far as I'm aware, you can't do what you want. You have to tell python which file the function is in.
I'm guessing you want to import the whole folder because it would take a while to add all the imports? I think you would just have to copy all the functions you want into one single file - or import each file/function as you already have done.
Over the years I have written a few hundred functions in matlab for space engineering purposes which I use on a daily basis. They are all nicely put in one folder ordered in subfolders, and in matlab I just have an addpath() command for the root folder in the startup.m file, and then I can use any of the functions right away after starting up matlab.
I am now trying to do the same in python.
As far as I understand in python I shouldn't have 1 file for every function like in matlab, but rather bundle all the functions together in 1 py file. Is this correct? I am trying to avoid this, since I have a strong preference for short scripts rather than 1 huge one, due to it being way more intuitive for me that way.
And then, once I have all my python scripts, can I place them anywhere in order to use them? Because I read that python works differently than matlab in this aspect, and scripts need to be in the working directory in order to import them. However, I want to load the scripts and be able to use them regardless of my active working directory. So I am guessing I have to do something with paths. I have found that I can append to pythonpath using sys.path.insert or append, however this feels like a workaround to me, or is it the way to go?
So considering I have put all my rewritten matlab fuctions in a single python file (lets call it agfunctions.py) saved in a directory (lets call it PythonFunctions). The core of my startup.py would then be something like (I have added the startup file to PYTHONSTARTUP path):
# startup.py
import os, sys
import numpy as np
import spiceypy as spice
sys.path.append('C:\Users\AG5\Documents\PythonFunctions')
import agfunctions as ag
Does any of this make sense? Is this the way to go, or is there a better way in python?
Well, the python package is probably the best way to solve your problem. You can read more here. Python packages have no need to be built and not always are created for sharing, so do not worry.
Assume you had this file structure:
Documents/
startup.py
PythonFunctions/
FirstFunc.py
SecondFunc.py
Then you can add file __init__.py in your PythonFunctions directory with next content:
__all__ = ['FirstFunc', 'SecondFunc']
Init file must be updated if you change filenames, so maybe it isn't best solution for you. Now, the directory looks like:
Documents/
startup.py
PythonFunctions/
__init__.py
FirstFunc.py
SecondFunc.py
And it's all - PythonFunctions now is a package. You can import all files by one import statement and use them:
from PythonFunctions import *
result_one = FirstFunc.function_name(some_data)
result_two = SecondFunc.function_name(some_other_data)
And if your startup.py is somewhere else, you can update path before importing as next:
sys.path.append('C:\Users\AG5\Documents')
More detailed explanations can be found here
There is no need to write all your matlab functions within one file. You can also keep (kind of) your desired folder structure of your library. However, you should write all functions in the deepest subfolder into one *.py file.
Suppose your MATLAB library is in the folder space_engineering and is set up like this:
space_engineering\
subfolder1\
functiongroup1\
printfoo.m
printbar.m
subfolder2\
...
subfolder3\
...
...
After doing addpath(genpath('\your_path\space_engineering')) all functions in subfolders subfolder* and functiongroup* are available in the global namespace like this
>> printfoo % function printfoo does only do fprintf('foo')
foo
I understand this is an behaviour your want to preserve in your migrated python library. And there is a way to do so. Your new python library would be structured like this:
space_engineering\
__init__.py
subfolder1\
__init__.py
functiongroup1.py
functiongroup2.py
subfolder2\
__init__.py
...
subfolder3\
__init__.py
...
...
As you can see the deepest subfolder layers functiongroup* of the MATLAB structure is replaced now by functiongroup*.py files, so called modules. The only compromise you have to allow for is that functions printfoo() and printbar() are now defined in this .py modules instead of having individual .py files.
# content of functiongroup1.py
def printfoo():
print(foo)
def printbar():
print(bar)
To allow for doing the same function calling as in MATLAB you have to make the function names printfoo and printbar available in the global namespace by adjusting the __init__.py files of each subfolder
# content of space_enginieering\subfolder1\__init__.py
from .functiongroup1 import *
from .functiongroup2 import *
as well as the __init__.py of the main folder
# content of space_engineering\__init__.py
from .subfolder1 import *
from .subfolder2 import *
from .subfolder3 import *
The from .functiongroup1 import * statement loads all names from module functiongroup1 into the namespace of subfolder1. Successively from .subfolder1 import * will forward them to the global namespace.
Like this you can do in an python console (or in any script) e.g.:
>>> sys.path.append('\your_path\space_engineering')
>>> from space_engineering import *
>>> printfoo()
foo
This way you can use your new python library in the same way as you former used the MATLAB library.
HOWEVER: The usage of from xyz import * statement is not recommend in python (see here why, it is similar to why not using eval in MATLAB) and some Pythonistas may complain recommending this. But for your special case, where you insist on creating a python library with MATLAB like comfort, it is a proper solution.
Is it possible to import a test.py file in whole project instead of importing it in each file?
for example calling it some where in start.py
No.
However, suppose you have three modules, a.py, b.py, and c.py. Further, assume that c.py imports b, and b.py imports a. Then in c.py, you can refer to items in a using
b.a.foo
since b imported a into its namespace.
even if it is ... (which in programming is almost always the case) a better question is why would you want to.
It would be difficult to do, and it adds nothing... in fact it can severely hinder development, and you may experience very strange bugs that become very hard to track down
[edit based on OP comment]
your use case sounds like you just want it to be in builtins
start.py
import __builtin__
import my_weird_test_class
__builtin__.tester = my_weird_test_class
from main_entry import main
main()
now in any file you can use
tester.do_something()
without importing tester or whatever
but as I said this tends to be a very bad idea... it is much better to explicitly import it in all your files ...
Suppose you have the following
b
b/__init__.py
b/c
b/c/__init__.py
b/c/d
b/c/d/__init__.py
In some python packages, if you import b, you only get the symbols defined in b. To access b.c, you have to explicitly import b.c or from b import c. In other words, you have to
import b
import b.c
import b.c.d
print b.c.d
In other cases I saw an automatic import of all the subpackages. This means that the following code does not produce an error
import b
print b.c.d
because b/__init__.py takes care of importing its subpackages.
I tend to prefer the first (explicit better than implicit), and I always used it, but are there cases where the second one is preferred to the first?
I like namespaces -- so I think that import b should only get what's in b itself (presumably in b/__init__.py). If there's a reason to segregate other functionality in b.c, b.c.d, or whatever, then just import b should not drag it all in -- if the "drag it all in" does happen, I think that suggests that the namespace separation was probably a bogus one to start with. Of course, there are examples even in the standard library (import os, then you can use os.path.join and the like), but they're ancient, by now essentially "grandfathered" things from way before the Python packaging system was mature and stable. In new code, I'd strongly recommend that a package should not drag its subpackages along for the ride when you import it. (Do import this at the Python prompt and contemplate the very last line it shows;-).
__all__ = [your vars, functions, classes]
Use syntax above in package b's __init__.py to auto load things listed in dict. :)
Assume I have this barebones structure:
project/
main.py
providers/
__init.py__
acme1.py
acme2.py
acme3.py
acme4.py
acme5.py
acme6.py
Assume that main.py contains (partial):
if complexcondition():
print providers.acme5.get()
Where __init__.py is empty and acme*.py contain (partial):
def get():
value=complexcalculation()
return value
How do I change these files to work?
Note: If the answer is "import acme1", "import acme2", and so on in __init__.py, is there a way to accomplish that without listing them all by hand?
hey! two years later but... maybe could be helpfull to some one
make your providers/__init__.py like that:
import os
import glob
module_path = os.path.dirname(__file__)
files = glob.glob(os.path.join(module_path, 'acme*.py'))
__all__ = [os.path.basename(f)[:-3] for f in files]
you don't have to change it later if add or remove any providers/acme*.py
then use from providers import * in main.py
If I'm reading your question correctly, it looks like you're not trying to do any dynamic importing (like in the question that Van Gale mentioned) but are actually trying to just import all of the modules in the providers package. If that's the case, in __init__.py you would want to have this statement:
__all__ = ["acme1", "acme2", "acme3", "acme4", "acme5", "acme6"]
Then to import everything you would use from ... import *
from providers import *
And then instead of using the package name explicitly in the code, you would just call the imported classes
acme1.get()
acme2.get()
If you have enough modules in the providers package that it becomes a problem populating the __all__ list, you may want to look into breaking them up into smaller packages or storing the data some other way. I personally wouldn't want to have to deal with dynamic importing schennagins every time I wanted to re-use the package.
This question asked today, Dynamic Loading of Python Modules, should have your answer.