I have been working on a python project and I am new to it.
I have made a small library for my project in which I have several different modules doing different tasks.
For example: I have 5 modules namely add, subtract, multiply, divide and root.
I call all these .pyc files into my main.py file and my code runs properly if all of them are in the same folder.
Now, I want to store my main.py at: D:\project\main.py
and these 5 .pyc files at : D:\project\Lib\ (In the Lib folder)
I found a solution as to mention the path of the folder Lib into the code but I can not do so as I need to submit the code somewhere and if they try to run this on their PC, it might not import these files.
What would be the possible solution to this?
Try creating a package.
Use a directory structure like this:
.
+-- main.py
+-- lib
+-- __init__.py
+-- add.pyc
+-- substract.pyc
+-- ...
Then, in your main.py file, you can import them like this:
from lib import add
More on packages on Python docs
Inside D:\project\Lib create an __init__.py file. and put all your modules in D:\project\Lib now lib works as a python package.you dir structure should now look like this:
D:\project\Lib
|
+--- __init__.py
+--- add.py
+--- sub.py
+--- multiply.py
Now from any file inside (say for ex main.py) D:\project call any module you want like this.
from Lib.add import something.
final dir structure will roughly look like this.
D:\project
|
+-- main.py
+-- Lib
|
+--- __init__.py
+--- add.py
+--- sub.py
+--- multiply.py
Related
I'm having some troubles trying to schedule the execution of one project.
The structure is:
main folder
|- lib
| |- file1.py
| |- file2.py
|
|- data
| |- file.csv
|
|- temp
| |- file.json
|
|-main.py
The contrab line is:
*/5 * * * * python3 /home/myName/main_folder/main.py
I've been trying this command line with simple python scripts without dependences and works fine. The problem is that in this case the main.py import classes and functions inside lib and I think it can deal with it.
On my main.py I'm importing like this from lib import file1, file2. Exists another way maybe using os that the program knows the absolute path?
Please try to add an empty file named: __init__.py in lib directory.
I am working on a program and access a set of custom modules. I organized the modules in a subfolder as own package with an __init__.py. Moreover, on the main level of the directory I have created a virtual environment that holds my dependencies. The folder structure is as follows:
project
+-- main_program.py
+-- venv
| +-- cool_package.py
+---mypackage
| +-- module1.py
| +-- module2.py
| +-- __init__.py
The issue is, that module2.py depends on a package I installed in venv. Running module2.py from main_program.py gives an error "cool_package.py" not found.
How do I organize stuff like that so that I can accesses cool_package.py from main_program.py with all the other needed packaged. And make cool_package.py accessible for the custom package with module2.py as well?
I may have misunderstood what you mean by your virtual env but based on your folder and file layout I think you need to add an __init.py__ file to your venv folder to make it a package and then you should be able to import venv.cool_package.
thanks for all answers - it eventually worked by properly activating the environment before running the script. Something must have gone wrong the first time - all works now and the folder structure is correct.
Best
Moritz
I know this is a worn out topic but the import mechanism/s in python is still confusing the masses. What I want is the ability to import a custom module that is in a parent directory in a way that allows me to take a project to another environment and have all of the imports work.
For example a structure like this:
repo
|--- folder1
| |--- script1.py
|--- folder2
| |--- script2.py
|--- utils
|--- some-util.py
How can I import from some-util.py in both script1 and script2? The idea is that I could clone the repo into a remote host and run scripts from folders 1 and 2 that may have the shared dependency of some-util.py only I don't want to have to run anything before hand. I want to be able to:
connect to box
git clone repo
python repo/folder1/script1.py
contents of script1 and script2:
import some-util
<code>
EDIT:
I forgot to mention that occasionally the scripts need to be run from another directory like:
/nas/some_folder/repo/folder1/script1.py args..
Also, the box is limited to python 2.7.5
The trick is to implement as your scripts as modules (read here and here for an overview of what the python -m switch means).
Here is a structure, also notice every directory contains an (empty file) named __init__.py:
repo/
|____utils/
| |____someutil.py
| |___ __init__.py
|___ __init__.py
|____folder1/
|____script1.py
|___ __init__.py
utils.someutil may contain something like this:
def say_hello():
return "Hello World."
And your script1.py may contain something like:
from ..utils.someutil import say_hello
if __name__ == "__main__":
print(say_hello())
Then running the following:
python -m repo.folder1.script1
... produces:
Hello World.
I'm quite new to working with more complex modules, but I think is about time to implement them in my work flow. Also, I don't come from a software engineering background so some of my languages might be inaccurate, please bear with me. My module folder structure is like this:
+-- module_name
| +-- ml.py
| +-- exp_an.py
| +-- plotting.py
+-- etl
| +-- machine_learning.py
| +-- data_manipulation.py
The reason I have two folders is because the scripts directly on module_name are our personal library for use on most projects and the etl has code that is specific to this project.
In the beginning I had both folders on the same directory but I was having trouble importing from module_name to etl.
The thing is, in machine_learning.py I want to call a function from ml.py. I tried doing:
import sys
sys.append('../')
import module_name as mn
But this seems to bring some recursivity issues, because when I try to call mn.ml I get a mn has no attribute called ml error.
So, my question is, what is the right way to do this? Let's say I want to call a function called transform() that is inside ml.py in my machine_learning.py script. Is there a way to do this? Is there a better way to do this? Thanks
In order for your directories to be interpreted as modules, you need to add __init__.py in each directory. Your directory structure will look like this.
+-- module_name
| +-- __init__.py
| +-- ml.py
| +-- exp_an.py
| +-- plotting.py
+-- etl
| +-- __init__.py
| +-- machine_learning.py
| +-- data_manipulation.py
Then you'd use relative imports to get the modules. Example ->
# Inside machine_learning.py you are importing ml.py
import ..ml as ml
ml.transform()
Here is an example of a larger project. You can see how relative imports are used and the directories have their __init__.py.
https://github.com/TileThePlane/docusign-python-client
It sounds like you want to add the parent directory of module_name and etl (whatever that directory is) to your PYTHONPATH.
After doing that, you should be able to do all the imports you asked for.
To explain the mn has no attribute called ml error, that happened because you did not import ml. You just tried to use it as an attribute of mn, and you can't do that.
I am having problems importing anything into my testing files that I intend to run with py.test.
I have a project structure as follows:
/ProjectName
|
|-- /Title
| |-- file1.py
| |-- file2.py
| |-- file3.py
| |-- __init__.py
|
|-- /test
| |-- test_file1.py
I have not been able to get any import statements working with pytest inside the test_file1.py file, and so am currently just attempting to use a variable declared in file_1.py and print it out when test_file1.py is run.
file1.py contains:
file1_variable = "Hello"
test_file1.py contains:
import sys
import os
sys.path.append(os.path.abspath('../Title'))
import file1
def test_something():
assert file1.file1_variable == "Hello"
print(file1.file1_variable)
The import statement from my testing file is taken from https://stackoverflow.com/a/10272919/5477580 which works by editing the PYTHONPATH variable. This allows me to run the test_file1.py script and successfully execute the print statement.
However, attempting to run py.test from the /ProjectName directory gives me an error saying ImportError: No module named 'file1'
Is there some way in which I can structure things better so that pytest will be possible? Do I need to add something to my __init__.py file?
No you don't need to add anything to __init__.py. This file tells python to treat the parent directory as module which can be imported as described here.
Just add it to your Title directory and import it like
from ..Tiltle.file import file1_variable
here we are moving 1 hierarchy above in the directory just like cd ..
Note .. is for reaching the Title package from test directory. If you need to run your file from ProjectName directory you will have to do
from Tiltle.file import file1_variable