Python project architecture - python

I'm a java developer new to python. In java, you can access all classes in the same directory without having to import them.
I am trying to achieve the same behavior in python. Is this possible?
I've tried various solutions, for example by importing everything in a file which I import everywhere. That works, but I have to type myClass = rootFolder.folder2.folder3.MyClass() each time I want to access a foreign class.
Could you show me an example for how a python architecture over several directories works? Do you really have to import all the classes you need in each file?
Imagine that I'm writing a web framework. Will the users of the framework have to import everything they need in their files?

Put everything into a folder (doesn't matter the name), and make sure that that folder has a file named __init__.py (the file can be empty).
Then you can add the following line to the top of your code:
from myfolder import *
That should give you access to everything defined in that folder without needing to give the prefix each time.
You can also have multiple depths of folders like this:
from folder1.folder2 import *
Let me know if this is what you were looking for.

Related

Including other Python file in current project

I have the following file. It contains several functions that I would use frequently. Instead of writing the same function over and over in every project, I think it would be more efficient if we are able to include the files in each project.
I want to include functions inside this file:
However, I don't know what technique is called and I am not sure if it is possible to make it work.
tools is not found:
Here's the folder that I want to include:
I have created __init__.py as explained in https://stackoverflow.com/a/4116384/17869806
If you are using a Jupyter notebook, create a new one in the folder ivm_github_python with this content:
from tools import ivm_string_print
ivm_string_print.myfunction()
in this case myfunction is defined in ivm_string_print.py
So in the file which is the main file for each project you can do
import filename and import it as a module in the start of the project.
Note: the file which you want to import should be in the same folder as your main file. and suppose the file name you want to import is "functions.py"
when you import it write import functions and not import functions.py
pls upvote if helps :)

When I import a package in my python script, does Python also import the sub package?

I am doing an online Python course and currently learning about modules and packages, but can't seem to get my head around the logic. I have used modules (or maybe they were packages who knows?) before in current job but without really knowing what was happening when I would do import Pandas for example.
By following the course I have created a folder called "MyMainPackage" and in this folder there is another folder called MySubPackage.
In "MyMainPackage":
init.py and my_main_script.py; in my_main_script.py there is a function called main_func()
in "MySubPackage":
init.py and my_sub_script.py; in my_sub_script.py there is a function called sub_func()
In Sublime Text editor I wrote a script called myprogram2.py:
import MyMainPackage
MyMainPackage.my_main_script.main_func()
However, this hasn't worked; when I call python myprogram2.py in Windows Command Prompt I get the following message
Attribute Error: module MyMainPackage has no attribute my_main_script
However, as per the online lecture, what does work is this:
from MyMainPackage import my_main_script
from MyMainPackage.MySubPackage import my_sub_script
my_main_script.main_func()
my_sub_script.sub_func()
Why can't I just import the entire package and access the modules like I have tried to above instead of the way the online lecture does it? I thought it would be the same thing. I am just struggling to understand the logic.

How to correctly import modules when working in a subdirectory?

I have a project where I want to structure the code in layers. The different parts of the program do very different things, and I wish to have a clean upper layer which binds all of the code in sub-directories together.
However, I struggle with importing modules correctly.
Say I have the structure
Project
└──manage.py
└──part a
├──script_a.py
├──__init__.py
└──modules_a
├──module_a1.py
├──module_a2.py
├──module_a3.py
└──__init__.py
└──part b
├──script_b.py
├──__init__.py
└──modules_b
├──module_b1.py
├──module_b2.py
├──module_b3.py
└──__init__.py
If I am writing code in script_a.py that depends on something from module_a1.py I use
from modules_a import module_a1
This works, but VS Code is never happy about the importing, always marking the imports with error. Therefore, I am wondering if there is something that I have logically misunderstood, especially since the script_a.py is not in the root folder?
If you are within a package and you want to access a sub package you have to put a . in front of the sub package. So change your import statement from
from modules_a import module_a1
to
from .modules_a import module_a1
Then the error disappears.
I decided to solve it by adding a testing file in the root folder and only running the script from the testing file, which will have similar functionality to the manage.py that will be my execution script later.

share files and functions through multiple projects

it's a kind of open question but please bear with me.
I am working on several projects (mainly with pandas) and I have created my standard approach to manage them:
1. create a main folder for all files in a project
2. create a data folder
3. have all the output in another folder
and so on.
One of my main activities is data cleaning, and in order to standardize it I have created a dictionary file where I store the various translation of the same entity, e.g. USA, US, United States, and so on, so that the files I am producing are consistent.
Every time I create a new project, I copy the dictionary file in the data directory and then:
xls = pd.ExcelFile(r"data/dictionary.xlsx")
df_area = xls.parse("area")
and after, to translate the country name into my standard, I call:
join_column, how_join = "country", "inner"
df_ct = pd.concat([
df_ct.merge(df_area, left_on=join_column, right_on="country_name", how=how_join),
df_ct.merge(df_area, left_on=join_column, right_on="alternative01", how=how_join),
and finally I check that I am not losing an record with a miss-join.
Over and over the same thing.
I would like to have a way to remove all this unnecessary cut and paste (of the file and of the code). Also, the file I used on the first projects are already deprecated and I need to update them (and sometime the code) when I need to process new data. Sometimes I also lose track of where is the latest dictionary file! Overall it's a lot of maintenance, which I believe might be saved.
Creating my own package is the way to go or is it a little too much ambitious?
Is there another shortcut? Overall it's not a lot of code, but multiplied by several projects.
Thanks for any insight, your time going through this is appreciated.
At the end I decided to create my own package.
It required some time so I am happy to share the details about the process (I run python on jupyter and windows).
The first step is to decide where to store the code.
In my case it was C:\Users\my_user\Documents
You need to add this directory to the list of the directories where python is looking for packages. this is achieved running the following statement:
import sys
sys.path.append("C:\\Users\\my_user\\Documents")
In order to run the above statement each time you start python, it must be included into a file in the directory (this directory might vary depending on your installation):
C:\Users\my_user\.ipython\profile_default\startup
the file can be named "00-first.py" ("50-middle.py" or "99-last.py" will also work)
To verify everything is working, restart python and run the command:
print(sys.path)
you should be able to see your directory at this point.
create a folder with the package name in your directory, and a subfolder (I prefer not to have code in the main package folder)
C:\Users\my_user\Documents\my_package\my_subfolder
put an empty file named "_ _init__.py" (note that there should be no space between underscores, but I do not know how to achieve it with the editor) in each of the two folders: my package and my_subfolder. At this point you should be able already to import your empty package from python
import my_package as my_pack
inside my_subfolder create a file (my_code.py) which will store the actual code
def my_function(name):
print("Hallo " + name)
modify the outer _ _init__.py file to include shortcuts. Add the following:
from my_package.my_subfolder.my_code import my_function
You should be able now to run the following in python:
my_pack.my_function("World!")
Hope you find it useful!

python modules accessing files in "local" directory

I have simple question.
I have a python module "dev1.py" that needs a file "dev1_blob"
If I have everything in one directory.
my_app loads the dev1 like
from dev1 import func1
it works fine.
I want to move dev1.py to a directory "./dev_files" with init.py in it.I can load the dev1.py as
from dev_files.dev1 import func1
However when func1 runs to access the "device_blob" -- it barfs as:
resource not found ..
This is so basic that I believe I am missing something.
I can't figure out why great minds of python want everything to refer to __file__ (cwd) and force me to modify dev1.py based on where it's being run from. i.e. in dev1.py refer to the file as: 'dev_files/device_blob'
I can make it work this way, but it's purely absurd way of writing code.
Is there a simple way to access a file next to the module files or in the tree below?
Relative pathing is one of Python's larger flaws.
For this use case, you might be able to call open('../dev_files/device_blob') to go back a dir first.
My general solution is to have a "project.py" file containing an absolute path to the project directory. Then I call open(os.path.join(PROJECT_DIR, 'dev_files', 'device_blob')).

Categories