We Keep Coding

Relative Paths in different IDEs

lets assume following workspace is opened in an IDE:
myWorkspace
|- folder1
|- file1.py
|- folder2
|- file2.csv
I edit and execute file1.py and I want to use file2.csv in it. When I use PyCharm as an Editor, the correct way to do so is to use the path "../folder2/file2.csv". But when I use VSCode I need to use the path "folder2/file2.csv". So VSCode assumes the start to be the workspace folder, and it seems that my PyCharm Editor just takes the path where the file is executed as starting point.
My question: Which of these two ways is more common? What property causes such a behavior?

I would say that the more common way is the way VS code works. At least when running your program outside your IDE. The myWorkspace folder is your root folder for the project if I'm correct here.
But I don't see why you don't have anything in your root folder. Maybe you should create the main file for the entry point in your root folder and import the file from folder1.
This behavior is determined by how you execute your program.
If you run the script like this:
$ pwd
.../myworkplace
$ python folder1/file1.py
then the CWD (current working directory) will be the myworkplace folder and the folder2/file2.csv approach will work.
If you run the script like this:
$ pwd
.../myworkplace/folder1
$ python file1.py
then the CWD will be the folder1 folder and the ../folder2/file2.csv route will be the correct one.

How to create subdirectories while saving Python files on Mac?

I'm learning Python using a book that uses Window's based examples and they use these as examples (the comments are the filenames):
# dir1\__init__.py
print('dir1 init')
x = 1
# dir1\dir2\__init__.py
print('dir1 init')
y = 2
# dir1\dir2\mod.py
print('in mod.py')
z = 3
Now using IDLE, it attempts to import the files using:
import dir1.dir2.mod
Which will display:
dir1 init
dir2 init
in mod.py
When I attempt to do this on IDLE on my Mac it comes up with an error saying there is no module named dir1.
The point of the exercise is to package imports but I don't if I'm formatting the files incorrectly or importing incorrectly.

I think the files are not in the same directory from where the IDLE is running or looking at.
Try to see where the IDLE is on your Mac, complete path. Once find that out may be try putting the files in the same place so that python can see those files.
As an experiment, I'd suggest, writing your scripts in an editor like VS Code. cd into the directory where you have saved the script. Run the script by python yourfilename.py and see what it says then. Personally, I have never used the IDLE but I have come across python not being able to find the file.

I'm not sure what exactly you are asking, but your file structure should look like this in Finder:
which is:
dir1
|-- __init__.py
|-- dir2
|-- __init__.py
|-- mod.py
And you need to make sure that your working directory in IDLE is the directory where dir1 is located. I don't think you can change that in IDLE, but you could do this to make it work:
>>> import sys
>>> sys.path.insert(0,'/path/to/directory/where/dir1/is/stored')

Python FileNotFoundError when using open()

I'm using with open('myFile', 'rb') as file: to read a file. When running the program with python myProgram.py everything works fine. But as soon I try to run it without cd-ing into the directory of myProgram.py and use an absolute path instead (like python /home/myName/myCode/myProgram.py I always get this error message: FileNotFoundError: [Errno 2] No such file or directory.
So why does open() behave differently depending on how the Python program is started? And is there a way to make things work even if starting with an absolute path?
I've already tried open('/home/myName/myCode/myfile', 'rb') but without success...

So why does open() behave differently depending on how the Python program is started? And is there a way to make things work even if starting with an absolute path?
Because that is standard behavior. Not just for Python, but for all applications (at least all that I know of). A relative path (like 'myfile') is assumed to be in the current directory. That path will change depending on what directory you are sitting in when you launch the script from a shell. If you don't give an absolute path, how would Python know which file to use in a directory structure like this?
root
|-- dir1
| `-- myFile
`-- dir2
`-- myFile
You were on the right track with open('/home/myName/myCode/myfile', 'rb').

Let's say you run pwd and it returns /home/myName. If you then run /home/myName/code/myProgram.py, the working directory of your program is not /home/myName/code; it's /home/myName. The working directory of a process in inherited from the parent process, not set based on where the script is located.

Get the directory of where my script is stored

I am writing a Python script that has to execute a shell script. In my GIT repo I have plans to commit both (Python program + Shell script) in same directory in my repo.
My issue is that when someone pulls out my code and wants to run my Python script from any relative / absolute location - I need to refer the shell script in the directory where my Python script resided.
I am not sure which one I should make use of
os.path.dirname(os.path.realpath(__file__))
OR
os.path.abspath(os.path.dirname(__file__))
OR
os.path.dirname(os.path.abspath(__file__))
If I run my python script in same directory it prints same values - even if I execute from a separate directory and run my script as mentioned below I still get same values:
python ./test/test1/1.py
/x/home02/myhome/test/test1
If it so which one I should actually make use of? What is the difference in between each of them?
========== Updated =========
I created a symbolic link to my code like as mentioned below:
cd
ln -s /x/home02/myhome/test/test1/1.py 2.py
Now when I re-run my code as mentioned below -
cd
python 2.py
cd test
python ../2.py
I get the below output
/x/home02/myhome/test/test1
/x/home02/myhome
/x/home02/myhome
So I think the below one is the correct one as I am always getting the expected output:
os.path.dirname(os.path.realpath(__file__))

Well first you want the directory, so let's use os.path.dirname(...)
And os.path.abspath(__file__) should get you the path of the script, the difference between this and os.path.realpath() is the latter eliminates symbolic links, so not sure if you want that
So I'd use os.path.dirname(os.path.abspath(__file__))

The difference between os.path.realpath and os.path.abspath is the former eliminates symbolic links whereas the latter be the absolute path of a symbolic link.
In this case, the order of os.path.dirname and abspath or realpath shouldn't matter -- It would only matter if the file you passed in didn't have a directory portion.

You should use absolute path with directory name of pathname, like this:
import os
BASE_DIR = os.path.dirname(os.path.dirname(os.path.abspath(__file__)))
Find the common pathname manipulations (os.path) documentation to find the difference between the usage here: https://docs.python.org/3/library/os.path.html

Problem with python packages and tests

I have a directory struture like that:
project
| __init__.py
| project.py
| src/
| __init__.py
| class_one.py
| class_two.py
| test/
| __init__.py
| test_class_one.py
Which project.py just instantiate ClassOne and run it.
My problem is in the tests, I don't know how to import src classes. I've tried importing these ways and I got nothing:
from project.src.class_one import ClassOne
and
from ..src.class_one import ClassOne
What am I doing wrong? Is there a better directory structure?
----- EDIT -----
I changed my dir structure, it's like this now:
Project/
| project.py
| project/
| __init__.py
| class_one.py
| class_two.py
| test/
| __init__.py
| test_class_one.py
And in the test_class_one.py file I'm trying to import this way:
from project.class_one import ClassOne
And it still doesn't work. I'm not using the executable project.py inside a bin dir exactly because I can't import a package from a higher level dir. :(
Thanks. =D

It all depends on your python path. The easiest way to achieve what you're wanting to do here is to set the PYTHONPATH environment variable to where the "project" directory resides. For example, if your source is living in:
/Users/crocidb/src/project/
I would:
export PYTHONPATH=/Users/crocidb/src
and then in the test_one.py I could:
import project.src.class_one
Actually I would probably do it this way:
export PYTHONPATH=/Users/crocidb/src/project
and then this in test_one.py:
import src.class_one
but that's just my preference and really depends on what the rest of your hierarchy is. Also note that if you already have something in PYTHONPATH you'll want to add to it:
export PYTHONPATH=/Users/crocidb/src/project:$PYTHONPATH
or in the other order if you want your project path to be searched last.
This all applies to windows, too, except you would need to use windows' syntax to set the environment variables.

From Jp Calderone's excellent blog post:
Do:
name the directory something related to your project. For example, if your
project is named "Twisted", name the
top-level directory for its source
files Twisted. When you do releases,
you should include a version number
suffix: Twisted-2.5.
create a directory Twisted/bin and put your executables there, if you
have any. Don't give them a .py
extension, even if they are Python
source files. Don't put any code in
them except an import of and call to a
main function defined somewhere else
in your projects. (Slight wrinkle:
since on Windows, the interpreter is
selected by the file extension, your
Windows users actually do want the .py
extension. So, when you package for
Windows, you may want to add it.
Unfortunately there's no easy
distutils trick that I know of to
automate this process. Considering
that on POSIX the .py extension is a
only a wart, whereas on Windows the
lack is an actual bug, if your
userbase includes Windows users, you
may want to opt to just have the .py
extension everywhere.)
If your project is expressable as a single Python source file, then put it
into the directory and name it
something related to your project. For
example, Twisted/twisted.py. If you
need multiple source files, create a
package instead (Twisted/twisted/,
with an empty
Twisted/twisted/__init__.py) and place
your source files in it. For example,
Twisted/twisted/internet.py.
put your unit tests in a sub-package of your package (note - this means
that the single Python source file
option above was a trick - you always
need at least one other file for your
unit tests). For example,
Twisted/twisted/test/. Of course, make
it a package with
Twisted/twisted/test/__init__.py.
Place tests in files like
Twisted/twisted/test/test_internet.py.
add Twisted/README and Twisted/setup.py to explain and
install your software, respectively,
if you're feeling nice.
Don't:
put your source in a directory called src or lib. This makes it hard
to run without installing.
put your tests outside of your Python package. This makes it hard to
run the tests against an installed
version.
create a package that only has a __init__.py and then put all your code into __init__.py. Just make a module
instead of a package, it's simpler.
try to come up with magical hacks to make Python able to import your module
or package without having the user add
the directory containing it to their
import path (either via PYTHONPATH or
some other mechanism). You will not
correctly handle all cases and users
will get angry at you when your
software doesn't work in their
environment.