We Keep Coding

Navigating directories

After getting the path to the current working directory using:
cwd = os.getcwd()
How would one go up one folder: C:/project/analysis/ to C:/project/ and enter a folder called data (C:/project/data/)?

In general it a bad idea to 'enter' a directory (ie change the current directory), unless that is explicity part of the behaviour of the program.
In general to open a file in one directory 'over from where you are you can do .. to navigate up one level.
In your case you can open a file using the path ../data/<filename> - in other words use relative file names.
If you really need to change the current working directory you can use os.chdir() but remember this could well have side effects - for example if you import modules from your local directory then using os.chdir() will probably impact that import.

As per Python documentation, you could try this:
os.chdir("../data")

GitPython: retrieve changed files between commits in specific sub directory

The repo structure looks like this:
- folder_a
- folder_b
- folder_c
- ...
I am particularly interested in the files that changed in a specific commit, but only the ones in folder_a. My solution is
for filename, details in commit.stats.files.items():
if not filename.startswith('folder_a'):
continue
# ...
but it seems the performance is not quite good if there are a great number of files in the other folders. Is there any better way to skip the files I don't care about?

If I understand correctly : you want stats on modifications from a commit, only on one specific subfolder.
Using plain git :
git show [commit] --stat folder_a
will display exactly what you want.
Have a look at what : git.show('<commit identifier>', '--stat', 'folder_a'); returns in your python script.

Deploying 1 python package under 2 different names

I am trying to have 2 simultaneous versions of a single package on a server. A production and a testing one.
I want these 2 to be on the same git repository on 2 different branches (The testing would merge into production) however i would love to keep them in the same directory so its not needed to change any imports or paths.
Is it possible to dynamically change the package name in setup.py, depending on the git branch?
Or is it possible to deploy them with different names using pip?
EDIT : i may have found a proper solution for my problem here : Git: ignore some files during a merge (keep some files restricted to one branch)
Gitattributes can be setup to ignore merging of my setup.py, ill close this question after i test it.

This could be done with a setup script that looks like this:
#!/usr/bin/env python3
import pathlib
import setuptools
def _get_git_ref():
ref = None
git_head_path = pathlib.Path(__file__).parent.joinpath('.git', 'HEAD')
with git_head_path.open('r') as git_head:
ref = git_head.readline().split()[-1]
return ref
def _get_project_name():
name_map = {
'refs/heads/master': 'ThingProd',
'refs/heads/develop': 'ThingTest',
}
git_ref = _get_git_ref()
name = name_map.get(git_ref, 'ThingUnknown')
return name
setuptools.setup(
# see 'setup.cfg'
name=_get_project_name(),
)
It reads the current git ref directly from the .git/HEAD file and looks up the corresponding name in a table.
Inspired from: https://stackoverflow.com/a/56245722/11138259.

Using .gitattributes file with content "setup.py merge=ours" and also setting up the git config --global merge.ours.driver true. Makes the merge "omit" the setup.py file (it keeps our file instead). This works only if both master and child branch have changed the file since they firstly parted ways. (it seems)

Deploy files to a directory with a custom structure in Python

I have number of file objects which I would like to redeploy to a directory with a new structure based on requirements stated by the user.
As example could be having these file objects:
1)root\name\date\type\filename
2)root\name\date\type\filename
3)root\name\date\type\filename
...that I want to save (or a copy of them) in a new structure like below after the user defined a need to split type->date->name:
1)root\type\date\name\filename
2)root\type\date\name\filename
3)root\type\date\name\filename
... or even losing levels such as:
1)root\type\filename
2)root\type\filename
3)root\type\filename
I can only come up with the option to go the long way round, taking the initial list and though a process of filtering simply deploy in the new calculated folder structure using basic string operations.
I feel though that someone has probably done this before in a smart way and potentially that a library/module already exists to do this. Does anyone have any ideas?

Here is a solution using Python glob:
The current levels are: name, date, type and filename:
curr_levels = "name\\date\\type\\filename"
curr_levels = curr_levels.split("\\")
The user want other levels: type, date, name and filename:
user_levels = "type\\date\\name\\filename"
user_levels = user_levels.split("\\")
We can use glob.iglob to iterate the tree structure on 4 levels.
The glob pattern is something like: <src_dir>\*\*\*\* (but, we use a more generic way here).
The user structure can be defined with a simple string format.
For instance: {type}\{date}\{name}\{filename} on Windows.
Wee need to create the directory structure first and then copy (or move) the file.
pattern = os.path.join(source_dir, *("*" * len(curr_levels)))
fmt = os.sep.join(['{{{key}}}'.format(key=key) for key in user_levels])
for source_path in glob.iglob(pattern):
source_relpath = os.path.relpath(source_path, source_dir)
parts = source_relpath.split(os.sep)
values = dict(zip(curr_levels, parts))
target_relpath = fmt.format(**values)
target_path = os.path.join(target_dir, target_relpath)
parent_dir = os.path.dirname(target_path)
if not os.path.exists(parent_dir):
os.makedirs(parent_dir)
shutil.copy2(source_path, target_path)
Note: if your source_dir is the same as target_dir (the root in your question), you need to replace glob.iglob by glob.glob in order to store the whole list of files in memory before processing. This is required to avoid glob.iglob browse the directory tree you are creating…

If you are in UNIX environment, the simpler way to achieve this will be using shell script with cp command.
For example, for copying all files from: /root/name/date/type/filename as /root/date/filename; you need to just do:
cp /root/*/date/*/filename /root/date/filename
OR, if you want to move the file, use mv command:
mv /root/*/date/*/filename /root/date/filename
You may run these commands via Python as well using os.system() as:
import os
os.system("cp \root\*\date\*\filename root\date\filename")
For details, check: Calling an external command in Python
Edit based on comment. For copying /root/name/date/type/filename into /root/date/name/type/filename, you need to just do:
cp /root/name/date/type/filename /root/date/name/type/filename
But make sure that directory /root/date/name/type exists before doing it. In order to make sure it exists, and if not create a directory by using mkdir with -p option as:
mkdir -p /root/date/name/type

Running python script with new directories

I have recently begun working on a new computer. All my python files and my data are in the dropbox folder, so having access to the data is not a problem. However, the "user" name on the file has changed. Thus, none of my os.chdir() operations work. Obviously, I can modify all of my scripts using a find and replace, but that won't help if I try using my old computer.
Currently, all the directories called look something like this:
"C:\Users\Old_Username\Dropbox\Path"
and the files I want to access on the new computer look like:
"C:\Users\New_Username\Dropbox\Path"
Is there some sort of try/except I can build into my script so it goes through the various path-name options if the first attempt doesn't work?
Thanks!

Any solution will involve editing your code; so if you are going to edit it anyway - its best to make it generic enough so it works on all platforms.
In the answer to How can I get the Dropbox folder location programmatically in Python? there is a code snippet that you can use if this problem is limited to dropbox.
For a more generic solution, you can use environment variables to figure out the home directory of a user.
On Windows the home directory is location is stored in %UserProfile%, on Linux and OSX it is in $HOME. Luckily Python will take care of all this for you with os.path.expanduser:
import os
home_dir = os.path.expanduser('~')
Using home_dir will ensure that the same path is resolved on all systems.

Thought the file sq.py with these codes(your olds):
C:/Users/Old_Username/Dropbox/Path
for x in range:
#something
def Something():
#something...
C:/Users/Old_Username/Dropbox/Path
Then a new .py file run these codes:
with open("sq.py","r") as f:
for x in f.readlines():
y=x
if re.findall("C:/Users/Old_Username/Dropbox/Path",x) == ['C:/Users/Old_Username/Dropbox/Path']:
x="C:/Users/New_Username/Dropbox/Path"
y=y.replace(y,x)
print (y)
Output is:
C:/Users/New_Username/Dropbox/Path
for x in range:
#something
def Something():
#something...
C:/Users/New_Username/Dropbox/Path
Hope its your solution at least can give you some idea dealing with your problem.

Knowing that eventually I will move or rename my projects or scripts, I always use this code right at the beginning:
import os, inspect
this_dir = os.path.dirname(os.path.abspath(inspect.getfile(inspect.currentframe())))
this_script = inspect.stack()[0][1]
this_script_name = this_script.split('/')[-1]
If you call your script not with the full but a relative path, then this_script will also not contain a full path. this_dir however will always be the full path to the directory.