We Keep Coding

Run Python Script From Script Directory/Current Directory

[Introduction]
Hi! I'm working on a python script to automate installation of UWP-Apps, it's been a long time i'm not touching Python; until this day. The script uses Depedencies inside the script directory, i've looking up on my older scripts and found this specific code.
os.chdir(os.path.dirname(sys.argv[0]))
[Problematic]
However, using the above code doesn't work on my current script but it's working fine on older scripts. When using above, it shows:
OSError: [WinError 123] The filename, directory name, or volume label syntax is incorrect: ''
Already looking up on Internet about this topic; but most of them was talking about running the script from outer/different directory that leads me to dead end.
Any helps is appreciated :)

The easiest answer is probably to change your working directory, then call the .py file from where it is:
cd path/to/python/file && python ../.py
Of course you might find it even easier to write a script that does it all for you, like so:
Save this as runPython.sh in the directory where you're running the python script from, is:
#!/bin/sh
cd path/to/python/file
python ../script.py
Make it executable (for yourself):
chmod +x ./runPython.sh
Then you can simply enter your directory and run it:
./runPython.sh
If you want to only make changes to the python script:
mydir = os.getcwd() # would be the folder where you're running the file
mydir_tmp = "path/to/python/file" # get the path
mydir_new = os.chdir(mydir_tmp) # change the current working directory
mydir = os.getcwd()
The reason you got an error was because sys.argv[0] is the name of the file itself, instead you can pass the directory as a string and use sys.argv[1] instead.

import os
from os.path import abspath, dirname
os.chdir(dirname(abspath(__file__)))
You can use dirname(abspath(__file__))) to get the parent directory of the python script and os.chdir into it to make the script run in the directory where it is located.

python - error occurs on relative path but not absolute path in macOS

I'm having a strange issue. I'm just trying to do a basic show directory contents for relative path.
Created a test directory on the desktop
test directory contents
test.py
test1 -- folder
sample.txt
contents of test.py
import os
dataDir = "test1"
for file in os.listdir("/Users/username/Desktop/test"):
print(file)
Summary - absolute path
works - in visual studio code
works - macOS terminal python3 /Users/username/Desktop/test/test.py
however when use the variable I get an error:
contents of test.py
import os
dataDir = "test1"
for file in os.listdir(dataDir):
print(file)
Summary - relative path
works - in visual studio code
ERROR - macOS terminal python3 /Users/username/Desktop/test/test.py
Traceback (most recent call last):
File "/Users/username/Desktop/test/test.py", line 4, in
for file in os.listdir(dataDir):
FileNotFoundError: [Errno 2] No such file or directory: 'test1'

I all depends on what folder you are in when you launch the code.
Let's say your .py file is in folder1/folder2/file.py and your test1 folder is folder1/folder2/test1/.... You open a terminal in the folder1 and launch python3 /folder2/file.py. The program is going to check for files in folder1/test1/.
Try navigating to the actual file folder, that should work. If you want to use it wherever you launch the code you will have to do some checks on the current directory and manually point the code to the wanted path.

The following solution is inspired by that answer
A simple way to solve your problem is to create the absolute path. (I recommend using that always when you're working with different directories and files)
First of all, you need to care about your actual working directory. While using VS Code your working directory is in your desired directory (/Users/username/Desktop/test/).
But if you use the command line your actual working directory may change, depending where you're calling the script from.
To get the path where script is actually located, you can use the python variable __file__. __file__ is the full path to the directory where your script is located.
To use your script correctly and being able to call it using both ways, the following implementation can help you:
import os
dataDir = "test1"
# absolute path to the directory where your script is in
scriptDir = os.path.dirname(__file__)
# combining the path of your script and the 'searching' directory
absolutePath = os.path.join(scriptDir, dataDir)
for file in os.listdir(absolutePath):
print(file)

'No such file or directory' error when using nosetests

I have a little Python project of which I recently made a Conda package. Making the package was a pain on its own, however, I recently started adding tests to this using nosetests, which made it even harder.
To introduce: my package takes a certain input, performs a lot of (quantum chemical) calculations and then generates a folder in the same directory as the script which calls the package, containing the output data (some .png files, .txt files and binary files)
Using nosetests, I would like to check whether these output files are how they should be. I created a Python test script (using unittest) which creates the input and calls my package. Next, it imports the created file and the test file. However, this is where it goes wrong. I get the error that this file does not exist:
FileNotFoundError: [Errno 2] No such file or directory: 'results\\output.txt'
The directory looks like this:
project-path\
- tests\
- test-script.py
- results\
- output.txt
I call nose by running this in Anaconda prompt:
project-path> nosetests tests
And I import the file in the Python test script using:
result_file = open('results\\output.txt', 'r').read()
Does anyone know what goes wrong here? I think it has to do with the fact that the tests are executed in a test environment. In that case: how do I import my files?

Get the absolute path to output.txt, it is indeed the most reliable way to locate and open it.
import os, sys
basedir = os.path.dirname(sys.argv[0])
filename = "output.txt"
path = os.path.join(basedir, "results", filename)
result_file = open(path, 'r').read()

Executing .py file from Command Prompt raises "ImportError: no module named geopandas"- Script works in Spyder (using anaconda)

I have a python script that accomplishes a few small tasks:
Create new directory structure
Download a .zip file from a URL and unzip contents
Clean up the data
Export data as a .csv
The full .py file runs successfully giving desired output when in Spyder, but when trying to run the .py from Command Prompt, it raises "ImportError: no module named geopandas"
I am using Windows 10 Enterprise version 1909, conda v4.9.2, Anaconda command line client v 1.7.2, Spyder 4.2.3.
I am in a virtual environment with all the needed packages that my script imports.
The first part of my script only needs os and requests packages, and it runs fine as its own .py file from Command Prompt:
import os
import requests
#setup folders, download .zip file and unzip it
#working directory is directory the .py file is in
wd = os.path.dirname(__file__)
if not os.path.exists(wd):
os.mkdir(wd)
#data source directory
src_path = os.path.join(wd, "src")
if not os.path.exists(src_path):
os.mkdir(src_path)
#data output directory
output_path = os.path.join(wd,"output")
if not os.path.exists(output_path):
os.mkdir(output_path)
#create new output directories and define as variables
out_parent = os.path.join(wd, "output")
if not os.path.exists(out_parent):
os.mkdir(out_parent)
folders = ["imgs", "eruptions_processed"]
for folder in folders:
new_dir = os.path.join(out_parent, folder)
if not os.path.exists(new_dir):
os.mkdir(new_dir)
output_imgs = os.path.join(out_parent, "imgs")
if not os.path.exists(output_imgs):
os.mkdir(output_imgs)
output_eruptions = os.path.join(out_parent, "eruptions_processed")
if not os.path.exists(output_eruptions):
os.mkdir(output_eruptions)
if not os.path.exists(os.path.join(src_path,"Historical_Significant_Volcanic_Eruption_Locations.zip")):
url = 'https://opendata.arcgis.com/datasets/3ed5925b69db4374aec43a054b444214_6.zip?outSR=%7B%22latestWkid%22%3A3857%2C%22wkid%22%3A102100%7D'
doc = requests.get(url)
os.chdir(src_path) #change working directory to src folder
with open('Historical_Significant_Volcanic_Eruption_Locations.zip', 'wb') as f:
f.write(doc.content)
file = os.path.join(src_path,"Historical_Significant_Volcanic_Eruption_Locations.zip") #full file path of downloaded
But once I re-introduce my full list of packages in the .py file:
import os
import pandas as pd
import geopandas as gpd
import requests
import datetime
import shutil
and run again from Command Prompt, I get:
Traceback (most recent call last):
File "C:\Users\KWOODW01\py_command_line_tools\download_eruptions.py", line 17, in <module>
import geopandas as gpd
ImportError: No module named geopandas
I am thinking the problem is something to do with not finding my installed packages in my anaconda virtual environment, but I don't have a firm grasp on how to troubleshoot that. I thought I had added the necessary Anaconda file paths to my Windows PATH variable before.
The path to my virtual environment packages are in
"C:\Users\KWOODW01\Anaconda3\envs\pygis\Lib\site-packages"
echo %PATH% returns:
C:\Users\KWOODW01\Anaconda3\envs\pygis;C:\Users\KWOODW01\Anaconda3\envs\pygis\Library\mingw-w64\bin;C:\Users\KWOODW01\Anaconda3\envs\pygis\Library\usr\bin;C:\Users\KWOODW01\Anaconda3\envs\pygis\Library\bin;C:\Users\KWOODW01\Anaconda3\envs\pygis\Scripts;C:\Users\KWOODW01\Anaconda3\envs\pygis\bin;C:\Program Files (x86)\Common Files\Oracle\Java\javapath;C:\WINDOWS\system32;C:\WINDOWS;C:\WINDOWS\System32\Wbem;C:\WINDOWS\System32\WindowsPowerShell\v1.0;C:\WINDOWS\System32\OpenSSH;C:\Program Files\McAfee\Solidcore\Tools\GatherInfo;C:\Program Files\McAfee\Solidcore\Tools\Scanalyzer;C:\Program Files\McAfee\Solidcore;C:\Program Files\McAfee\Solidcore\Tools\ScGetCerts;C:\Users\KWOODW01\AppData\Local\Microsoft\WindowsApps;C:\Users\KWOODW01\Anaconda3\Library\bin;C:\Users\KWOODW01\Anaconda3\Scripts;C:\Users\KWOODW01\Anaconda3\condabin;C:\Users\KWOODW01\Anaconda3;.
So it appears that the path to the directory where my pygis venv packages live are already added to my PATH variables, yet from Command Prompt the script still raises the "ImportError: no module named geopandas". Pretty stuck on this one. Hoping someone can provide some more troubleshooting tips. Thanks.

I figured out I wasn't calling python in command prompt before executing the python file.
The proper command is python modulename.py instead of modulename.py if you want to execute a .py file from the command prompt. Yikes. Let this be a lesson for other python novices.

Python directory file go wrong

I am having a silly problem with reading a file in python.
I have a folder 'a' that contains my test.py and file test.json for reading.
My test.py looks like this:
config_path = 'test.json'
with open(config_path) as config_buffer:
config = json.loads(config_buffer.read())
When I go outside the directory structure of folder 'a' and I run the command:
python a\test.py
And the console returns this error:
FileNotFoundError: No such file or directory: 'test.json'
I try using the absolute file path using pathlib:
config_path = Path('end2end.json').absolute()
with open(config_path) as config_buffer:
config = json.loads(config_buffer.read())
But it still returns this error to me:
FileNotFoundError: No such file or directory: 'D:\\test.json'
Can anyone help me to get exactly the right directory file?

If you want to call your python script from any folder and let it access its local files without specifying paths, you can change the directory in the begining:
import os
os.chdir(os.path.dirname(os.path.realpath(__file__)))
config_path = 'test.json'
...

The Problem is that you should place the test.json file in the current working directory.
i.e)For Example, The python file is located in C:\users\Desktop\Code\test.py then you should copy the file into C:\users\Desktop\Code\
(OR)
You should give the path of the file in with open($PATH TO JSON FILE) as config_buffer