I am trying to create a simple package for my python code. Here is the structure I have created:
.test_dive_software
| measurements
| mytxt.txt
| mycode
| __init__.py
| mycode.py
| README.txt
| setup.py
Now, the mycode.py file when executed needs to access the mytxt.txt file to load some data.
The process I am doing to build the package is pip install . and then I run the command I have set (mycode-exec) in the setup.py file as the entry point (I am using setuptools).
My problem is this:
In order to access the txt file from mycode.py, I use the command path=str(os.getcwd()).
When I try to run my code (after I have built the package) by double clicking the mycode.py icon, the path=str(os.getcwd()) command returns the .test_dive_software\mycode directory, from where I can access the measurements folder.
When I execute my program as described above, through the mycode-exec command, the path=str(os.getcwd()) returns the current directory I happen to be in, and not that of the mycode.py file.
Is there a way to achieve consistency in my program, so that it can work fine, no matter how it is executed?
Related
Just as reference imagine the following project structure:
/testapp
| __main__.py
| firstclass.py
| secondclass.py
To make installation and deploy easier I decided to bundle these files using the trick below:
zip the .py files
echo '#!/usr/bin/env python3' | cat - testapp.zip > testapp
The issue now is that "firstclass.py" needs the abspath of "testapp".
If I run this bundled testapp and asks for __file__ it returns the script path, for example the path for "firstclass.py" not "testapp" (the executable).
I also tried using sys.executable, it returns the path for "/usr/bin/python3".
Is there an easy way to get the abspath of "testapp"?
I only care for linux environments.
Also the whole point of this is due to the fact that I cannot rely on the name of "testapp", final installer can change that name and I need a reference to it.
Thanks.
lately i've started using pdoc3 to document my whole project,
right now i have a problem generating a HTML that contain the whole project in 1 command, and not going every.py file one by one
the project structure is like this:
--pythonProject
| file_a.py
| file_b.py
--folder
| file_c.py
| file_d.py
--docs
i tried the following command : pdoc --html .\file_a.py --output-dir docs --force and it genetrates the following file in docs folder -> file_a.html
and when using this pdoc --html .\ --output-dir docs --force i receive error for pytest library that not even in the project
i wanted to know if there is a command to generate the whole project (im using PyCharm)
thanks alot :)
Context
While trying to generate type hints for all python files in some project named projectname, I am experiencing some difficulties applying the stubs. The project directory is:
projectname/
|-- src/
| |-- projectname/__main__.py
| |-- projectname/some_folder/some_script.py
|-- tests/
| |-- projectname/test_something.py
| |-- projectname/another_folder/test_another_thing.py
|
|-- setup.py
|-- README.md
The main code is executed with:
python -m src.projectname
and the tests are executed with:
python -m pytest
Output Attempt I
Based on this issue, I tried running:
monkeytype run src/snncompare/__main__.py
Which generates the monkeytype.sqlite3 file in the root directory of the project. However, when I try to apply the generated type hints with:
monkeytype stub src.snncompare.__main__.py
monkeytype apply src.snncompare.__main__.py
or:
monkeytype stub src.snncompare.__main__
monkeytype apply src.snncompare.__main__
I get:
No traces found for module src.projectname.__main__
No traces found for module src.projectname.__main__.py
respectively. And for:
monkeytype apply src.snncompare
It says:
No traces found for module src.snncompare
Output Attempt II
Based on this issue, in which it says one should apply the stub to the "modulename" I tried running:
monkeytype run src/projectname/some_folder/some_script.py
monkeytype run src/projectname.some_folder.some_script.py
monkeytype run src/projectname.some_folder.some_script
monkeytype run some_script.py
monkeytype run some_script
from the root directory of the script, and they all produce the same error. I do not yet know exactly how to determine what the "modulename" of my project is.
Attempt III
Based on this answer, I think I should create an additional file that calls the __main__.py file and executes its code, to be able to generate stubs for it, or the modules imported by it. However, that would seem in conflict with the quote:
We don't collect traces for main because it could be different on subsequent runs and it would be confusing to look up such traces.
Question
How can I apply the generated stubs to all the (touched) .py files in the project?
Misunderstanding
I think the key misunderstanding I had can be clarified with:
MonkeyType currently does not automatically create typings for a Python project.
Issue
As described in the issues and the readme, monkey type only generates type hints for the some.py files that a main.py file imports and uses. Even though I am relatively confident that a large fraction of my files are used when I run __main__.py, most of them are not directly imported by __main__.py. Instead, they are imported by the code that is imported by my __main__.py.
Manual-"Solution"
To "automatically"/semi-manually generate the type hints, you need to write another_python.py file that:
imports each other.py file for which you want to automatically create type-hints.
Calls each function in every other.py file for which you want to automatically create type-hints.
Bit of automation
Once could at least automate applying the stubs that you did find by walking over the directories and trying to apply the stubs if they exists:
monkeytype run src/projectname/__main__.py
# List all .py files in the project:
for f in $(find src/ -name '*.py'); do echo $f; done
# List all .py files in the project with `.` instead of `/`:
for f in $(find src/ -name '*.py'); do echo ${f//\//.}; done
# Apply monkeytype type hints to each file for which they were found.
for f in $(find src/ -name '*.py'); do monkeytype apply ${f//\//.}; done
Additional Issues
I had a file src/projectname/some_file.py which contained some_function(): which was definitely called by __main__.py yet no stubs like: some_function() -> None: were generated for this file. In total 1 out of 50 files were changed. I retried auto generating the stubs using the tests I wrote, with pyannotate by dropbox and that worked more effectively.
I have a python script for interacting with aws secrets manager (aws_secrets) that I am calling in the program_to_run.py file.
Inside of program_to_run.py I have
from aws_secrets import aws_read_secrets
In pycharm it finds this successfully, below is the file structure.
+-- aws_secrets
| +-- aws_secrets2.py
+-- folder
| +-- program_to_run.py
I am trying to run program_to_run.py in Jenkins using a shell script.
python3 root/folder/program_to_run.py
But when I run it I get this
ImportError: cannot import name 'aws_secrets2' from 'aws_secrets' (/home/jenkins-agent/miniconda3/lib/python3.7/site-packages/aws_secrets/init.py)
So my question is, how do I correctly call aws_secrets2.py from program_to_run.py using a shell script?
I've noticed that Jenkins seems to have trouble referencing packages if they were installed on the machine from outside (as opposed to within Jenkins). So try adding these two lines to the pipeline before calling python3 root/folder/program_to_run.py
pip uninstall aws-secrets
pip install aws-secrets
I have the following folder structure for my Python project (in Pycharm):
Tool
|___Script
|___logs
|___ tests
| |___setA
| |___setB
| |___setC
| |___testSetC.py
|___ __init__.py
|______script.py
and I'm trying to import methods defined in script.py in testSetC.py.
I have from Script.script import * at the top of testSetC.pyand the script runs without any issues inside Pycharm. However, when I tried to run the same from the command line, it failed with the error message ModuleNotFoundError: No module named 'Script'.
I poked around and found that Pycharm's run configuration automatically adds the Tool directory to PYTHONPATH essentially masking the error.
What is the correct way to do this without manually amending sys.path in every test script? It looks hackish IMO.