I'm experiencing an odd error where certain modules can't be imported. When attempting to import them, I get this error.
RuntimeError: unable to get file status from '/path/to/file.py'
So far, all of these unimportable modules are located on a network drive, but other modules also located on the network drive are importable, and so far I've not been able to find a difference between the ones that are and are not importable. All of these modules were working until recently.
A quick search of google doesn't show any similar errors related to "file status". Has anyone experienced this error before?
It appeared that this problem was related to the fact that the bad modules all had invalid file creation times (i.e. stat.st_ctime). I think the problem was likely caused by some backup software that runs in linux that somehow managed to clear the file creation times.
When importing a python module, I believe it compares the creation times from the .py file and the .pyc file, if it exists, to know whether it needs to re-compile a .pyc. In this case, it was breaking when it couldn't find a file creation time for the file.
I made a script that walked through the entire directory tree, reading, deleting, then overwriting every .py file to force a creation time. After that, everything seemed to work as normal again.
Related
Recently I uninstalled python and then reinstalled it to see what affects it would have on my code and ensure that I knew how to fix any path or directory issues. I mostly succeeded but am having trouble within my venvs. Testing out a basic django app I made any of my python manage.py ... code returns the error No Python at '"C:\Python311\python.exe'.
After doing some research it appears that the issue is that within .env\scripts there are two files called "python.exe" and "pythonw.exe" are causing the problem because they are trying to access a python.exe that no longer exists. (Note this is my understanding based off of pythons venv document and I could be incorrect)
My question is what is the best way to deal with this issue. I see two options but am not sure how to accomplish either.
Fully reset the .env folder so it points to the correct python.
Just change those two files so they point to the correct instance of python.
So far the only things I have tested are running code outside of venv's entirely and creating new ones to run the code from. Both work totally fine so it is definitely something from the historic venvs.
I have also looked at this question and I believe the two to be related but could be mistaken. Unfortunately it does not seem an answer was given there.
Hi there wise people of stack. I'm having trouble with importing pyd files as python objects.
The story:
I have an internal repo on gitlab that runs python files as well as C++ files. the repo uses pybind for both languages to speak to one another. The whole project is built with CI/CD and the artefact I have access to are .pyd extension files.
What I was given as a task would be to access some .pyd files (in different folders) with a single python script and access their classes (encoded inside this .pyd file) to mock them using python.
The problem:
What I was told was that I would need a simple include to be able to access the .pyd as an object through python just like you would with a library. However, I came across errors in the whole process. I have gone through this post and this one, but it seems that none of them works for me.
What was tried:
The first thing I did was start a remote folder with a single .pyd file from the project(let's call it SomeClass.pyd). I then created a python file test.py in the same directory as the pyd file.
The whole architecture looks like the following:
|--folder
|--SomeClass.pyd
|--test.py
Then, in the test.py file, I tried running
import SomeClass.pyd
import SomeClass
import SomeClass.pyd as sc
from SomeClass.pyd import *
from SomeClass import *
which all yielded the same following error:
ImportError: dynamic module does not define module export function
Now, I know that pyd files are similar to dlls, but I was told multiple time that a simple import would let me access the object information without needing anything in particular.
I recall reading about adding the PYTHONPATH before launching the whole process. However, I need that file to access the pyd without having to add any variable to the path as I will likely not always have access rights to the PYTHONPATH.
The project is quite big, so I'm trying to keep it bare minimum, but if you need more info, I'll try to give some more.
Thank you for your feedback!
Alright, after some time and a lot of researching, I found the weird answer for the problem that occured. I really hope it will help anyone encountering the same issue.
The problem was caused by the fact that pycharm has sometimes issues with the whole dynamic import.
First problem: dynamic import
This was solved simply by going on pycharm --> files --> invalidate cache and then tick "Clear file system cache and Local History" as well as "Clear VCS Log caches and indexes". You should then be prompted to reboot.
I also add a note that even after fixing the issue, sometime, for no apparent reason, I still have to invalidate cache again.
Second problem: venv
Once rebooted, you might be able to import manually the path to your pyd file, but you probably won't be able to auto complete. What solved this for me was compiling manually the code responsible for the pyd in order to generate a wheel. In my case, I used poetry:
poetry build
Once the wheel was created, I did a manual pip install of the wheel created by the poetry build to install it directly into the venv:
pip install dist/the_name_of_your_wheel_file.whl
These steps were the ones to fix my problem. I hope this will help anyone encountering the same problem!
I am currently writing a python script for a project, but to move on the with the project,
I need to know where all the downloaded files/programs[exe] will go to in Windows.
I know that it will go to download folder of the user's, as long as the user did not change the default location for the downloaded folder, or they did not do save as to a certain location but I am asking if there is a way to locate all the files downloaded regardless of where they are saved?
Any help will be a great help, I have googled but haven't found the answer i was looking for so hoping someone here could provide some insight.
The fact that a file was downloaded once-upon-a-time isn't something you can observe from the filesystem. In other words, no, you can't do this.
However, NTFS does store the fact that a file was JUST downloaded in an alternate data stream (ADS) which you can read. This is how Windows warns you that a file was downloaded from the internet and might be dangerous.
The problem with that is, if the file is ever opened and the user says the file is safe, that data is removed. You can't know that a file was previously downloaded, only if it was downloaded and has never been opened.
If your python script needs to act upon some other files, you should ask the user where the files are, either on the command line or interactively.
There's a few things to consider.
You can always scan the entire drive to find the latest files. It's slow, but possible. That's your worst case scenario.
By extension, you can leverage the Windows file index - Windows Search. This will speed up searching and allow sorting by date, but is still just a faster version of the first option. In other words, it doesn't tell you anything new.
To get the current user's default downloads folder, consider using Windows environment variables, such as: %USERPROFILE%\downloads. This can greatly simplify programmatically finding the current user's folder without having to know their username.
(Edited) I had stated that there was no way to track the origin, but as pointed out in another answer it is possible to tell if it was downloaded using ADS. Specifically, the Zone.Identifier stream signifies that the file came from a different "security zone", i.e. not this computer. Other than that, it doesn't provide details of where it came from, but perhaps that's not important for your use case.
I just committed a major noob blunder while playing around with basic functionalities of NumPy. I created a small file called "numpy.py" just so that play around with problems related to numpy. I could also later come back for reference - and the name would have helped. Obvious error :-
import numpy
File "C:\Users\USERNAME\workspace\StackOverflow\python\numpy.py", line 25, in <module>
AttributeError: 'module' object has no attribute 'random'
I realized the problem - python was considering my current code as actual numpy and looking for random. I deleted that file and created a new one.
And the problem persists. This image explains it better than my words would.
Now there's a completely new file with a different name - but still it's looking for some methods in the deleted file.
I've already tried:
Cleaning the Project
Restarting Eclipse
Run another .py (successfully) and then this one again
Removed all the conflicting files, created a totally new file. Still same. If the program contains import numpy, it would give same error.
I'd like to know (1) What actually happened wrong here? Why is eclipse so confused (2) How to resolve? (3) What are the best practices in such situations?
I'm the OP. Found a solution after a few minutes of posting this question.
Browsed to the folder containing the code. I noticed that there was a .pyc file for the numpy.pyc I had created earlier. Deleted that one and things are working file now.
However I'd still like to know how to avoid situations like this or resolve directly through eclipse. Though it works now!
This is called an orphaned .pyc file. Eclipse/PyDev now has a setting to automatically handle these (as of Oxygen / v4.7):
PyDev > Builders > "How to handle .pyc/$py.class deletion?" :
Delete any orphaned .pyc file
Knowing Eclipse, presumably that action only gets triggered when you or it refreshes or restarts the workspace, or triggers a rebuild. There are some switches to try to make that automatic too. Or Eclipse > Restart if all else fails.
I'm have difficulty finding any docs describing Python's expected symlink behavior.
I've tried testing it myself but I'm getting weird results - sometimes it appears to work for a while then stops working.
Anyway, I have a situation where MoinMoin keeps it's data files and plugin modules in the same directory but I need to keep the data files outside my git repository (I'm using Openshift PaaS) so it persists. So I want to create this symlink:
~/data/plugins/theme/ -> ~/repo/plugins/theme/
Should I expect this to work?
Should work just fine - symlinks are handled on the OS level, meaning as long as python isn't trying to determine if something is a symlink or not, calling an open on it should just open the file it's pointing to.
Is there anything in particular that isn't working? What are you seeing when it stops working?