Using the Google App Engine to develop in python yesterday it stopped running the current version of the script.
Instead of executing the most recent version it seems to run the previously pre-compiled .pyc even if the .py source was changed.
Error messages actually quotes the correct line from the most current source. Except if the position of the line changed, then it quotes the line which is in the place where the error occurred previously.
Deleting .pyc files causes them to be recreated from the current version. Deleting all .pycs is a poor workaround for now.
How can I get to the root cause of the problem?
Did you check your system clock? I believe python determine whether to use the .pyc or .py based on timestamps. If your system clock got pushed back, then it would see the .pyc files as newer until the system clock caught up to the last time they were built.
Are you editing the .py files on a different system than where they are being compiled ?
The compiler recompiles the .py files if its modification date is newer than the modification date of the .pyc file.
The fact that it is picking the .pyc file for use points to the fact that your .py file has an older modification date. This is only possible if your .py file is being modified on a different system and then being copied to the one where it is to be used and the editing environment/system's clock is set behind the runtime enviroment/system's clock.
The following steps solved the issue temporarily:
Delete GoogleAppEngineLauncher from your Applications folder.
Rename the file ~/Library/Application Support/GoogleAppEngineLauncher/Projects.plist (e.g. Project.plist.backup
Rename the file ~/Library/Preferences/com.google.GoogleAppEngineLauncher.plist (e.g. com.google.GoogleAppEngineLauncher.plist.backup)
Download and install Google App Engine Launcher again.
Use "File", "Add existing application…" to add your projects again, do not forget to set any flags you had set before.
Alternatively it might even work starting GAEL once, closing it and putting your backed up preference files back into place as to avoid having to reconfigure.
Edit: Turns out that fixes it… temporarily. Not exactly a very easy issue to debug.
Weirdly enough it works when running the appserver from the command line, such as
dev_appserver.py testproject/ -p 8082 --debug
Related
This answer tells me that a .pyc file gets created when a .py file is run, which I understand saves loading time when re-run. Which makes me wonder what the point of the .py file is after the .pyc is created.
When backing up my code, or sharing it, I don't want to include redundant or extraneous files. Which filetype should I focus on?
Side question: I have one script that calls another. After running them, the called script got a .pyc file written, but the master script that does the calling did not. Why would that be?
Python .pyc files are generated when a module is imported, not when a top level script is run. I'm not sure what you mean by calling, but if you ran your master script from the command line and it imported the other script, then only the imported one gets a .pyc.
As for distributing .pyc files, they are minor version sensitive. If you bundle your own python or distribute multiple python-version sensitive files, then maybe. But best practice is to distribute the .py files.
Python's script and module rules seem a bit odd until you consider its installation model. A common installation model is that executables are installed somewhere on the system's PATH and shared libraries are installed somewhere in a library path.
Python's setup.py does the same thing. Top level scripts go on the PATH but modules and packages go in an library path. For instance on my system, pdb3 (a top level script) is at /usr/bin/pdb3 and os (an imported module) is at /usr/lib/python3.4/os.py. Suppose python compiled pdb3 to pdb3.pyc. Well, I'd still call pdb3 and the .pyc is useless. So why clutter the path?
Its common for installs to run as root or administrator so you have write access on those paths. But you wouldn't have write access to them later as a regular user. You can have setup.py generate .pyc files during install. You get the right .pyc files for whatever python you happen to have, and since you are running as root/admin during install you still have acess to the directories. Trying to build .pyc files later is a problem because a regular user doesn't have access to the directories.
So, best practice is to distribute .py files and have setup.py build the .pyc during install.
If you simply want to run your Python script, all you really need is .pyc which is the bytecode generated from your source code. See here for details on running a .pyc file. I will warn that some of the detials are bit twisty.
However I recommend including your source code and leaving out your .pyc files as they are generated automatically by the Python Interpreter. Besides, if you, or another person would want to revise/revisit your source code at a later point, you would need the .py files. Furthermore, it is usually best practice to just include your source code.
After 3 intensive hour, I was testing my script on terminal. However, my editor messed up and it overwrote my script when it was being still executed on terminal. Well, I didn't terminate running script, so I was wondering that does python interpreter keep the currently running file in a temporary folder or somewhere else so that I can recover my script?
Python tries to cache your .pyc files. How that's done has changed over time (see PEP 3147 -- PYC Repository Directories. Top level scripts are not cached but imported ones are. So, you may not have one.
.pyc files are compiled byte codes so its not just a question of renaming it .py and you can't figure them out just by looking at them. There are decompilers out there like the one refenced here:
Decompiling .pyc files.
Personally, I create mercurial repos for my scripts and check them in frequently.... because I've made a similar mistake a time or two. git, svn, and etc... are other popular tools for maintaining repos.
Depending on your operating system and editor, you may have a copy in Trash or even saved by the editor. You may also be able to "roll back" the file system.
If you're running Linux, you may still be able to find a handle to open files in the /proc/ directory if the process is still running. This handle will keep the file from being deleted. Details see: https://superuser.com/questions/283102/how-to-recover-deleted-file-if-it-is-still-opened-by-some-process
I am working on a project in PyCharm that involves extensive computations, with long runtimes.
I would like to do the following: I come up with a version of my code; then run it, then I edit the code some more; however, the run I started before still only uses the old version of the code (i.e. the snapshot at the point of running).
Is this possible in PyCharm?
I run my project by selecting the Run 'projectname' option from the Run menu.
I understand the run works by pre-compling the .py files to .pyc files stored in the __pycache__ folder. However, I don't know the following.
Will saving the file in PyCharm cause the .pyc files to be replaced by new versions? This is something I want to avoid since I want one run to only use one snapshot of the source tree, not multiple versions at different points of execution.
What if some python class is only needed, say, 20 minutes after the run has started. Will the .pyc file be created at the beginning of the run, or on-demand (where the corresponding .py file might already have changed)?
I use PyCharm in my classes. My experience is that the all the required code, including the imported modules, are compiled at runtime. If you change anything in that suite you need to start running from scratch for it to take effect.
I'm not a professional programmer so my experience is with small apps. I'd love to hear form an expert.
I'm trying to change some code in a production server, but when I test the new code it's not being executed. I think the issue is that when I upload the updated source file, it is not being compiled into a .pyc file.
-rw-r--r-- 1 ubuntu ubuntu 47872 Jul 13 04:39 admin_email.py
-rw-r--r-- 1 root root 48212 Feb 10 03:12 admin_email.pyc
As you can see from the timestamps above, the .pyc file has not been updated.
Am I correct in assuming the reason the changes in admin_email.py are not being applied is because it is not being compiled? If so, can someone please offer a suggestion on how to get it to do so? Could the issue be that I need to restart the server?
Resetting the application probably does the trick. Since you are using gunicorn/nginx I assume you are also supervisord (if this is not the case please say so, then I can update my answer to also add that). In that case you can do sudo suporvisorctl restart <app_name> to restart the application.
Another issue that might be there, as brainless coder and Marius also stated, is that it seems the application was (at least once) ran as root, you should avoid this. You should remove the .pyc files and change the user the application runs under.
The .pyc file is owned by the root user for some reason, and not writeable by other users. Your Python process probably runs as non-root, and can't create new .pyc files.
Either delete the old .pyc files (make a backup first), or chown them to the same user as the process that runs the .py files, and it will probably work.
You can delete it if needed but it's really not. I don't see how that's the reason your code isn't executing.
.pyc files are byte codes that the python interpreter compiles the source to, so as long as you have access to the .py file you're safe to delete.
Are you running the script through shell? Did you give the file executable (+x) permission before doing so?
I am running a web app on python2.7 with mod_wsgi/apache. Everything is fine but I can't find any .pyc files. Do they not get generated with mod_wsgi?
By default apache probably doesn't have any write access to your django app directory which is a good thing security wise.
Now Python will byte recompile your code once every apache restart then cache it in memory.
As it is a longlive process it is ok.
Note: if you really really want to have those pyc, give a write access to your apache user to the source directory.
Note2: This can create a hell lot of confusion when you start with manage.py a test instance shared by apache as this will create those pyc as root and will keep them if you then run apache despite a source code change.
When a module is imported for the first time, or when the source is more recent than the current compiled file, a .pyc file containing the compiled code will usually be created in the same directory as the .py file.
So if you are not importing the module then no files will be created.
Besides this, a .pyc file may not be created is permissions problems with the directory. This can happen, for example, if you develop as one user but run as another, such as if you are testing with a web server. Creation of a .pyc file is automatic if you’re importing a module and Python has the ability (permissions, free space, etc.) to write the compiled module back to the directory.
Note - Running a script is not considered an import and no .pyc will be created.
If you need to create a .pyc file for a module that is not imported, you can use the py_compile and compileall modules.