I am working on a project in PyCharm that involves extensive computations, with long runtimes.
I would like to do the following: I come up with a version of my code; then run it, then I edit the code some more; however, the run I started before still only uses the old version of the code (i.e. the snapshot at the point of running).
Is this possible in PyCharm?
I run my project by selecting the Run 'projectname' option from the Run menu.
I understand the run works by pre-compling the .py files to .pyc files stored in the __pycache__ folder. However, I don't know the following.
Will saving the file in PyCharm cause the .pyc files to be replaced by new versions? This is something I want to avoid since I want one run to only use one snapshot of the source tree, not multiple versions at different points of execution.
What if some python class is only needed, say, 20 minutes after the run has started. Will the .pyc file be created at the beginning of the run, or on-demand (where the corresponding .py file might already have changed)?
I use PyCharm in my classes. My experience is that the all the required code, including the imported modules, are compiled at runtime. If you change anything in that suite you need to start running from scratch for it to take effect.
I'm not a professional programmer so my experience is with small apps. I'd love to hear form an expert.
Related
I have a question for you that is making me crazy.
I am working on my python package. I'm building my package using hatchling and pyproject.toml and I have also three GUI targets.
To make them I'm using the project.gui-scripts section in the project files and indeed, three exe files are included in the scripts directory after the package installation.
All of a sudden two out of three exe files are not starting at all. If you execute it from CMD, you don't get any error message, nothing. I said all of a sudden because in the past they all used to work.
The third one, instead is still working nicely.
Do you know if there is anything I can do to check what's going wrong? Can I debug those exe files?
Thanks in advance,
After 3 intensive hour, I was testing my script on terminal. However, my editor messed up and it overwrote my script when it was being still executed on terminal. Well, I didn't terminate running script, so I was wondering that does python interpreter keep the currently running file in a temporary folder or somewhere else so that I can recover my script?
Python tries to cache your .pyc files. How that's done has changed over time (see PEP 3147 -- PYC Repository Directories. Top level scripts are not cached but imported ones are. So, you may not have one.
.pyc files are compiled byte codes so its not just a question of renaming it .py and you can't figure them out just by looking at them. There are decompilers out there like the one refenced here:
Decompiling .pyc files.
Personally, I create mercurial repos for my scripts and check them in frequently.... because I've made a similar mistake a time or two. git, svn, and etc... are other popular tools for maintaining repos.
Depending on your operating system and editor, you may have a copy in Trash or even saved by the editor. You may also be able to "roll back" the file system.
If you're running Linux, you may still be able to find a handle to open files in the /proc/ directory if the process is still running. This handle will keep the file from being deleted. Details see: https://superuser.com/questions/283102/how-to-recover-deleted-file-if-it-is-still-opened-by-some-process
Say I run a Python (2.7, though I'm not sure that makes a difference here) script. Instead of terminating the script, I tab out, or somehow switch back to my editing environment. I can then modify the script and save it, but this changes nothing in the still-running script.
Does Python load all source files into memory completely at launch? I am under the impression that this is how the Python interpreter works, but this contradicts my other views of the Python interpreter: I have heard that .pyc files serve as byte-code for Python's virtual machine, like .class files in Java. At the same time however, some (very few in my understanding) implementations of Python also use just-in-time compilation techniques.
So am I correct in thinking that if I make a change to a .py file while my script is running, I don't see that change until I re-run the script, because at launch all necessary .py files are compiled into .pyc files, and simply modifying the .py files does not remake the .pyc files?
If that is correct, then why don't huge programs, like the one I'm working on with ~6,550 kilobytes of source code distributed over 20+ .py files, take forever to compile at startup? How is the program itself so fast?
Additional Info:
I am not using third-party modules. All of the files have been written locally. The main source file is relatively small (10 kB), but the source file I primarily work on is 65 kB. It was also written locally and changes every time before launch.
Python loads the main script into memory, compiles it into bytecode and runs that. If you modify the source file in the meantime, you're not affecting the bytecode.
If you're running the script as the main script (i. e. by calling it like python myfile.py, then the bytecode will be discarded when the script exits.
If you're importing the script, however, then the bytecode will be written to disk as a .pyc file which won't be recompiled when imported again, unless you modify the corresponding .py file.
Your big 6.5 MB program consists of many modules which are imported by the (probably small) main script, so only that will have to be compiled at each run. All the other files will have their .pyc file ready to run.
First of all, you are indeed correct in your understanding that changes to a Python source file aren't seen by the interpreter until the next run. There are some debugging systems, usually built for proprietary purposes, that allow you to reload modules, but this bring attendant complexities such as existing objects retaining references to code from the old module, for example. It can get really ugly, though.
The reason huge programs start up so quickly is the the interpreter tries to create a .pyc file for every .py file it imports if either no corresponding .pyc file exists or if the .py is newer. The .pyc is indeed the program compiled into byte code, so it's relatively quick to load.
As far as JIT compilation goes you may be thinking of the PyPy implementation, which is written in Python and has backends in several different languages. It's increasingly being used in Python 2 shops where execution speed is important, but it's along way from the CPython that we all know and love.
I have developed an application for a friend. Aplication is not that complex, involves only two .py files, main.py and main_web.py, main being the application code, and _web being the web interface for it. As the web was done later, it's kept in this format, I know it can be done with one app but not to complicate it too much, I kept it that way. Two two communicate with some files, and web part uses Flask so there's "templates" directory too.
Now, I want to make a package or somehow make this easier for distribution, on a OSX system. I see that there is a nice py2app thingy, but I am running Windows and I can't really use it since it won't work on Win. I also don't know will py2app make problems since some configs are in text files in the directory, and they change during the runtime.
So, I am wondering, is there any other way to make a package of this, some sort of setup like program, or maybe some script or something? Some simple "way" of doing this would be to just copy the files in the directory in the "Documents", and add some shortcuts to the desktop to run those two apps, and that would be it, no need for anything else. DMG would be fine, but not mandatory.
I believe what you are looking for is to add: #!/usr/bin/python to the first line of your code will allow your friend to just double click on the file and it should open. Just as a warning osx does not tell us what version and as such what version of python and what standard libraries are going to be present.
Also, just make sure that if they have played around with their settings to much and they double click on python it does not work they will have to choose to open the file in "terminal.app" in the Utilities Applications folder (/Applications/Utilities/terminal.app)
The other idea is borrow a mac and compile it with the py2app program that you already mentioned. Otherwise there is no generic binary file that you will be able to compile in windows and have run on mac.
I have a python project developed in eclipse. Independantly there are various data directories in various locations and it is desirable to operate on each with a different main.py that imports the python project. If I store a main.py with my data, how can I open it in eclipse and run the debugger?
I can run ipython in each directory without a problem; but, I am not clear on how to move between different main.py files within eclipse. The run configuration does allow you to go outside the project and I would prefer not having to change this each time anyway. Any insight would be great!
You could do this by setting up each main.py + data directory as its own Eclipse project, each with a custom run/debug configuration. You do have extra projects, but they don't require any ongoing maintenance.
After the initial setup, you continue work on the real python project and debugging any given data directory consists of selecting the appropriate project and invoking the debugger. Not maximally elegant, but not bad.
If you're truly loath to make new projects, you could bring the main.py files into the project and store the location of the associated data directory inside each of them. Each main begins by changing to the data directory, and then runs as usual. In eclipse you set up multiple run configurations, one for each main.py which you can then run without any extra tinkering.
It is inelegant and fragile to keep the data directory information inside each main.py, but maybe better for your purposes.