Strange behavior after deleting *.pyc and rerunning .py script - python

I have three modules as:
one.py:
def abc():
print "Heeeeeeeeeeeiiiiiioooooooooo"
two.py:
import one
def defg():
one.abc()
three.py:
import os
from time import sleep
import two
two.defg()
sleep(20)
directory = os.listdir('.')
for filename in directory:
if filename[-3:] == 'pyc':
print '- ' + filename
os.remove(filename)
I have three doubts.
When I run three.py for the first time one.pyc and two.pyc will be created. I can see it since I gave 20 sec delay. After executing the statement os.remove(filename), they get removed. Until here its fine.
Again without closing the IDLE as well as the script, I ran three.py. This time no .pyc file was created. Why is this so?
If I close IDLE as well as the script, .pyc will be created as before.
Why the compiled code is not getting created again and again?
Also, if I make a change in one.py it will not be shown if I run without closing the shells. I need a solution for this also.
Third doubt is if the compiled code is getting deleted first time itself, then how the second run gives the same result without .pyc?

The .pyc is not getting created again because there is a reference to your imported module in code. When it is re-run, this reference is used.
This is why the .pyc is not generated again, and also why the extra changes you make are not getting run.
You can either remove all references and call a garbage collect, or use the built in reload() function on the modules. e.g.:
reload(three)

I think that IDLE is caching the bytecode within its own Python process, so it doesn't need to regenerate the it each time the file is run.

Edit ~/.bashrc and add this shell function to it
$ cd ; pyclean
pyclean () {
find . -type f -name "*.py[co]" -delete
find . -type d -name "__pycache__" -delete
}

Related

Setting CWDir as a variable for MS-Dos environment

Can someone help me with this please?
I am trying to compile a program in this case programmed in python that I can run in win9Xdos, that I can call/start from a 9xDos batchfile, that will find the Current working Dir & by that I mean identify the cwd (current working directory) from where the python program and batchfile are executed. The python program will be called getcwd.py which I am hoping someone will outline what I need to do to convert to EXE/COM file. there is a program called Py2EXE but not sure if this will compile for Ms-dos file. Anyways heres my simple code thus far. Can someone tell me if I am on the right track please? Oh by the way what I am trying to do is find the CWD & inject the resultant path into a variable that can be read from 9Xdos. The current Var would be %cwd%
# importing os module
import os
# some websites say use: del cwd (to remove variable if it exists)
cwd = none
cwd = os.getcwd()
print(cwd)
The print line may need interchanging with code below, not sure help needed:
print(type(path))
# <class 'str'>
would the above code work, say in the root e.g. C:\ with & work in obtaining the CWD variable & if correct how would I proceed to compile it to an exe/com file? do I need to take into account LFN's & Spaces between possible paths e.g C:\Program Files & possible backslashes etc?
Your code isn't perfect but it is on the right track. All you need is this:
import os
if __name__ == '__main__':
print(os.getcwd())
There is no need for an auxiliary variable, and I don't know what websites are recommending that you delete the variable before creating it. Trying to delete a nonexistent Python variable is a runtime error. So I would stay away from those websites.
But your question is about setting an environment variable. Calling print() won't do that. All it will do is echo the current working directory to the console. There is no way to change the environment of a running process that will affect the parent process. This is not a Python restriction nor a Windows restriction: it is quite general. The OS sets up the environment of the process when it creates the process. You can make changes to the environment (using os.environ[env-var]) but those changes will only be visible inside that Python process and will not be visible to the environment of the batch file that runs the Python program. To do that, you need to pass the value back to the calling process.
One way to do that is like this:
In Python:
import os
if __name__ == '__main__':
print(f"set CWDIR={os.getcwd()}", file=open("mycd.bat","w"))
I haven't had a Python 1.5.2 environment for 15 years, so I can't test this, but I think the equivalent would have been
if __name__ == '__main__':
print >> open("mycd.bat","w"), "set CWDIR=%s" % (os.getcwd(),)
In a cmd.exe console:
call mycd.bat
Though if your Win9XDos doesn't provide %cd% (which, as far as I recall, was available in MS-DOS 5, or maybe further back still) there is no way of telling if it supports call either. (Are you maybe running command.com instead of cmd.exe? That would explain why things that should be there are missing).
I used pyinstaller to create a 64-bit .exe and that resulted in a file of about 6MB. Now, 32-bit executables are smaller, but it might be that the resulting executable is still too big to load.
So I think the Python route may turn out to be more trouble than it is worth.

different result from F5 and F9 when running the os.path.realpath

I feel very confused about os.path.realpath(os.path.dirname(sys.argv[0]))
Here is my confusion:
(1) If I open my script in spyder (the first time) and run the selected lines below (F9):
import os
import sys
dir_path = os.path.realpath(os.path.dirname(sys.argv[0]))
It returns:
dir_path = C:\Python27\lib\site-packages\spyderlib\widgets\externalshell
which is not the result I want.
(2) However, if I run my whole script (F5), I can get what I expect (which is the current directory of my script):
dir_path = C:\Users\abc\Desktop\py
(3) Additionally if I:
Run the whole script,
%reset variables,
Run the same selected lines as before,
I can still get the current directory of my script, as long as I don't exit spyder:
dir_path = C:\Users\abc\Desktop\py
Gould anyone please explain something on this? It will be very appreciated. Thank you a lot!
To get your current complete pathname you can use
os.path.realpath(os.path.curdir)
As for the confusion, print the sys.argv to inspect it. Its content can hold different values, depending on how your script has been called. If I just enter into the python interpreter it holds a list with an empty string, but if I call python myscript.py, it will hold the script name followed by any arguments.

How to make an executable to use in a shell - Python

I have a Python script and I was wondering how I can make it executable; in other words how can I run it by using a shell like bash.
I know the first thing is to stick on the first line #! /usr/bin/env python but then do I need for example the functions to be in a specific order (i.e., the main one at the top or the bottom). What's more do I need to keep the extension .py for my python file (can I just call the function Dosomething?).
To be short, could you provide a simple guide, the important points someone has to take into account to make a Python file executable?
This is how I make an executable script. It doesn't take eggs or anything like that into account. It's just a simple script that I want to be able to execute. I'm assuming you are using linux.
#! /usr/bin/env python
import sys
def main():
#
# Do something ... Whatever processing you need to do, make it happen here.
# Don't shove everything into main, break it up into testable functions!
#
# Whatever this function returns, is what the exit code of the interpreter,
# i.e. your script, will be. Because main is called by sys.exit(), it will
# behave differently depending on what you return.
#
# So, if you return None, 0 is returned. If you return integer, that
# return code is used. Anything else is printed to the console and 1 (error)
# is returned.
#
if an_error_occurred:
return 'I\'m returning a string, it will be printed and 1 returned'
# Otherwise 0, success is returned.
return 0
# This is true if the script is run by the interpreter, not imported by another
# module.
if __name__ == '__main__':
# main should return 0 for success, something else (usually 1) for error.
sys.exit(main())
Now, if you're permissions are set correctly, you can execute this script.
One thing to realize is as your script is processed each line is executed in the interpreter. This is true, regardless of how the processor "gets it". That is importing a script as a module and executing it as a script essentially both work the same, in that they both execute each line of the module.
Once you realize your script is simply executing as it runs, you realize that the order of functions don't matter. A function declaration is a function declaration. It's when you call the function that matters.
So, in general, the layout of your script looks like this
def func1():
pass
def func2():
pass
def main():
return 0
if __name__ == '__main__':
sys.exit(main())
You create the functions you want to use first, then you use them. Hope it helps.
Delete the first space. That is,
#!/usr/bin/env python
Should be the very first line of your file. Then, make sure you make set the permisions for the the file to executable with:
chmod u+x your_script.py
Python scripts execute in sequential order. If you have a file filled with functions, common practice is to have something that looks like this at the very end of your file:
if __name__ == '__main__':
main()
Where main() starts execution of your script. The reason we do it this way, instead of a bare call to main() is that this allows you to make your script act like a module without any modification; if you just had a single line that called main(), your module would would execute the script's main. The if statement just checks if your script is running in the __main__ namespace, i.e., it is running as a script.
The only thing (like you said it) is to include:
#! /bin/env python
on the first line. And is not even mandatory, but recommended.
After that, you can just call it writing:
python [filename].py
in a terminal or in a bash file.
You'll also have to give it execution rights:
chmod u+x yourfile.py
Your code should follow the template
# any functions I want to define, and will be accessible when imported as module
# or run from command line
...
if __name__ == '__main__':
# things I want to do only when I run it from the command line
...
If you want to be able to run it without having to use python fileName.py but rather just ./fileName.py then you will want to make the first line of your file
#!/usr/bin/env python
And make the file executable by the user at least
chmod u+x fileName.py
If you do not add a .py extension to your file then it will still be runnable from the command line ... but not importable by other modules.
Place #!/usr/bin/python in the first line
You can name your python script anything, like: myPythonScript (No, you do not need to keep .py extension)
chmod +x myPythonScript
Run it: ./myPythonScript
Example: myPythonScript
#!/usr/bin/python
print "hello, world!"
You need to add sha bang as you described, e.g.
#!/usr/bin/python
or
#!/usr/bin/env python
as the first line in your file and you need to make it executable by running
chmod +x Dosomething
You do not need to do anything else, in particular file name may be anything including Dosomething. Your PATH probably doesn't include the directory where the file resides, so you should run it like this (assuming your current working directory is where the file is):
./Dosomething

Problem with reading in parameters with special characters in Python

I have a scripts (a.py) reads in 2 parameters like this:-
#!/usr/bin/env python
import sys
username = sys.argv[1]
password = sys.argv[2]
Problem is, when I call the script with some special characters:-
a.py "Lionel" "my*password"
It gives me this error:-
/swdev/tools/python/current/linux64/bin/python: No match.
Any workaround for this?
Updated-
It has been suspected that this might be a shell issue rather than the script issue.
I thought the same too, until i tried it out on a perl script(a.pl), which works perfectly without any issue:-
#!/usr/bin/env perl
$username = $ARGV[1];
$password = $ARGV[2];
print "$username $password\n";
%a.pl "lionel" "asd*123"
==> lionel asd*123
No problem.
So i guess , this looks to me more like a PYTHON issue.
Geezzz ........
The problem is in the commands you're actually using, which are not the same as the commands you've shown us. Evidence: in Perl, the first two command-line arguments are $ARGV[0] and $ARGV[1] (the command name is $0). The Perl script you showed us wouldn't produce the output you showed us.
"No match" is a shell error message.
Copy-and-paste (don't re-type) the exact contents of your Python script, the exact command line you used to invoke it, and the exact output you got.
Some more things to watch out for:
You're invoking the script as a.py, which implies either that you're copying it to some directory in your $PATH, or that . is in your $PATH. If the latter, that's a bad idea; consider what happens if you cd info a directory that contains a (possibly malicious) command called ls. Putting . at the end of your $PATH is safer than putting it at the beginning, but I still recommend leaving it out altogether and using ./command to invoke commands in the current directory. In any case, for purposes of this exercise, please use ./a.py rather than a.py, just so we can be sure you're not picking up another a.py from elsewhere in your $PATH.
This is a long shot, but check whether you have any files in your current directory with a * character in their names. some_command asd*123 (without quotation marks) will fail if there are no matching files, but not if there happens to be a file whose name is literally "asd*123".
Another thing to try: change your Python script as follows:
#!/usr/bin/env python
print "before import sys"
import sys
print "after import sys"
username = sys.argv[1]
password = sys.argv[2]
This will tell you whether the shell is invoking your script at all.
That error comes from your shell, not from Python. Do you have a shopt -s failglob set in your .bashrc or somewhere?
/swdev/tools/python/current/linux64/bin/python: No match.
I think the problem is that the python env is not set:
Does python run at all on your machine ?

In Python, how do I make a temp file that persists until the next run?

I need to create a folder that I use only once, but need to have it exist until the next run. It seems like I should be using the tmp_file module in the standard library, but I'm not sure how to get the behavior that I want.
Currently, I'm doing the following to create the directory:
randName = "temp" + str(random.randint(1000, 9999))
os.makedirs(randName)
And when I want to delete the directory, I just look for a directory with "temp" in it.
This seems like a dirty hack, but I'm not sure of a better way at the moment.
Incidentally, the reason that I need the folder around is that I start a process that uses the folder with the following:
subprocess.Popen([command], shell=True).pid
and then quit my script to let the other process finish the work.
Creating the folder with a 4-digit random number is insecure, and you also need to worry about collisions with other instances of your program.
A much better way is to create the folder using tempfile.mkdtemp, which does exactly what you want (i.e. the folder is not deleted when your script exits). You would then pass the folder name to the second Popen'ed script as an argument, and it would be responsible for deleting it.
What you've suggested is dangerous. You may have race conditions if anyone else is trying to create those directories -- including other instances of your application. Also, deleting anything containing "temp" may result in deleting more than you intended. As others have mentioned, tempfile.mkdtemp is probably the safest way to go. Here is an example of what you've described, including launching a subprocess to use the new directory.
import tempfile
import shutil
import subprocess
d = tempfile.mkdtemp(prefix='tmp')
try:
subprocess.check_call(['/bin/echo', 'Directory:', d])
finally:
shutil.rmtree(d)
"I need to create a folder that I use only once, but need to have it exist until the next run."
"Incidentally, the reason that I need the folder around is that I start a process ..."
Not incidental, at all. Crucial.
It appears you have the following design pattern.
mkdir someDirectory
proc1 -o someDirectory # Write to the directory
proc2 -i someDirectory # Read from the directory
if [ %? == 0 ]
then
rm someDirectory
fi
Is that the kind of thing you'd write at the shell level?
If so, consider breaking your Python application into to several parts.
The parts that do the real work ("proc1" and "proc2")
A Shell which manages the resources and processes; essentially a Python replacement for a bash script.
A temporary file is something that lasts for a single program run.
What you need is not, therefore, a temporary file.
Also, beware of multiple users on a single machine - just deleting anything with the 'temp' pattern could be anti-social, doubly so if the directory is not located securely out of the way.
Also, remember that on some machines, the /tmp file system is rebuilt when the machine reboots.
You can also automatically register an function to completely remove the temporary directory on any exit (with or without error) by doing :
import atexit
import shutil
import tempfile
# create your temporary directory
d = tempfile.mkdtemp()
# suppress it when python will be closed
atexit.register(lambda: shutil.rmtree(d))
# do your stuff...
subprocess.Popen([command], shell=True).pid
tempfile is just fine, but to be on a safe side you'd need to safe a directory name somewhere until the next run, for example pickle it. then read it in the next run and delete directory. and you are not required to have /tmp for the root, tempfile.mkdtemp has an optional dir parameter for that. by and large, though, it won't be different from what you're doing at the moment.
The best way of creating the temporary file name is either using tempName.TemporaryFile(mode='w+b', suffix='.tmp', prifix='someRandomNumber' dir=None)
or u can use mktemp() function.
The mktemp() function will not actually create any file, but will provide a unique filename (actually does not contain PID).

Categories