Heyo,
Just started writing an assembler for the imaginary computer my class is creating wire-by-wire since the one the TA's provided sucks hard. I chose python even though I've never really used it that much (but know the basic syntax) and am loving it.
My favorite ability is how I can take a method I just wrote, paste it into the shell and then unit test it by hand (I'm using IDLE).
I'm just wondering if there is a way to expose all the symbols in my python code to the shell automatically, so I can debug without copying and pasting my code into the shell every time (especially when I make a modification in the code).
Cheers
you can import the module that your code is in. This will expose all of the symbols prefixed with the module name.
The details for the easiest way to do it depend on your operating system but you can always do:
>>> sys.path.append('/path/to/directory/that/my/module/is/in/')
>>> import mymod #.py
later after you make a change, you can just do
>>>> reload(mymod)
and the symbols will now reference the new values. Note that from mymod import foo will break reload in the sense that foo will not be updated after a call to reload. So just use mymod.foo.
Essentially the trick is to get the directory containing the file on your PYTHONPATH environment variable. You can do this from .bashrc on linux for example. I don't know how to go about doing it on another operating system. I use virualenv with has a nice wrapper and workon command so I just have to type workon foo and it runs shell scripts (that I had to write) that add the necessary directories to my python path.
When I was just starting off though, I made one permanent addition to my PYTHONPATH env variable and kept module I wrote in there.
Another alternative is to execute your module with the -i option.
$ python -i mymod.py
This will execute the module through to completion and then leave you at the interpreter. this isn't IDLE though, it's a little rougher but you are now in your module's namespace (or rather the module's namespace is the global namespace)
Check IPython. It's enhanced interactive Python shell. You can %run your script and it will automatically expose all your global objects to the shell. It's very easy to use and powerful. You can even debug your code using it.
For example, if your script is:
import numpy as np
def f(x):
return x + 1
You can do the following:
%run yourScript.py
x = np.eye(4)
y = f(x)
Related
I have a python program that I usually run as a part of a package:
python -m mymod.client
in order to deal with relative imports inside "mymod/client.py." How do I run this with pdb - the python debugger. The following does not work:
python -m pdb mymod.client
It yields the error:
Error: mymod.client does not exist
EDIT #1 (to address possible duplicity of question)
My question isn't really about running two modules simultaneously python, rather it is about how to use pdb on a python script that has relative imports inside it and which one usually deals with by running the script with "python -m."
Restated, my question could then be, how do I use pdb on such a script while not having to change the script itself just to have it run with pdb (ie: preserving the relative imports inside the script as much as possible). Shouldn't this be possible, or am I forced to refactor in some way if I want to use pdb? If so what would be the minimal changes to the structure of the script that I'd have to introduce to allow me to leverage pdb.
In summary, I don't care how I run the script, just so long as I can get it working with pdb without changing it's internal structure (relative imports, etc) too much.
I think I have a solution.
Run it like this:
python -m pdb path/mymod/client.py arg1 arg2
that will run it as a script, but will not treat it as a package.
At the top of client.py, the first line should be:
import mymod
That will get the package itself loaded.
I am still playing with this, but it seems to work so far.
This is not possible. Though unstated in documentation, Python will not parse two modules via the -m command line option.
I think I already answered my own question in my mind but in case there are other reasons I don't see, I am wondering what's the benefit/difference between running a python script with
python script.py
and using the shebang
#!/usr/local/bin/python
I usually run my scripts as self executing scripts so I use the shebang. I just came across a team member who runs his programs using
python script.py
My question is which way is better or is this a personal preference thing?
If I run with shebang, I can specify which version I want/need to use. The only problem is I need to make sure that version is installed at that location.
If I run it the other way (not sure what to call it, non-shebang way?), I need to make sure the version I want to use is in my path or defined correctly in my path. I could also type in the full path of python but that can get rather tiring if the path is very long.
Are there other benefits/drawbacks that I don't see? Thanks.
If you want to run your script with shebang, you'll have to make sure that the user running the script has execution rights on the script in question (chmod u+x script.py). That's not the case if you call python with the script as an argument.
Another issue with the shebang solution is that you're forcing the location of the python executable. If I try to run your script and my version of python is in /usr/bin/python instead of /usr/local/bin/python, I'll have to edit your script. On other platforms, such as Windows, I'll have to edit it too.
With the shebang the script must be located on a volume that permits execution, must be allowed to execute by any security modules, and the interpreter specified in the shebang must be present.
Invoking the interpreter directly only requires the script to be readable.
In general environment they are the same.
However, shebang gives you an extra benefit that you can replace your executable very easily without changing any other files just by substituting the file. The second(better) implementation can potentially be anything. For a script based environment, I always use shebang, with a fixture of languages.
for the shebang line, the argument that you need to be concerned about the location of python binary is not valid. you can always write:
#!/usr/bin/env python
to have local environment to decide for you.
If you are treating python as scripting language that one might execute from a POSIX compliant shell then it really doesn't matter.
That said, given that Python is more or less becoming standardized with its package management and installation, I would argue none of the above. What should be done is that all scripts be turned into callables (either functions or callable class instances) that live within modules (preferably defined a namespace package). That callable is the entry point to your program/script. Then a setuptools compliant setup.py should be placed in the root directory of the project for this module and define that as the entry point to your program.
An example:
In example.package/example/package/script.py:
def main():
print('hello world')
The setup.py:
from setuptools import setup, find_packages
setup(
name='example.package',
description='An example package with an entry point',
...
entry_points="""
# -*- Entry points: -*-
[console_scripts]
script = example.package.script:main
""",
)
Note that the script is defined to be example.package.script:main, where the element before the : corresponds to the package and the second element is the callable. In this case, it corresponds to the main function in example/package/script.py (standard python module layout).
Installing the package (preferably in a virtualenv) will generate the correct "executable" that will be immediately accessible from the shell, across all operating systems (i.e. on Windows, a script.exe will be created that basically will start Python with the script).
Is there a command I can use in a Python file to clear all variables?
I've been searching for this for a while but all I could find was to use %reset. However, this only seems to work in IPython, but not when I try to run a python file.
FYI, I am working with the free version of enthought canopy.
[ADDED from comments:] I have several python files I run, which might have shared variables. I would like to be able to clear all variables before running any of the files to guarantee that I have defined variables correctly and that they are taking the correct values.
tl;dr -- what you describe is not an issue.
But it is worth understanding, to avoid other points of confusion:
1) When you are running IPython (including Canopy's Python pane, which is a standard IPython QtConsole), IPython has its own global namespace (list of variables and modules and functions etc) which is distinct from the namespace of the scripts which run within it. This can be confusing, but it is actually a feature.
2) When you run each script normally, it starts with an empty namespace, just as if it were running in plain Python. That's why your concern is a non-issue. But this can also confuse beginners, because your script also doesn't know about the modules that have already been imported in IPython. For more on this, see this article.
3) When the script completes, its global namespace is copied into the IPython global namespace (overwriting any same-named variables that were already there).
4) Thus normal visibility is one-way -- IPython sees the results of the scripts that you ran, so you can work with them more at the prompt, but your scripts don't see the results of previous scripts that you ran (not even the same script), or of anything you do at the prompt.
5) There is one huge difference, though, from when you run your script in plain Python. Namely, IPython itself is not re-initialized between runs (unless you reset the kernel), and in particular, any modules that have been imported are still initialized and won't be re-loaded or re-initialized when you import them in subsequent scripts. For more info, see this article.
6) A side note: The %run command can be given a -i option to make namespace visibility 2-way, so your scripts will start in the IPython namespace (as I think you were expecting), but this is unusual and not the default, since usually one wants to ensure (as you apparently do) that the script is running "clean". The main reason to use this option would be to have your scripts build on each other, but there are more robust, portable ways to achieve this (namely passing variable names from the IPython namespace as parameters to the functions that you define in your script).
I am just getting started with Python.
How do I call a test script from C:\X\Y\Z directory when in Python interpreter command line in interactive mode? How do I specify the full path for the file when it is not in the current working directory?
I can call a test script when using the windows run command with "python -i c:\X\Y\Z\filename.py" and it runs fine. But I want to be able to call it form the Python terminal with the ">>>" prompt.
(I searched and searched for two hours and could not find an answer to this, although it seems like it should be a common question for a beginner and an easy thing to do.)
Thanks
Since you are using backslashes for the file path, python interprets those as "escape characters." When writing the file path in Python, make sure to use forward slashes.
with open("C:/X/Y/Z/filename.py", "r") as file:
exec(file.read())
Double backslashes also work, but I prefer the cleaner look of forward slashes.
If you want to import it into the REPL:
import sys
sys.path.append('c:\X\Y\Z')
import filename
If you want to execute code from a file within the interpreter, you can use execfile
execfile('C:/X/Y/Z/filename.py')
(/ works as path separator in all operating systems, if you use \, you need to escape them ('C:\\X\\Y\\Z\\filename.py')or use raw string literal (r'C:\X\Y\Z\filename.py'))
If you are using IPython (and you should use, it's much more useful than vanilla interactive Python), you can use magic function run (or with % prefix: %run):
run C:\\X\\Y\\Z\\filename.py
%run C:\\X\\Y\\Z\\filename.py
See this link for more information about magic functions.
And by the way, it has even auto completion of filenames.
Exec the heck out of it
Python 2.x:
execfile("C:\\X\Y\\Z")
Python 3+:
with open("C:\\X\Y\\Z", "r") as f:
exec(f.read())
Still, that is very bad practice - it executes code from a string (at some point), instead of using preferred and safer way of importing modules. Still, when you import module and have some of its code after "-f __name__ == '__main__':", that parts won't work (because __name__ in imported module won't be __main__, and it would be, if you ran it as single script).
It is bad for many reasons, in some sense strongly connected to Zen of Python, but if you're beginner, this should speak to you:
When you do anything in interactive mode, you work on some namespace (this term is very important for understanding python, if you don't know it, check it in python language reference). When you exec()/execfile() something without providing globals()/locals(), you may end up with modified namespace.
Modified namespace?
What does it mean? Lets have a script like that:
radius = 3
def field_of_circle(r):
return r*r*3.14
print(field_of_circle(radius))
Now, you have following session:
>>>radius = 5
>>>execfile("script_above.py")
28.26
>>>print(radius)
3
You see what happens? Variables defined by you in interactive session will get overwritten by values from end of script. The same goes for modifying already imported external modules. Lets have a very simple module:
x = 1
and executed script:
import very_simple_module
very_simple_module.x = 3
Now, here's a interpreter interactive session:
>>>import very_simple_module
>>>print(very_simple_module.x)
1
>>>execfile("executed_script.py")
>>>print(very_simple_module.x)
3
Run another interpreter
Interactive sessions are very useful for many things, but not for many things, but running python scripts is not one of them.
Unless... you wanna play tough and use python shell as system shell. Then, you can use subprocess (in standard library) or sh (which can be found on PyPI):
>>>import subprocess
>>>subprocess.call(["python", "C:\\X\Y\\Z"], shell=True)
>>>from sh import python
>>>python("C:\\X\Y\\Z")
Those won't have this problem with modifying interactive interpreters namespace
See script as module
Also, there is one more option: in interactive session add directory with script to pythonpath, and import module named as script:
>>>import sys
>>>if "C:\\X\\Y" not in sys.path:
sys.path.append("C:\\X\\Y")
>>>import Z
Remember that directory in which interpreter was started is automatically on pythonpath, so if you've ran python in the same directory as your script, you just have to use 3rd of lines above.
Interpreters namespace won't change, but code after "-f __name__ == '__main__':" won't be executed. Still you can access scripts variables:
>>>radius = 5
>>>import first_example_script
>>>print(radius)
5
>>>print(first_example_script.radius)
3
Also, you can have module name conflict. For example, if your script was sys.py, then this solution will work, because python will import builtin sys module before yours.
Is it possible to run a python script (not module) from inside ipython without indicating its path? I tried to set PYTHONPATH but it seems to work only for modules.
I would like to execute
%run my_script.py
without being in the directory containing the file.
from within the directory of "my_script.py" you can simply do:
%run ./my_script.py
How to run a script in Ipython
import os
filepath='C:\\Users\\User\\FolderWithPythonScript'
os.chdir(filepath)
%run pyFileInThatFilePath.py
That should do it
The %run magic has a parameter file_finder that it uses to get the full path to the file to execute (see here); as you note, it just looks in the current directory, appending ".py" if necessary.
There doesn't seem to be a way to specify which file finder to use from the %run magic, but there's nothing to stop you from defining your own magic command that calls into %run with an appropriate file finder.
As a very nasty hack, you could override the default file_finder with your own:
IPython.core.magics.execution.ExecutionMagics.run.im_func.func_defaults[2] = my_file_finder
To be honest, at the rate the IPython API is changing that's as likely to continue to work as defining your own magic is.
In python there is no difference between modules and scripts; You can execute both scripts and modules. The file must be on the pythonpath AFAIK because python must be able to find the file in question. If python is executed from a directory, then the directory is automatically added to the pythonpath.
Refer to What is the best way to call a Python script from another Python script? for more information about modules vs scripts
There is also a builtin function execfile(filename) that will do what you want
Not exactly the answer to your question, but you can drop into ipython at the end of a script's execution by using the -i parameter to ipython:
ipython -i my_script.py
At the end of the script you're dropped into the ipython prompt with the script's variables available to you, just like python -i.
for Python 3.6.5
import os
os.getcwd()
runfile('testing.py')