Is there a way to import a module through the command line, that gets exposed to the running script (or ideally through all execution)?
My use case is to debug: I have a set of debugging utils (to display images, histograms...) that I only want to import while debugging (and that other people on my team do not need to care about, the debugging code doesn't get pushed to the main repo, it doesn't get imported at execution time...).
For example, when debugging on my end, I would like to do: python --option "import debug_utils.py" main.py, and my personal debugging functionalities would be visible to the running scripts.
At execution time others and me would simply execute, without package/import conflicts:
python main.py, and the debugging utils are not imported.
Thanks!
There is no such option to Python, but you could gate the import of your debugging utilities using an environment variable, like this:
import os
if 'MY_APP_DEBUG' in os.environ:
import debug_utils
else:
debug_utils = None
And then in your code, you can write:
if debug_utils is not None:
debug_utils.show_debugging_data()
Related
ran into a weird problem where there is a shared-object import error only when I run the script from command line. It succeed if i run the script in the python console using exec(...)
I have a class that needs a shared object: foo.py:
import os
cur_dir = os.curdir()
os.chdir('/tmp/dir_with_shared_object/')
import shared_object_class
os.chdir(cur_dir)
class Useful:
... # use the shared object import
Then there is a script like this:
from foo import Useful
If I enter python console and run:
exec(open('script.py').read())
Everything works fine.
If I run this on command line:
python script.py
I will get
ModuleNotFoundError: No module named 'shared_object_class'
The python is the same. It is 3.7.3, GCC 7.3.0 Anaconda. Anyone knows what is causing this discrepancy in behavior for shared object import?
A standard way of importing from a custom directory would be to include it in the PYTHONPATH environmental variable, with export PYTHONPATH=/tmp/dir_with_shared_object/.
update
It could also be done dynamically with
import sys
p = '/tmp/dir_with_shared_object/'
sys.path.insert(0, p)
PS
I think I have an explanation for why OP's original code didn't work. According to this python reference page, the import system searches, inter alia, in "[t]he directory containing the input script (or the current directory when no file is specified)." So the behavior in the REPL loop is different from how it is when running a script. Apparently the current directory is evaluated each time an import statement is encountered, while the directory containing the input script doesn't change.
In Python interpreter (Python 3.9.2, Win 10) I've already run
import numpy as np
In a plain text file "myscript.py" in the same (current working) directory is the single line
A = np.array((1,2,3,4))
Running at the interpreter
import myscript
gives the error message (in part)
NameError: name 'np' is not defined
I'm sure it's a namespace-ish thing; I'm a long-time R user just starting to explore Python. Just puzzled why np isn't defined, despite having imported that alias at the interpreter previously.
I'm looking for something equivalent to R's source() function whereby I can have the Python interpreter and the source code text file both open, make changes to the source code file, and rerun it in the interpreter with each such change. (I'm aware of using importlib.reload() for the subsequent re-runs, but the "name undefined" issue keeps me from getting out of the starting gate.)
If you import myscript in the interpreter, Python just executes that code (similar to using python3 myscript). If you haven't put import numpy as np at the top of your script, then np is not defined. Because it is not defined.
So to start off, you need to add that line to myscript.py:
import numpy as np
A = np.array((1,2,3,4))
Then you can run your script interactively in the interpreter with
>>> exec(open('myscript.py').read())
or, as you suggested, with
>>> import importlib
>>> import myscript
>>> importlib.reload(myscript)
The first two lines must be at the start of your interpreter session, and everytime you edit myscript.py you run importlib.reload(myscript).
This is my code. I'm pretty new to this.
from subprocess import call
call(["cd", "/etc/apache2/"])
However, when this function is run, I get
Errno 2: No such file or directory
I am running Django within Apache*. This is my views.py file. Ask for additional code, and you shall receive.
edit - It should be noted that /etc/apache2/ does indeed exist.
If you want to change the working directory of the Python process you can use chdir from the os module:
import os
os.chdir('/etc/apache2')
First of all, you will not get what you expect if you run this. Try
import os
os.chdir('/etc/apache2')
Second, try /path/to/cd as process may not know cd alias.
I have a software that has python 2.5.5. I want to send a command that would start a script in python 2.7.5 and then proceed with the script.
I tried using
#!python2.7.5
and http://redsymbol.net/articles/env-and-python-scripts-version/
But I cant get it to work...
In my python 2.5.5 I can execute script as
execfile("c:/script/test.py")
The problem is that the 2.7.5 has a module comtypes + few other. I dont know how to install it for my 2.5.5 so I'm trying to start a separate script and run it under python27. Now another reason why I want it its because I want to take the load off program. I have 2 heavy tasks to perform. The second task is the one that need comptypes so sending it to external shell/app would do perfect trick. Is there a way to do it ?
I wish I could just type run("C:/Python27/python.exe % C:/script/test,py")
Thanks, bye.
Little update. I try to run
import os
os.system("\"C:\Python27\python.exe\" D:\test\runTest.py")
But I'm getting a quick pop up and close window saying that
Import Error : no module named site...
This works if I run from external shell but not from here :(
So I've tried another approach this time to add modules to python... in any case I run this :
import os
import sys
sys.path.append("C:/python27")
sys.path.append("C:/Python27/libs")
sys.path.append("C:/Python27/Lib")
sys.path.append("C:/Python27/Lib/logging")
sys.path.append("C:/Python27/Lib/site-packages")
sys.path.append("C:/Python27/Lib/ctypes")
sys.path.append("C:/Python27/DLLs")
import PyQt4
print PyQt4
import comtypes
import logging
but it crashes with C error...
Runtime Error :
Program: c:\Pr...
R6034
An application has made attempt to load the C runtime library incorectly.
blablabla....
How can I import it ? Maybe if I can import it I can run it directly from my app rather than starting separate python...
Traceback (most recent call last):
File "<string>", line 18, in <module>
File "C:\Python27\Lib\site-packages\comtypes\__init__.py", line 22, in <module>
from ctypes import *
File "C:\Python27\Lib\ctypes\__init__.py", line 10, in <module>
from _ctypes import Union, Structure, Array
ImportError: DLL load failed: A dynamic link library (DLL) initialization routine failed.
Another update to isseu
so I run now
import os
os.system("start cmd {D:\test\runTest.py}")
now this works and he open CMD with c:\Python27 as directory but he dont run the file... any hitns how to fix it?
Use "raw" strings so that you don't need to escape as much; I think the backslashes are what was breaking your code since backslash is considered an escape character except in raw strings.
Also, use the subprocess module. It makes it easy to avoid manually making a safe command string (the module takes care of that for you). All you need to do is pass it a list of arguments.
Your code would then look something like this:
import subprocess
proc = subprocess.Popen([r"C:\Python27\python.exe", r"D:\test\runTest.py"])
# then either do this
proc.wait() # wait until the process finishes
# or this
while True:
# NOTE: do something else here
# poll the process until it is done
if proc.poll() is not None:
break # break out of loop
See subprocess docs for Python 2 here. Be sure to check if a feature was added after Python 2.5 (the 2.5 docs aren't available online anymore AFAIK).
UPDATE:
I just noticed that you tried to use the Python 2.7 libraries and modules in your 2.5 code. This probably won't work due to new features added after 2.5. But it got me thinking how you might be able to make 2.7 work.
It may be that your Python2.7 install can't find its libraries; this is probably why you get the error Import Error : no module named site. You can do something like the above and modify the PYTHONPATH environment variable before starting the subprocess, like this:
import os
import subprocess
paths = [r"C:\python27", r"C:\python27\libs", r"C:\python27\Lib\site-packages", r"C:\python27\DLLs"]
paths += os.environ.get('PYTHONPATH', '').split(os.pathsep)
env27 = dict(os.environ)
env27['PYTHONPATH'] = os.pathsep.join(paths)
proc = subprocess.Popen([r"C:\Python27\python.exe", r"D:\test\runTest.py"], env=env27)
I want to use an embedded IPython shell with a user_ns dictionary and a my profile configuration (ipython_config.py and the startup files). The purpose is to run a Django shell with models imported on startup. django-extensions implements a command called shell_plus that does this:
https://github.com/django-extensions/django-extensions/blob/master/django_extensions/management/commands/shell_plus.py
from IPython import embed
embed(user_ns=imported_objects)
The problem is that this does not load my startup files. embed() calls load_default_config() which I figure loads ipython_config.py.
How do I make the embedded IPython instance run my profile startup files?
I used the following workaround to run my own IPython startup script but still take advantage of shell_plus:
Create a file called shell_plus_startup.py in the same directory as manage.py. For example:
# File: shell_plus_startup.py
# Extra python code to run after shell_plus starts an embedded IPython shell.
# Run this file from IPython using '%run shell_plus_startup.py'
# Common imports
from datetime import date
# Common variables
tod = date.today()
Launch shell plus (which launches an embedded IPython shell).
python manage.py shell_plus
Manually run the startup script.
In [1]: %run shell_plus_startup.py
Then you can use variables you've defined, modules you've imported, etc.
In [2]: tod
Out[2]: datetime.date(2012, 7, 14)
Also see this answer: scripting ipython through django's shell_plus
I found a way that works if you are using django-extensions-shell_plus. It is a bit hacky, but with this way your startup file is loaded fully automatically and you don't have to type any run-command at the beginning of your ipython-session.
Therefore I edited the file shells.py from the django_extensions dir, which is in my case located in /usr/local/lib/python2.7/dist-packages/django_extensions/management/shells.py. I added these lines inside the function import_objects(options, style):, so it imports the content of the file startup.py defined by the environment param PYTHONSTARTUP.
def import_objects(options, style):
# (...)
import os, sys, pkgutil
if 'PYTHONSTARTUP' in os.environ:
try:
sys.path.append(os.environ['PYTHONSTARTUP'])
import startup
content = [element for element in dir(startup) if not element.startswith('__')]
for element in content:
imported_objects[element] = getattr(startup, element)
except Exception, ex:
sys.exit("Could not import startup module content, Error:\n%s" % ex)
Now when I launch the shell_plus-shell, I give it the environment variable to my startup python script. My bash script to launch the shell with everything in place looks like this:
#!/bin/bash
export PYTHONSTARTUP=/home/ifischer/src/myproject/startup.py # tells shell_plus to load this file
python /home/ifischer/src/myproject/manage.py shell_plus --ipython
Now I have access to all methods and variables defined in startup.py from the beginning of the ipython session.
So you can reuse that and have custom startup files for every project, pre-loading different aspects.
Maybe there is a cleaner way where to include the lines I added to the shells.py? But this approach works fine for me for now.
It does automatically load your ipython configuration starting from django-extensions==1.5.6. You can also pass additional arguments to ipython via IPYTHON_ARGUMENTS. Docs:
http://django-extensions.readthedocs.org/en/latest/shell_plus.html#configuration
Another way is using a class that derives from InteractiveShellEmbed and InteractiveShellApp. Sample, incomplete code:
from IPython.terminal.embed import InteractiveShellEmbed
from IPython.terminal.ipapp import InteractiveShellApp
class ISE(InteractiveShellEmbed, InteractiveShellApp):
def init_shell(self):
self.shell = self.instance()
self.extra_args = []
ise = ISE()
ise.init_path()
ise.init_shell()
ise.init_code()
ise.shell()