How to debug into subprocess.call() in Python? - python

I am trying to understand a project's source code these days. I run the project, line by line, everything works fine until this line:
res = subprocess.call(command, env=os.environ)
I checked the variable "command" and realized that this function just throw a command to another python script and try to execute it in a subprocess. So I jumped out of Eclipse, and tried to execute the command through Terminal while under the same directory.
Now this is what I got:
Traceback (most recent call last):
File "/home/elderry/Projects/git/tahoe-lafs/support/bin/tahoe", line 6, in <module>
from pkg_resources import load_entry_point
File "/usr/lib/python2.7/site-packages/pkg_resources.py", line 2850, in <module>
working_set.require(__requires__)
File "/usr/lib/python2.7/site-packages/pkg_resources.py", line 696, in require
needed = self.resolve(parse_requirements(requirements))
File "/usr/lib/python2.7/site-packages/pkg_resources.py", line 594, in resolve
raise DistributionNotFound(req)
pkg_resources.DistributionNotFound: allmydata-tahoe==1.10.0.post27
Then I completely lost my direction, where did the subprocess continue to run? Why did the script work well in program but not in Terminal? Since that script is also included in the project, with some hope I set some break points in it in Eclipse, which didn't catch anything. Is there any way to debug into the subprocess, not dive into the code of subprocess module's code itself?

I guess your main project alters PYTHONPAH (sys.path). Look in os.environ of your project and try to run second script with this environment.

Related

Packaging/Deploying Maven from Python

Big picture is I'm trying to automate my deployment process of building with maven and deploying to a web logic server. Little picture is I'm using subprocess to see if I can call maven from within python. When I attempt this subprocess mistakes mvn for a file.
Here is my code so far:
import subprocess
def main():
print(subprocess.check_output(["mvn", "-v"]))
if __name__ == '__main__':
main()
And here's my error:
C:\pythondev\python.exe "C:/pythondev/development/deployment scripts/redploy-to-localhost.py"
Traceback (most recent call last):
File "C:/pythondev/development/deployment scripts/redploy-to-localhost.py", line 9, in <module>
main()
File "C:/pythondev/development/deployment scripts/redploy-to-localhost.py", line 5, in main
subprocess.check_output(["a"])
File "C:\pythondev\lib\subprocess.py", line 376, in check_output
**kwargs).stdout
File "C:\pythondev\lib\subprocess.py", line 453, in run
with Popen(*popenargs, **kwargs) as process:
File "C:\pythondev\lib\subprocess.py", line 756, in __init__
restore_signals, start_new_session)
File "C:\pythondev\lib\subprocess.py", line 1155, in _execute_child
startupinfo)
FileNotFoundError: [WinError 2] The system cannot find the file specified
Process finished with exit code 1
Although my issue is with subprocess I'm open to answers that suggest a better alternative.
I ran into the same issue and was hesistant to use shell=True, because the internet tells me this is evil.
When I run where mvn in my cmd.exe, I can see that there are two matches:
mvn, which is a Unix shell-script (it starts with #!/bin/sh),
mvn.cmd, which is a Windows batch file.
I think what happens when you execute mvn something -something in cmd.exe is the following: Windows tries finding an executable called mvn. It finds the mvn file, but realizes that this file is not executable. It then tries finding files like mvn.com, mvn.exe, ... (see the %PATHEXT% system variable). When it finds mvn.cmd, it executes that and everyone is happy.
As far as I understand it, the problem with subprocess.check_output (and subprocess.run, and so on) is that the path-"expansion" via %PATHEXT% is not being performed. So the solution is that you have to give the extension manually and run your command as
print(subprocess.check_output(["mvn.cmd", "-v"]))
Try this it worked for me.
print(subprocess.check_output(["mvn", "-v"], shell=True))

How to get into the python shell within Odoo 8 environment?

I would like to use the Odoo Framework from the shell.
I installed the module "Shell command backport" (technical name: shell), but I couldn't make it work.
$ ./odoo.py shell --addons-path=/opt/odoo_8/src/linked-addons -d database_name
Traceback (most recent call last):
File "./odoo.py", line 160, in <module>
main()
File "./odoo.py", line 157, in main
openerp.cli.main()
File "/opt/odoo_8/src/OCA/OCB/openerp/cli/__init__.py", line 58, in main
for m in module.get_modules():
File "/opt/odoo_8/src/OCA/OCB/openerp/modules/module.py", line 351, in get_modules
plist.extend(listdir(ad))
File "/opt/odoo_8/src/OCA/OCB/openerp/modules/module.py", line 346, in listdir
return map(clean, filter(is_really_module, os.listdir(dir)))
OSError: [Errno 2] No such file or directory: '/opt/odoo8/openerp/addons'
Where is defined the path /opt/odoo8/openerp/addons? I checked this similar question as well.
If I don't write the addons path in the command the error appears again.
I read the answer of this other question, I tried the module and the script option but they didn't work. What should I do to make it work? Any hint would help.
Check your .opererp_serverrc for the user you are executing the command as. In the users home directory you will find this file. There may be reference to the addons_path. The path it appears to be looking for /opt/odoo8/openerp/addons differs from what you have specified in your command. I would check your config files.

Trying to make PLY work for the first time

I'm new to Python and I'm having some problems trying to make PLY works. For now, all I want is to successfully run the example from the PLY homepage.
At first I tried to just download PLY-3.8, put the ply folder in the same directory I saved the example (calc.py) and ran it. The calc.py file is at the C:\Users\...\Python directory and the ply folder is the C:\Users\...\Python\ply, just to make it clearer. But I got an ImportError: No module named 'ply'.
Then I searched for a while, tried to update something called distutils and install the modules through the Windows PowerShell and so on and so forth, but none of that worked and I just reset the whole thing (reinstalling Python and all of that). But then I finally got it to work by simply inserting into the sys.path the directory path where the script I was running (edit: in interactive mode) was, by doing this:
import sys
sys.path.insert(0,'C:\\Users\\ ... \\Python')
This fixed the ImportError but, and this is where I am now, there are a bunch of other errors:
Traceback (most recent call last):
File "C:\Users\...\Python\calc.py", line 48, in <module>
lexer = lex.lex()
File "C:\Users\...\Python\ply\lex.py", line 906, in lex
if linfo.validate_all():
File "C:\Users\...\Python\ply\lex.py", line 580, in validate_all
self.validate_rules()
File "C:\Users\...\Python\ply\lex.py", line 822, in validate_rules
self.validate_module(module)
File "C:\Users\...\Python\ply\lex.py", line 833, in validate_module
lines, linen = inspect.getsourcelines(module)
File "c:\users\...\python\python35\lib\inspect.py", line 930, in getsourcelines
lines, lnum = findsource(object)
File "c:\users\...\python\python35\lib\inspect.py", line 743, in findsource
file = getsourcefile(object)
File "c:\users\...\python\python35\lib\inspect.py", line 659, in getsourcefile
filename = getfile(object)
File "c:\users\...\python\python35\lib\inspect.py", line 606, in getfile
raise TypeError('{!r} is a built-in module'.format(object))
TypeError: <module '__main__'> is a built-in module
Now I have absolutely no idea what to do. I tried to search for a solution but had no luck. I appreciate if anyone can help me out.
I'm on Windows 10, using Python 3.5.0 and iep as my IDE (www.iep-project.org) if these informations are of any importance.
In short: I just want to successfully run the example from the PLY homepage and then I think I can figure out the rest.
EDIT: I found out that if I do:
import inspect
inspect.getfile(__main__)
I get the exact same (last) error from before:
Traceback (most recent call last):
File "<console>", line 1, in <module>
File "c:\users\...\python\python35\lib\inspect.py", line 606, in getfile
raise TypeError('{!r} is a built-in module'.format(object))
TypeError: <module '__main__'> is a built-in module
I think this is the culprit, but I still don't know how to fix it.
EDIT 2: I got it to work and answered the question explaining how, but if someone have a more complete answer, I would love to hear it.
To anyone having this problem, I found what was the issue. I still don't know why exactly is like that, so if anyone have a more complete answer to provide I would appreciate (I'm still a newbie at Python).
Anyway, it seems this code can't be executed in Interactive mode, it needs to be executed as a script. To do that on IEP it's Run > Run file as script or Ctrl+Shift+E. On IDLE you need to Open... the file (Ctrl+O) and then Run Module (F5).
As to why it can't be executed in Interactive mode, here's a little bit about the difference between interactive mode and running as script from the IEP wizard:
Interactive mode vs running as script
You can run the current file or the main file normally, or as a script. When run as script, the shell is restared (sic) to provide a clean environment. The shell is also initialized differently so that it closely resembles a normal script execution.
In interactive mode, sys.path[0] is an empty string (i.e. the current dir), and sys.argv is set to [''].
In script mode, __file__ and sys.argv[0] are set to the scripts filename, sys.path[0] and the working dir are set to the directory containing the script.
That explains a bit about why the inspect.getfile(__main__) was throwing an error: the __main__ had no attribute __file__. And also why I had to insert the current directory into sys.path: sys.path didn't had the current directory in interactive mode.
I hope this helps someone.

calling python script from another script

i'm trying to get simple python script to call another script, just in order to understand better how it's working. The 'main' code goes like this:
#!/usr/bin/python
import subprocess
subprocess.call('kvadrat.py')
and the script it calls - kvadrat.py:
#!/usr/bin/python
def kvadriranje(x):
kvadrat = x * x
return kvadrat
print kvadriranje(5)
Called script works on its own, but when called through 'main' script error occurs:
Traceback (most recent call last):
File "/Users/user/Desktop/Python/General Test.py", line 5, in <module>
subprocess.call('kvadrat.py')
File "/System/Library/Frameworks/Python.framework/Versions/2.6/lib/python2.6/subprocess.py", line 444, in call
File "/System/Library/Frameworks/Python.framework/Versions/2.6/lib/python2.6/subprocess.py", line 595, in __init__
File "/System/Library/Frameworks/Python.framework/Versions/2.6/lib/python2.6/subprocess.py", line 1106, in _execute_child
OSError: [Errno 2] No such file or directory
Obviously something's wrong, but being a beginner don't see what.
you need to give it the full path to the script that you are trying to call, if you want to do this dynamically (and you're in the same directory), you can do:
import os
full_path = os.path.abspath('kvadrat.py')
Have you tried:
from subprocess import call
call(["python","kvadrat.py"]) #if in same directory, else get abs path
You should also check if your file is there:
import os
print os.path.exists('kvadrat.py')
Subprocess.call requires that the file is executable and found in path. In unix systems, you can try to use subprocess.call(['./kvadrat.py']) to execute a kvadrat.py file in the current working directory and make sure the kvadrat.py has executable permissions on; or you can copy it to a directory in your PATH, such as /usr/local/bin - then it is executable from anywhere as you wanted.
Most of the time you do not want to run other python applications using subprocess but instead just importing them as modules, however...

Running python scripts with subprocess in windows. Python code checker wrappers from the emacswiki yield the same error

So i'm trying to setup the python code checkers suggested in the emacs wiki. However, I'm unable to run those scripts in my command shell let alone emacs.
The section is found here:
http://www.emacswiki.org/emacs/PythonProgrammingInEmacs#toc7
And I tried the script located here and here
In both cases I changed the first line from #!usr/bin python with the full path of my python executable and when I run the scripts via
python pylint_etc_wrappers.py someModule.py
or
python pycheckers.py soemModule.py
both boil down to the same error, most likely because they try to open a subprocess. Here's the trace:
Traceback (most recent call last):
File "pycheckers.py", line 254, in <module>
runner.run(source_file)
File "pycheckers.py", line 91, in run
process = Popen(args, stdout=PIPE, stderr=PIPE)
File "C:\devel\Python\Python-2.7\Lib\subprocess.py", line 672, in __init__
errread, errwrite)
File "C:\devel\Python\Python-2.7\Lib\subprocess.py", line 882, in _execute_child
startupinfo)
WindowsError: [Error 2] The system cannot find the file specified
The second script suggests to change the first line to the path of the interpreter (which I did) and to change the path in the main function which looks something like :
os.environ['PATH'] = \
path.dirname(sys.executable) + ':' + os.environ['PATH']
which was a bit unclear to me. Any ideas?
I have pylint 0.25.1, installed using easy_install (Python 2.7, Win XP). Both pylint and pylint.bat were installed in Python27/Scripts (this directory is in my PATH).
I too get the "The system cannot find the file specified" error when running the pylint_etc_wrapper.py script unchanged.
Running pylint from the script does work if
command = 'pylint'
is changed to
command = 'pylint.bat'
Another way to make it work is to add shell=True to the Popen() call.
I can't really explain all this, but there is an unresolved Python bug that looks like it might be relevant: http://bugs.python.org/issue8557.

Categories