Why does python subprocess zip fail but running at shell works? - python

I'm on Mac OS X using Python 2.7; using subprocess.call with zip fails yet running the same command at the shell succeeds. Here's a copy of my terminal:
$ python
Python 2.7.2 (default, Oct 11 2012, 20:14:37)
[GCC 4.2.1 Compatible Apple Clang 4.0 (tags/Apple/clang-418.0.60)] on darwin
Type "help", "copyright", "credits" or "license" for more information.
>>> import subprocess
>>> subprocess.call(['zip', 'example.zip', 'example/*'])
zip warning: name not matched: example/*
zip error: Nothing to do! (example.zip)
12
>>> quit()
$ zip example.zip example/*
adding: example/file.gz (deflated 0%)
I've also tried with full paths and had the same result.

Because running a command in the shell is not the same thing as running it with subprocess.call(); the shell expanded the example/* wildcard.
Either expand the list of files with os.listdir() or the glob module yourself, or run the command through the shell from Python; with the shell=True argument to subprocess.call() (but make the first argument a whitespace-separated string).
Using glob.glob() is probably the best option here:
import glob
import subprocess
subprocess.call(['zip', 'example.zip'] + glob.glob('example/*'))

Martijn's advice to use glob.glob is good for general shell wildcards, but in this case it looks as if you want to add all files in a directory to the ZIP archive. If that's right, you might be able to use the -r option to zip:
directory = 'example'
subprocess.call(['zip', '-r', 'example.zip', directory])

Try shell=True. subprocess.call('zip example.zip example/*', shell=True) would work.

Related

Installed Libraries are not being found by my IDE (Mac) [duplicate]

I am trying to import pyodbc module on a windows computer. It works in the terminal, but not the IDLE. The error message in IDLE is:
Traceback (most recent call last):
File "FilePath/Filename.py", line 3, in <module>
import pyodbc
ImportError: No module named pyodbc
This typically occurs when multiple versions of python are installed with different paths. You can check to see if you have multiple installations by opening up the IDLE terminal and using
import sys
sys.version
sys.path
These commands will print the system PATH and version of the current instance of python. Use this in both IDLE and the command line terminal to see where each differ. Once you know which version is the one you want then just remove the other. You could also remove all python instances and then reinstall a clean python environment but then you would have to re-install all of your modules using pip or easy_install
Open python in cmd (type python and press enter)
Import the module in cmd (type import modulename)
Type modulename.__file__
You will get the path where the module is stored
Copy the corresponding folder
In IDLE, import sys and typing sys.executable to get the paths where it looks for modules to import
Paste your module's folder in the path where IDLE looks for modules.
This method worked for me.
You can pip show after install package and know about location where package installed.
After that check in IDLE sys.path and if directory with package not in sys.path try to add it.
import sys
sys.path.append("/home/dm/.local/lib/python3.6/site-packages")
# or another folder that `pip show` about package.
this happen because of multiple python installed (32bit version, 64bit version) or 3v and 2.7v so to solve this problem you have to invoke the idle for that specific version like this
cd to the dir of the version that the import work fine in cmd in that folder type this command below
pythonw.exe Lib\idlelib\idle.pyw
this command will invoke idle for that version
and the import will work fine
Me too had the same issue while trying to import a module which was successfully imported on terminal and not able to install on IDLE.
How I fixed?
Assuming you know how to execute commands on terminal as well as inside of python interpreter
Open your Terminal & execute the below commands :
:~$ python3
Python 3.6.9 (default, Jan 26 2021, 15:33:00)
[GCC 8.4.0] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>>
>>>
>>> import sys
>>> sys.version
'3.6.9 (default, Jan 26 2021, 15:33:00) \n[GCC 8.4.0]'
>>> sys.path
['', '/usr/lib/python36.zip', '/usr/lib/python3.6',
'/usr/lib/python3.6/lib-dynload', '/usr/local/lib/python3.6/dist-
packages', '/usr/lib/python3/dist-packages']
>>>
Now import your module inside of your python3 interpreter.
>>> import nester
>>>
>>> nester.__file__
'/usr/local/lib/python3.6/dist-packages/nester.py'
>>>
Open your IDLE and run the below commands and compare them
Python 3.9.1 (tags/v3.9.1:1e5d33e, Dec 7 2020, 17:08:21) [MSC v.1927
64 bit (AMD64)] on win32
Type "help", "copyright", "credits" or "license()" for more
information.
>>> import sys
>>> sys.version
'3.9.1 (tags/v3.9.1:1e5d33e, Dec 7 2020, 17:08:21) [MSC v.1927 64 bit
(AMD64)]'
>>> sys.path
['','C:\Users\username\AppData\Local\Programs\Python\Python39\Lib\idlelib', 'C:\Users\username\AppData\Local\Programs\Python\Python39\python39.zip', 'C:\Users\username\AppData\Local\Programs\Python\Python39\DLLs', 'C:\Users\username\AppData\Local\Programs\Python\Python39\lib', 'C:\Users\username\AppData\Local\Programs\Python\Python39', 'C:\Users\username\AppData\Local\Programs\Python\Python39\lib\site-packages']
>>> sys.executable
'C:\Users\username\AppData\Local\Programs\Python\Python39\pythonw.exe'
Now if you compare both outputs from Terminal & IDLE,
Terminal Module location is different from IDLE
I was using Ubuntu 18 terminal on windows machine
So I just copied my file to 'C' directory and ensured its file privileges. That's it.
:~$ cp -p /usr/local/lib/python3.6/dist-packages/nester.py /mnt/c/Users/username/AppData/Local/Programs/Python/Python39/Lib/
It worked!!
I Found the solution. It works for me
The problem is your installation directory does not match with the python version directory.
solution is >>>
type %localappdata% in your search bar then go to this folder.
here select the program folder. then select Programs , Python , Python version , Scripts
copy the location of the Scripts folder
open command prompt and type cd //yourpath (in my case cd C:\Users\3C HOUSE\AppData\Local\Programs\Python\Python37\Scripts)
if you wanna install numpy , now run pip install numpy
When you put your python scripts that have import pandas in the same folder as the site packages like pandas for example and use the same version of python that is used on CMD, it should help run your scripts in IDLE.
Check the path of your code, and that of the module. Copying the module to the path where code is worked for me.
'sys.executable' will give the path where code is stored.
For windows, open command prompt and enter pip show pyodbc to get the path of package and copy the path.
then open idle and run these lines
import sys
sys.path
Match the path from command prompt and the paths mentioned in the list provided by running above lines in IDLE. If the path is not mentioned then run these lines in idle
sys.path.append("Enter the copied path of package here")
After executing these lines, check again by importing the package that if it works for you.

Why doesn't python send output to stdout?

Here's a minimum working example (MWE), saved as mwe.py:
import sys
def f(n):
print("Testing print()...")
sys.stdout.write("Calculating f({})...".format(n))
When run from the command line I get no output whatsoever:
username#hostname:~/mydir$ python mwe.py 'f(99)'
username#hostname:~/mydir$
When run from within python
I get output (some info removed):
username#hostname:~/mydir$ python
Python 3.5.4 (default, DATE, HH:MM:SS)
[GCC X.X.X Compatible Apple LLVM X.X.X (clang-X.X.X)] on darwin
Type "help", "copyright", "credits" or "license" for more information.
>>> from mwe import f
>>> f(99)
Testing print()...
Calculating f(99)...
>>>
Why do these output statements work within python but not from the command line?
python mwe.py 'f(99)' doesn't mean "run the f function from mwe.py with argument 99". If you wanted to do that from the command line, you could execute
python -c 'import mwe; mwe.f(99)'
python mwe.py 'f(99)' means "run the script mwe.py with sys.argv[1] set to the string "f(99)"". The script mwe.py doesn't examine sys.argv or print anything at all; it just defines a function and ends.
This: python mwe.py 'f(99)' just shouldn't work. In this case, 'f(99)' is just passed as an argument to the program.
Try using python -c 'import mwe; mwe.f(99) instead. (also read more about command line usage of python by typing python -h)

stdin reading blocking when running sbt with python subprocess.Popen()

I'm launching sbt via Popen(), and my python process stdin reading is not working.
Here is an example:
On the first line I'm launching Popen, on the second line I'm trying to browse throught the history with an arrow key. This does not work for some time, printing ^[[A.
$ python
Python 2.7.10 (default, Jul 13 2015, 12:05:58)
[GCC 4.2.1 Compatible Apple LLVM 6.1.0 (clang-602.0.53)] on darwin
Type "help", "copyright", "credits" or "license" for more information.
>>> import subprocess; f = open("/dev/null", "rw"); subprocess.Popen(["sbt"], stdout=f, stderr=f, stdin=f)
<subprocess.Popen object at 0x10fc03950>
>>> ^[[A^[[A^[[A^[[A^[[A^[[A^[[A^[[A^[[Aimport subprocess; f = open("/dev/null", "rw"); subprocess.Popen(["sbt"], stdout=f, stderr=f, stdin=f)
This seems to only happen with sbt.
Any idea why and how to bypass this behavior ?
Thanks
My guess is that sbt is misbehaving when there is no pseudo-tty to interact with the user (probably because of jline).
Hence, let's use a python module to run the commands in a pseudo-tty. Install pexpect via pip (pip3 install pexpect for Python 3.x).
Then, run the following:
import pexpect, sys
f = open("sbt.log", "w+")
# Use `spawnu` for Python3, `spawn` otherwise
sbt = pexpect.spawnu("sbt -Dsbt.log.noformat=true \"version\" \"another-command\"", logfile=f)
# Do whatever is needed while sbt is running
# Force the process to expect EOF and file to be written
sbt.expect(pexpect.EOF)
Tested in Python 3.4.3 (Gentoo Linux) and works.

os.system not finding file that is really there

This happens in the python build:
#is it executable
print os.access("support/d8/d8", os.X_OK)
#is it there in the shell
os.system("test -f support/d8/d8 && echo \"found\" || echo \"not found\"")
and then:
#run it
os.system("support/d8/d8 --trace_exception with a bunch of files");
which outputs:
True
found
sh: 1: support/d8/d8: not found
I don't get it. It's there it's executable. Why is it not there when I start it.
link to the travis build: https://travis-ci.org/albertjan/skulpt/builds
and a link to the repository: https://github.com/albertjan/skulpt the build script is called m
You're running an x86_32 bit executable d8 (despite the comment, by the way). If the (Travis) system is x64, and/or does not have all of the x86_32 libraries
linux-gate.so.1
libpthread.so.0
libstdc++.so.6
libm.so.6
libgcc_s.so.1
libc.so.6
then the executable won't run, since the loader cannot find all required libraries. Build statically and/or for x64.
Why don't you try this:
os.system("./support/d8/d8 --trace_exception with a bunch of files");
I had a similar issue, while executing the ./ is some how required.
If your file "support/d8/d8" has a non existent "bang line" then this error occurs
$ cat support/d8/d8
#!/usr/bin/thisdoesnotexist
echo "hello"
$ chmod 755 support/d8/d8
$ python
Python 2.7.3 (default, Apr 10 2013, 06:20:15)
[GCC 4.6.3] on linux2
Type "help", "copyright", "credits" or "license" for more information.
>>> import os
>>> os.system("support/d8/d8 --wer")
sh: 1: support/d8/d8: not found
32512
Adding the full file path seems to resolve the Error in Python 3.x.
Example: /home/[your_username]/support/d8/d8
You can use pwd to get the path of your current directory. Then add your filename and you're good to go.
Example: Let's say pwd outputs /home/jakob/project and your script is called test.py the line has to look like this: os.system('/home/jakob/project/test.py').
PS: don't forget to import os at the beginning of your script.

Can python shell have some pre-input?

When testing in python shell, I always have to type some import like:
Python 2.5.4 (r254:67916, Jun 24 2010, 15:23:27)
[GCC 4.4.3] on linux2
Type "help", "copyright", "credits" or "license" for more information.
>>>import sys
>>>import datetime
Can someone help me to automatically finish these? It means I run some command to enter python shell it has already done import for me, and a python shell waiting for me to continue type command.
Thanks.
Try:
python -i -c "import sys; import datetime;"
More info:
-i : inspect interactively after running script; forces a prompt even
if stdin does not appear to be a terminal; also PYTHONINSPECT=x
&
-c cmd : program passed in as string (terminates option list)
Create a file with the commands you want to execute during startup, and set the environment variable PYTHONSTARTUP to the location of that file. The interactive interpreter will then load and execute that file. See http://docs.python.org/tutorial/interpreter.html#the-interactive-startup-file
On a sidenote, you might want to consider ipython as an improved Python shell when working in interactive mode.

Categories