I'm writing a script that emulates a unix environment with python (yes, I know that may sound silly).
Basically I set up the "pwd" and "ls" commands before the "cd" command. Now I need my python script to navigate itself around directories by moving itself. I was wondering if anyone could point me in the right direction? eg.What commands to use, etc
Note. I would prefer it if I only had to use built-in modules.
Thank You!
I am not sure I understand you properly, but if you are talking about moving through the standard unix file system and not copying the script to different locations, you could easily just wrap the three commands (pwd,ls,cd) as python functions, just as you were describing.
In fact, I did this as part of the wx_py project; they aren't perfect clones, but they work well enough. Here are the three functions:
import os
import glob
def pwd():
print os.getcwd()
def cd(path,usePrint=True):
os.chdir(os.path.expandvars(os.path.expanduser(path)))
if usePrint:
pwd()
def ls(str='*',fullpath=False):
g=glob.glob(os.path.expandvars(os.path.expanduser(str)))
if fullpath:
for i in g:
print i
else:
for i in g:
print os.path.split(i)[1]
The full file is at:
https://github.com/davidmashburn/wx_py/blob/master/wx_py/path.py
I'm sure others may have done a better job, so please just take these functions as a starting point.
Related
I am playing around with a python software-like code that I downloaded. I need to extract some parts of the code and so I am currently tracing it to understand it better. It would make my life much easier to trace the code in an IDE (I use spyder) so that I can put breakpoints and visually inspect the variables. I therefore want to modify the code but I don't know how to proceed.
This is the directory structure of the code. The main directory is "advance" and 2 of its subdirectories are "advance" and "examples". The code was installed using pip and to run the code, one must go to the examples directory, enter a sub-directory which represents a test case, and simply type "advance ." in the terminal (the "." represents pwd).
After some tracing, I found out that doing this calls the file driver.py with address advance/advance/driver.py. The driver.py code has the structure below:
# import stuff here
def main():
parser = argparse.ArgumentParser(description="Read working directory")
# some stuff here
if __name__ == "__main__":
try:
main()
except:
print(traceback.format_exc())
print("Execution failed")
As you can see, the main function takes an input which is the working directory.
My question now is: How can I modify this so that I can run this code in my IDE? Or better yet, can I write a script that calls the main function in driver.py and give it the path of the directory for the example test case I want to run? If so, how can I do it?
Thanks and apologies if it sounds too simple. I am coming from Matlab and transitioning to python.
I found in this reddit post that spyder offers to pass command line options, when running a script. See the following setting:
Run -> Configure -> Command line options
Unfortunately, I am not familiar with Spyder, so that's what I can tell so far. I guess it works similar to PyCharm (which I use). There it is like this: You adapt your run configuration and can set the parameters you want to pass a script. Afterwards, when running that run configuration PyCharm will always pass those particular parameters to the script. This allows to use the IDE's debugger to examine the code. I would bet that Spyder behaves almost the same way. Good luck trying the suggested settings :)
I am trying to do a bit of math not covered under the numpy or scipy packages. Since I have to do these calculations many times, I thought I'd create my own function in my own mathystuff.py module and import it into my workspace. Easy enough, except python doesn't like importing the function! Here's what I did:
First, I created my file mathystuff.py which is located in my venv/lib/python3.7/site-packages/ folder like the rest of my modules.
import numpy as np
def mathfunction(input1, input2):
#do some math calculations here - lots of np.stuff()
#and some more math calculations here, you get it
return answer
Simple as that. In my main project, I simply call:
from mathystuff import mathfunction
where I'm met with the error cannot import name 'mathfunction' from 'mathystuff' pointing me to the correct location of the .py file. I've done this a million times before and even done this with other modules I've made in the same project, so why would this happen?
There can be many reasons, but according to limited information provided I will go with the ones I think are most possible. (PS I would've commented to ask questions but unfortunately I don't have enough rep).
Check if:
Your virtual env is activated. (I know it sounds silly but it is possible to forget activating venv)
If it's active, then check the location of python by typing which python into terminal (or which python3 if you are running your code with python3). This will return the directory python (eg: path/to/folder/venv/bin/python3). While working on multiple venvs, it's easy to get confused and work on venvA while venvB is active.
If all above checks, go into venv/lib/ directory and confirm that you have only one python directory there. Like venv/lib/python3.8 or venv/lib/python3.7. If, for some reason, there are two (or more) python versions in venv/lib/ then make sure the one you use is the one your module is located in.
These are the most common errors I came across so far.
I get the wrong path back. The Datafile is in D:... and get everytime the path C:\Python27\lib\site-packages\xy back from python. I use the function
path = getcwd()
How can I fix it?
You may be executing the script in a different place than your intended directory.
Solution 1: Move the .py file to the target directory, and execute it there.
Pros:
Easy
Works cross-platform (and for other users - if you do this, use getcwdu for Unicode)
No hard-coded path strings
Cons:
File must be in the same or higher directory as target folder
Solution 2: Manually write the string of the path to the folder.
Pros:
'Just Works'
Cons:
Annoying bugs w/typos
Need to re-code every time you change directories
Won't work anywhere else
In my idea, make sure your IDE has your target directory opened as your project folder.
After all, it's a debug-time error, and won't affect the smooth running of your program in the runtime, if your program's launch script has the right address for your program to run, and does its part normally!
By the way #order, I'm totally against hardcoding a path into your code, because it's just a very lame programming exercise!
Just get yourselves used to going through the right exercise, although it doesn't seem very beneficial in the short run. in the coming years of your career, you're going to be against using the cons and pros of such a comparison, but it will become a right or wrong coding exercise!
Cheers.
Ok i have these commands used in batch and i wanted to know the commands in python that would have a similar affect, just to be clear i dont want to just use os.system("command here") for each of them. For example in batch if you wanted a list of commands you would type help but in python you would type help() and then modules... I am not trying to use batch in a python script, i just wanna know the similarities in both languages. Like in english you say " Hello" but in french you say "Bonjour" not mix the two languages. (heres the list of commands/functions id like to know:
change the current directory
clear the screen in the console
change the prompt to something other than >>>
how to make a loop function
redirections/pipes
start an exteral program (like notepad or paint) from within a script
how to call or import another python script
how to get help with a specific module without having to type help()
#8: (in batch it would be command /?)
EDITED COMPLETELY
Thanks in Adnvance!
You can't just mechanically translate batch script to Python and hope that it works. It's a different language, with different idioms and ways of doing things, not to mention the different purpose.
I've listed some functions related to what you want below, but there's no substitute for just going and learning Python!
os.chdir
os.system("cls") is probably the simplest solution
Change sys.ps1 and sys.ps2.
Nope, there are no gotos in Python. Use for and while loops instead.
Doesn't make sense, use Python's IO instead.
subprocess.Popen
Doesn't make sense, use import or subprocess.Popen instead.
help
Most of the things you've mentioned (start, cls etc.) are not "batch commands", they're executable programs which perform certain tasks. The DOS "shell" simply executes these when it encounters them in a file. In this sense, "python" is the equivalent of a single executable (like cls).
Now that that's clear, cd (and most other OS specific tasks) are accomplished using the os module. There's no single Python statement to clear the screen - that would be wasteful. Changing the prompt of the python interpreter can be done by assigning to sys.ps1. Loops are done using while or for. Redirection doesn't happen. YOu can however use the subprocess module to run subcommands and send their outputs to files or other streams. Starting commands is done using the subprocess.Popen function. For getting help, you can either do help("command") or if you're using ipython, just say command? and hit enter.
You should really go through the tutorial rather than trying to map batch commands to Python.
The Python docs are excellent, and are the place to start. For doing shell-script like things, you'll want to check out:
http://docs.python.org/library/os.html?module-os
http://docs.python.org/library/os.path.html#module-os.path
http://docs.python.org/library/shutil.html#module-shutil
http://docs.python.org/library/subprocess.html#module-subprocess
Python is not a system shell, Python is a multi-paradigm programming language.
If you want to compare .bat with anything, compare it with sh or bash. (You can have those on various platforms too - for example, sh for windows is in the MinGW package).
I am pretty much facing the same problem as you, daniel11. As a solution, I am learning BATCH commands and their meaning. After I understand those, I am going to write a program in Python that does the same or accomplishes the same task.
Thanks to Adam V. and katrielatex for their insight and suggestions.
Spinning off from another thread, when is it appropriate to use os.system() to issue commands like rm -rf, cd, make, xterm, ls ?
Considering there are analog versions of the above commands (except make and xterm), I'm assuming it's safer to use these built-in python commands instead of using os.system()
Any thoughts? I'd love to hear them.
Rule of thumb: if there's a built-in Python function to achieve this functionality use this function. Why? It makes your code portable across different systems, more secure and probably faster as there will be no need to spawn an additional process.
One of the problems with system() is that it implies knowledge of the shell's syntax and language for parsing and executing your command line. This creates potential for a bug where you didn't validate input properly, and the shell might interpet something like variable substitution or determining where an argument begins or ends in a way you don't expect. Also, another OS's shell might have divergent syntax from your own, including very subtle divergence that you won't notice right away. For reasons like these I prefer to use execve() instead of system() -- you can pass argv tokens directly and not have to worry about something in the middle (mis-)parsing your input.
Another problem with system() (this also applies to using execve()) is that when you code that, you are saying, "look for this program, and pass it these args". This makes a couple of assumptions which may lead to bugs. First is that the program exists and can be found in $PATH. Maybe on some system it won't. Second, maybe on some system, or even a future version of your own OS, it will support a different set of options. In this sense, I would avoid doing this unless you are absolutely certain the system you will run on will have the program. (Like maybe you put the callee program on the system to begin with, or the way you invoke it is mandated by something like POSIX.)
Lastly... There's also a performance hit associated with looking for the right program, creating a new process, loading the program, etc. If you are doing something simple like a mv, it's much more efficient to use the system call directly.
These are just a few of the reasons to avoid system(). Surely there are more.
Darin's answer is a good start.
Beyond that, it's a matter of how portable you plan to be. If your program is only ever going to run on a reasonably "standard" and "modern" Linux then there's no reason for you to re-invent the wheel; if you tried to re-write make or xterm they'd be sending the men in the white coats for you. If it works and you don't have platform concerns, knock yourself out and simply use Python as glue!
If compatibility across unknown systems was a big deal you could try looking for libraries to do what you need done in a platform independent way. Or you need to look into a way to call on-board utilities with different names, paths and mechanisms depending on which kind of system you're on.
The only time that os.system might be appropriate is for a quick-and-dirty solution for a non-production script or some kind of testing. Otherwise, it is best to use built-in functions.
Your question seems to have two parts. You mention calling commands like "xterm", "rm -rf", and "cd".
Side Note: you cannot call 'cd' in a sub-shell. I bet that was a trick question ...
As far as other command-level things you might want to do, like "rm -rf SOMETHING", there is already a python equivalent. This answers the first part of your question. But I suspect you are really asking about the second part.
The second part of your question can be rephrased as "should I use system() or something like the subprocess module?".
I have a simple answer for you: just say NO to using "system()", except for prototyping.
It's fine for verifying that something works, or for that "quick and dirty" script, but there are just too many problems with os.system():
It forks a shell for you -- fine if you need one
It expands wild cards for you -- fine unless you don't have any
It handles redirect -- fine if you want that
It dumps output to stderr/stdout and reads from stdin by default
It tries to understand quoting, but it doesn't do very well (try 'Cmd" > "Ofile')
Related to #5, it doesn't always grok argument boundaries (i.e. arguments with spaces in them might get screwed up)
Just say no to "system()"!
I would suggest that you only use use os.system for things that there are not already equivalents for within the os module. Why make your life harder?
The os.system call is starting to be 'frowned upon' in python. The 'new' replacement would be subprocess.call or subprocess.Popen in the subprocess module. Check the docs for subprocess
The other nice thing about subprocess is you can read the stdout and stderr into variables, and process that without having to redirect to other file(s).
Like others have said above, there are modules for most things. Unless you're trying to glue together many other commands, I'd stick with the things included in the library. If you're copying files, use shutil, working with archives you've got modules like tarfile/zipfile and so on.
Good luck.