(I am using Windows.)
I am trying to run maven from a python script. I have this:
import subprocess
mvn="C:\\_home\\apache-maven-2.2.1\\bin\\mvn.bat --version"
p = subprocess.Popen(mvn, shell=True, stdout = subprocess.PIPE)
stdout, stderr = p.communicate()
print p.returncode # is 0 if success
It works fine, but I am wondering about the following:
Better ways to add parameters instead of appending the string.
Maybe some specific way to run maven without the above.
A way to show the output (currently it only prints a 1 or 0 based on failure/success).
What I am trying to accomplish long term (I note this in case someone has a better method) is to make a simple script to build a list of projects and move another list of files (jars/other modified things) to a folder to deploy to VMs, it's a huge pain to do manually. I have this working in a batch script no sweat, I am just curious to learn Python and wonder if it'd be easier to manage because I could just make a couple of lists and iterate over each of the locations rather than have a line for each task in the batch script.
(Short version of what my batch script looks like.)
#set version=7.8.3
#set staging_folder=C:\Users\me\Desktop\staging
#set stage_was=%staging_folder%\was
#set stage_ear=%stage_was%\stuffui.ear
#set stage_war=%stage_ear%\stuff-%version%.war
:: delete stage contents
call del /s /q %staging_folder%
call rmdir /s /q %stage_was%
:: make folders
call mkdir %stage_ear%
call mkdir %stage_war%\WEB-INF\lib
:: maven builds
call mvn -f C:\workspace\pom.xml -pl proj1,proj2 clean install
:: copy to stage
call xcopy C:\workspace\proj1\target\thing1.jar %stage_ear%\ /i /y
call xcopy C:\workspace\proj2\target\thing2.jar %stage_ear%\ /i /y
call xcopy C:\workspace\proj2\target\thing2.jar %stage_war%\WEB-INF\lib\ /i /y
There is the Apache Maven Invoker API.
Mark's answer to Accessing Java API information with Python mentions:
Jython which is Python run on the Java VM.
See my answers there for an example on how to use the Maven Invoker (from within Java in this particular case).
Re:
in case someone has a better method
Re:
move another list of files (jars/other modified things) to a folder
Have you considered using Maven itself (the copy-resources goal of its Resources Plugin in this particular case)?
Re:
I am just curious to learn Python
Since you are working with Java anyway: Have you considered Groovy as the scripting language of your choice?
Some of its advantages:
The Java language is a subset of the Groovy language. I.e. every Java code is also Groovy code. You don't have to use the full-fledged Groovy syntax from the beginning while learning it.
It can be scripted.
It is supported as scripting and DSL in tools like Jenkins.
Related
I'm writing a program that executes short shell one liners (potentially including pipes and background tasks etc), and I'd like to make it "just work" cross platform as much as possible.
For mac/linux the following seems to work well:
shell = os.environ.get("SHELL", "/bin/bash")
subprocess.Popen([shell, "-c", script_content])
However given that on windows:
SHELL isn't usually set
Assuming that bash is installed, a usable bash executable might be found in a variety of different places
What's the best way to make this work as reliably as possible in windows?
Are you looking for C:\Windows\System32\cmd.exe?
And if you are doing cross platorm you could use sys.platorm to find the platform the user is using.
On Window, 'COMSPEC' holds the name of the current command program. You can write unconditional lookup to lookup in the environemnt. Usually, better to take this approach, as in many cases, python script may be executed from 'git-bash', WSL or similar. No need to explicitly program for specific platform.
First Using SHELL
If none, use COMSPEC
See: https://en.wikipedia.org/wiki/COMSPEC
If you have Windows git client installed, you should have git bash, so in a CMD window:
set SHELL="c:\Program Files\Git\bin\bash.exe"
then you can run you python program.
Our environment has a shell script to setup the working area. setup.sh looks like this:
export BASE_DIR=$PWD
export PATH=$BASE_DIR/bin
export THIS_VARIABLE=THAT_VALUE
The user does the following:
% . setup.sh
Some of our users are looking for a csh version and that would mean having two setup files.
I'm wondering if there is a way to do this work with a common python file. In The Hitchhiker's Guide to Python Kenneth Reitz suggests using a setup.py file in projects, but I'm not sure if Python can set environment variables in the shell as I do above.
Can I replace this shell script with a python script that does the same thing? I don't see how.
(There are other questions that ask this more broadly with many many comments, but this one has a direct question and direct single answer.)
No, Python (or generally any process on Unix-like platforms) cannot change its parent's environment.
A common solution is to have your script print the output in a format suitable for the user's shell. E.g. ssh-agent will print out sh-compatible global assignments with -s or when it sees that it is being invoked from a Bourne-compatible shell; and csh syntax if invoked from csh or tcsh or when explicitly invoked with -c.
The usual invocation in sh-compatible shells is $(eval ssh-agent) -- so the text that the program prints is evaluated by the shell where the user invoked this command.
eval is a well-known security risk, so you want to make this code very easy to vet even for people who don't speak much Python (or shell, or anything much else).
If you are, eh cough, skeptical of directly supporting Csh users, perhaps you can convince them to run your sh-compatible script in a Bourne-compatible shell and then exec csh to get their preferred interactive environment. This also avoids the slippery slope of having an ever-growing pile of little maintenance challenges for supporting Csh, Fish, rc, Powershell etc users.
Excuse the awkward question wording.
I've made a script. I would like for others to download it from github, and run it by typing programName argument1 argument2, similar to any other popular app used through the terminal such as Jupyter or even opening Atom/Sublime/etc. (ex:jupyter notebook, atom .). However, unlike Jupyter or sublime, my script isn't launching another app, it's a small app meant to be used in the shell.
Currently, to use my script, one must type into the command line python programName.py arg1 etc from within the file's directory.
How do I allow others to dl it and use it from anywhere (not having to be within the directory), without having to type out the whole python programName.py part, and only having to type programName arg1?
This blog post explains step by step how to create a distribution that you can install and it would turn into an executable.
You can refer to this github repo for a sample application.
The full documentation of setuptools is available here.
In general, you should configure your setup.py in order to use the command in the entry-point option:
setup(
name = "your_app_name",
packages = ["package_name"],
entry_points = {
"console_scripts": ['cmd_name = package_name.package_name:main']
},
....
)
This solution would work on every OS where you have installed python.
Your script may need to have an interpreter, "shebang", besides being "reachable" by the $PATH
#!interpreter [optional-arg]
For example, you could have something like
#!/usr/bin/env python
or to force a specific version
#!/usr/local/bin/python2.7
Next, your script needs to be available within the $PATH, check this answer that covers that part: https://unix.stackexchange.com/q/29608/53084
You can simply add your script to PATH variable in order to launch it from anywhere.
In Linux distros, you can simply do it by using a bash command PATH=$PATH:/path/to/your/script.
Make sure you don't have the space around the "=" operator.
Now, the second thing is you don't want your script to be named as pythonProgram.py.You can simply remove the extension .py from PythonProgram.py by adding a single line to the starting of your script.
Open up your script and at the very begining type #!/usr/bin/python.This should be the first line of your code.This line is called shebang and is used to tell the bash which interpreter to be used for compiling the script.
If everything went right, you will be able to run your script as pythonProgram arg1.
In addition to mabe02: Python is a scripting language and usually not compiled, which means you will need an interpreter to run your program.
Programms made in C f.e. can be run on its own because they were compiled into machine code by a compiler.
An interpreter is similar to a compiler as it reads your script and interprets it at runntime. That is why you need to write python before your programm, to tell your computer to interpret your script on runntime, using python. This doesn't mean that there are no possibilities to compile python as can be seen in the other answer and in this link Can a python program be run on a computer without Python? What about C/C++? (py2exe and py2app).
Basically, what I want to do is this (in a psuedo bash-ish code)
#create ramdisk raid
diskutil erasevolume HFS+ "r1" `hdiutil attach -nomount ram://4661720`;
diskutil erasevolume HFS+ "r2" `hdiutil attach -nomount ram://4661720`;
diskutil createRAID stripe SpeedDisk HFS+ /Volumes/r1 /Volumes/r2;
#copy minecraft server files to ramdisk
cp minecraft_Server /Volumes/SpeedDisk
#start minecraft_server
cd /Volumes/SpeedDisk/minecraft_server
java -Xms2G -Xmx2G -jar minecraft_server.jar nogui
#once I stop the server, copy the files to my harddrive
cd ~
cp /Volumes/SpeedDisk/minecraft_server minecraft_server/
I'm not sure about how to do this ^ in real life :p I was considering using python but it seems like there are problems with os.system for copying files.
Also, I would like to know if there is a way for me to eject the ramdisks when I am done. This is all going to be done in Mac OS X Leopard. The reason I'm doing all of this is to speed up my minecraft server a bit without buying an SSD.
I was considering using python but it seems like there are problems with os.system for copying files.
...then use the right tool for the job:
shutil.copytree()
Shell scripting seems to be the best solution for this kind of problem ( assuming that you want this to work on a single platform mac osx ). Write a shell script with these commands and use that script everytime you want to execute these commands.
I have a scala program that I would like to call from within django using subprocess:
encode_cmd = "/usr/local/share/scala/scala-2.10.0/bin/scala -cp /home/django/code/classes conn {}".format(self.id)
output = subprocess.Popen(encode_cmd, shell = True, stdout = subprocess.PIPE).stdout.read()
This code runs fine in the python shell, but when run as part of the normal webserver process it fails to find the scala dependencies (the scala class references the slick libraries for example) failing with the java.lang.NoClassDefFoundError.
I've tried specifying specific users as part of the mod_wsgi daemon process, but this makes no difference.
You should add jars in your command like that: -cp /home/django/code/classes:/path/to/slick.jar, otherwise it only includes the .class' and folders containing class files as per packages.
You can always rely on the shell expansion if you have many jars: /path/to/dependencies/*.jar
Another option is using SBT's xsbt-start-script-plugin or Maven's appassembler plugin to create a shell script