Closed. This question needs to be more focused. It is not currently accepting answers.
Want to improve this question? Update the question so it focuses on one problem only by editing this post.
Closed 7 years ago.
Improve this question
I am planning on creating a software that will be used to analyse biological data, some of us biologists would not have much programming skills yet we do alot of repetitive tasks and are required to write more scripts or programs that we always use using.
i have decided to narrow it down to only one very important task, its a searching script, i.e to query a database of genes-(data), there is already a program that does that called ncbi-blast, usually one needs to be familiar with command line and also requires ubuntu or windows, but its a hassle if you have to analyse the huge quantities of results generated.
so i have want to package some of my python scripts into a software so that others can find it convenient to analyse their work
Open vs. Closed Source (Licensing)
i intend to make some of the functionalities freely available to the user in the software, although at a later stage, i intend to incorpaorate pipelines that would require an affordable license
testing
so with this one function, it makes testing alot simpler-i dont know yet, and i would not mind starting with linux-ubuntu and windows as the platforms.
I appreciate your advice on choosing just one language, and i will go with python, how ever i would like tyhe software to atleast support other free programs created in other languages like R, as in plugins that a client installs when they need them.
I hope this shades more light to my already complex situation
Thank you
Sounds like you're at the beginning of an adventure and some new learning curves. I will applaud your willingness to create. Here are some things to consider as you get started on your journey.
fyi: "software" is typically a single word, no space.
Based on the context of "a soft ware" it seems like you mean "application".
I mention "software" and "application" not to give you a hard time about it, but because I think 1) it will help you in future searches when you look for ways to get something done (e.g. you'll get more useful hits on "python application installer" than "python soft ware installer", and 2) you can more clearly explain to people what you are attempting.
So. Your application(s) will need a user interface for your collection of scripts. Do you want a graphical user interface (GUI)? Or a command line interface (CLI)?
GUI: more complex than CLI:
For a GUI it will be more complex to get something that runs on both Windows & Linux.
So far as I know there isn't a trivial way to create a single GUI that runs on both Windows & Linux without adding another tool. Maybe python has easy GUI user-interface stuff in it and you could use that.
For perl I will point you here: "Perl GUI programming on Windows" Perl GUI programming on Windows
This lists some Perl approaches to consider, you will have to research them and see which ones are also able to run on Linux. Be careful about including additional libraries you'll need to package (or document how to install) with you application.
CLI: more simple than a GUI:
Since they are just scripts today, I would suggest starting with a CLI which probably means cleaning up your argument handling ( #ARGV in Perl, I can't comment on Python).
Read through your code and find literals that you change when you run a script for something new, those things will become your arguments. And if it starts to seem like you are creating too many arguments maybe you want to look into a configuration file (properties file of some kind).
It will be easier for you to get started with a CLI and you can always come back and add a GUI option in a future release. I find that designing a well thought out CLI makes it easier to focus on what is important for a GUI so it should help your eventual GUI be even better.
Packaging
Packaging your application is going to be a challenge.
Do your scripts need any libraries that aren't part of default with python & perl installs? If so you need to work out a way to supply those (e.g. include them with your distribution or include documentation that your users can follow to download and install the libs).
I can't comment on pyinstaller.
For Perl I will point you to this question on distributing a Perl Application : Distributing a Perl Application
User Skill
How much skill do you expect your users to have?
Will they be ok with installing a python and perl interpreters if necessary? Not every user has that technical skill.
Do you want to make sure your scripts verify the minimum versions of python & perl they need?
Documentation
See User Skill, above. You will need to make documentation available.
This will be driven by who is going to use your scripts. What is your target user like? Can you write a "5 line" summary that would make me want to download them and try them? That will be a helpful exercise to help you focus on how you want to present the scripts.
Open vs. Closed Source (Licensing)
This also ties into licensing. Are you aiming for a commercial product? Giving the world another gift of open source? Those are both fine things to do, but you'll want to choose a suitable license for your application. You also have some work to do if you want to avoid distributing your script's source code (I'm not suggesting you pursue one approach or the other, just that you make the decision before you put it on the internet at large).
Testing
Testing your application is going to be a challenge.
If you don't have automated tests for your scripts you will go crazy trying to verify it runs everywhere you want it to run.
Do you have unit tests & test suites for your scripts so you can verify they run correctly on different versions of Windows and various Linux distributions?
If your scripts are pretty simple maybe it doesn't matter if they're running on 32 bit vs. 64 bit operating systems.
Maybe it doesn't matter if they run on Windows 10, Windows 8.x, Windows 7.x, maybe various versions of Windows Server (2012, 2008, ...).
Maybe it doesn't matter if they run on RedHat, Suse, Ubuntu, Mint, ...
You probably want some sanity checks to verify that your install program worked correctly and the environment is suitable.
Without knowing what your scripts do it is kind of hard to say how much testing they benefit from.
Free Advice: choose just one scripting language and run with that
I will end with some completely free advice (worth what you're paying for it :-) )
Think really hard about just choosing one scripting language and writing everything in that.
You are going to have a LOT of additional complexity from supporting two scripting systems, if I were doing something like you describe I would Seriously Consider consolidating into a single language and just to Python or just do Perl. The time and energy it takes you to rewrite your Python scripts as Perl (or the other way around) may very well be easier compared to learning about creating installation guides and packaging your application using two different languages.
Related
I will be running a programming competition for high school students in the near future, and was originally going to use PC^2 (Programming Contest Control System) for the automated judging of the solutions. This software is commonly used in the ACM's International Collegiate Programming Contest regionals as well as the world finals. This is an excellent system which I have used before, but one of its pitfalls is its language support (Java, C, and C++). I'm a little bit concerned, as not all high school students who may be attending will have exposure to any of these languages. However, many local high schools teach introductory programming courses in Python. Is there an equivalent system to PC^2 which has Python support?
You can use some open source implementations of the online judges.
for example ONJ http://sourceforge.net/projects/onj/
It is easy to configure and it can be made compatible to run on languages of your choice. You dont need local copies of it on every system where the students will sit, you can just run it on a server and give the students its web-link to submit their solutions.
Do it with Sphere Online Judge, contest hosting is free of charge, many languages available.
I actually found a much better way to accomplish this. In their latest release, PC^2 added support for interpreted languages (such as Python, etc). Since PC^2 requires an executable, you need to "stub out" the compile command and simply invoke the interpreter with the script.
The full guide can be found at http://pc2.ecs.csus.edu/wiki/Interpreted_languages
Assuming you are still interested, PC^2 is very adaptable. I run a scripting contest using it and support Python, Perl, PHP and Ruby. Basically, PC^2 expects a language to compile and then run. You need to write a dinky script that pretends to compile the code and return success. Then the run script simply runs the scripting submission. Pretty easy to do.
You can visit http://acm2012.scusa.lsu.edu and look at the scripting contest info. Feel free to contact us for more info.
It's simpler than one may think. The following is for pc2 9.2.3-2565
Add language as follows (python here as an example):
Display Name: Python
Compile Cmd Line: touch OK
Executable Filename: OK
Program Execution Command Line: python {:mainfile}
python3.3 or python3.4 will work too.
pc2 could be easier, of course, but there does not seem to be much support left at CSUS. Reseting contest would be even greater feature; the current need to clone directories for test, practice, and actual contest is very awkward. Better management of the database (like the ability to remove things) would make it into a great tool. It is alright, but it could be great.
Is it possible to deploy python applications such that you don't release the source code and you don't have to be sure the customer has python installed?
I'm thinking maybe there is some installation process that can run a python app from just the .pyc files and a shared library containing the interpreter or something like that?
Basically I'm keen to get the development benefits of a language like Python - high productivity etc. but can't quite see how you could deploy it professionally to a customer where you don't know how there machine is set up and you definitely can't deliver the source.
How do professional software houses developing in python do it (or maybe the answer is that they don't) ?
You protect your source code legally, not technologically. Distributing py files really isn't a big deal. The only technological solution here is not to ship your program (which is really becoming more popular these days, as software is provided over the internet rather than fully installed locally more often.)
If you don't want the user to have to have Python installed but want to run Python programs, you'll have to bundle Python. Your resistance to doing so seems quite odd to me. Java programs have to either bundle or anticipate the JVM's presence. C programs have to either bundle or anticipate libc's presence (usually the latter), etc. There's nothing hacky about using what you need.
Professional Python desktop software bundles Python, either through something like py2exe/cx_Freeze/some in-house thing that does the same thing or through embedding Python (in which case Python comes along as a library rather than an executable). The former approach is usually a lot more powerful and robust.
Yes, it is possible to make installation packages. Look for py2exe, cx_freeze and others.
No, it is not possible to keep the source code completely safe. There are always ways to decompile.
Original source code can trivially be obtained from .pyc files if someone wants to do it. Code obfuscation would make it more difficult to do something with the code.
I am surprised no one mentioned this before now, but Cython seems like a viable solution to this problem. It will take your Python code and transpile it into CPython compatible C code. You also get a small speed boost (~25% last I checked) since it will be compiled to native machine code instead of just Python byte code. You still need to be sure the user has Python installed (either by making it a pre-requisite pushed off onto the user to deal with, or bundling it as part of the installer process). Also, you do need to have at least one small part of your application in pure Python: the hook into the main function.
So you would need something basic like this:
import cython_compiled_module
if __name__ == '__main__':
cython_compiled_module.main()
But this effectively leaks no implementation details. I think using Cython should meet the criteria in the question, but it also introduces the added complexity of compiling in C, which loses some of Python's easy cross-platform nature. Whether that is worth it or not is up to you.
As others stated, even the resulting compiled C code could be decompiled with a little effort, but it is likely much more close to the type of obfuscation you were initially hoping for.
Well, it depends what you want to do. If by "not releasing the source code" you mean "the customer should not be able to access the source code in any way", well, you're fighting a losing battle. Even programs written in C can be reverse engineered, after all. If you're afraid someone will steal from you, make them sign a contract and sue them if there's trouble.
But if you mean "the customer should not care about python files, and not be able to casually access them", you can use a solution like cx_Freeze to turn your Python application into an executable.
Build a web application in python. Then the world can use it via a browser with zero install.
Closed. This question is opinion-based. It is not currently accepting answers.
Want to improve this question? Update the question so it can be answered with facts and citations by editing this post.
Closed 9 years ago.
Improve this question
I need to write a script to do the following:
Monitor a queuing system, which is accessible by shell commands.
Create directories from templates using a mix of inline text editing, cp/mv, command line scripts, and compiled c++ programs.
Check for error conditions.
Write files on error conditions.
Note: 2D arrays would be mildly useful to my program, but I'm currently making due with several 1D arrays (due to the limitations of Bash script's arrays).
Those tasks all seems somewhat "shell heavy" in so much as it can easily be implemented with a bunch of shell commands, so I thought Bash scripting would be a natural way to go. My results thus far have been good, but before I begin refactoring and finalizing my code, I was wondering whether it'd be better to port it to Python. I've read in numerous places that Python is "superior" to bash. I've done a bit of Python scripting and as far as I can tell, that's because it has more built in libraries and has support for object-oriented programming. However, all the scripts I've seen using shell commands, such as this one:
http://magazine.redhat.com/2008/02/07/python-for-bash-scripters-a-well-kept-secret/
Implement obnoxious syntax, like having to define commands as variables, like so:
#You could add another bash command here
#HOLDING_SPOT="""fake_command"""
#Determines Home Directory Usage in Gigs
HOMEDIR_USAGE = """
du -sh $HOME | cut -f1
"""
#Determines IP Address
IPADDR = """
/sbin/ifconfig -a | awk '/(cast)/ { print $2 }' | cut -d':' -f2 | head -1
"""
...and requiring wrapper functions and other fun.
Am I just being silly or does that seem less intuitive? Is there a speed advantage for use with Python that would outweigh the simplicity advantage of Bash when it comes to shell script commands? Or is the syntax of bash (no 2D arrays, brace/parentheses intricacies) reason to jump to Python?
If you can't find a reason to switch it's probably because there's no reason.
I started to switch python because some of my scripts require some templating process that was easier to do in other than shell script.
There were also another scripts which required configuration file parsing or parameter parsing that make me learn a bit more of python, and finally another ones which deal with trac (coded in python) who made me switch to python.
But if you could outline your scripts in bash fast and cleanly without require another tool, remain in bash, it is a great tool.
Bash is good right at the actual interface with the system, because it's so easy to run external programs, and for processing data that comes in streams. Python is good at work with less surface area, involving more looping, logic, data structures, and so on. Horses for courses.
It's hard to advise you which to use, or how to use both, without knowing details of your problem. But from the sound of it, this is something i'd do entirely in shell script. I'd find a way of doing it without arrays, though; the key is to stop thinking in C++ and start thinking in bash.
It really depends on what you are trying to do, but here are some general differences.
Shells, including bash, want to treat all variables as strings and to access variable values through simple textual substitution. Sometimes this is convenient; sometimes not so much.
In shell scripts you tend to manipulate data by piping it through various utilities. Sometimes this is convenient; sometimes not so much.
Python has very good string manipulation, so if you are doing a lot of text file munging it can be faster (both to write and to run) to do it in Python rather than figuring out what sed or awk incantation you want.
That's another thing -- many of the tools you will use in shell scripts have so many options that they are essentially mini-languages of their own, some of which are terse to the point of being obtuse. Python can be much more readable than shell scripts that invoke a lot of these mini-languages since it is consistent syntactically.
Some projects lend themselves to an object-oriented approach, which Python supports handily, while bash... not so much. Python also supports some functional techniques, which are handy for some projects, while bash... you get the picture.
In short, a shell's scripting language is intended to be glue for tying together various single-function utilities, with some conveniences to make that easier, while Python is a robust programming language in which you can build complete GUI or Web applications if you like.
I personally also find the overall syntax of Python much nicer than a shell scripting language; my programs tend to do what I want without a lot of fiddling, and when they don't it's usually because of a logic error rather than an error translating what I want into the language. To me, Python is nearly unique in this regard, although I hear Ruby programmers say similar things about their favorite language.
If you are familiar with and like shell scripting, Perl might be an easier step up since it was originally intended to replace shell scripts (and sed/awk).
As it currently stands, this question is not a good fit for our Q&A format. We expect answers to be supported by facts, references, or expertise, but this question will likely solicit debate, arguments, polling, or extended discussion. If you feel that this question can be improved and possibly reopened, visit the help center for guidance.
Closed 10 years ago.
What are the advantages of Python, PowerShell, and other scripting environments? We would like to standardize our scripting and are currently using bat and cmd files as the standard. I think Python would be a better option than these, but am also researching PowerShell and other scripting tools.
The scripts would be used to trigger processes such as wget etc to call web services, or other applications/tools that need to run in a specific order with specific parameters.
We primarily work with the Windows stack, but there is a good chance we will need to support Unix in the future.
Python works as a great, all-purpose tool if you're looking to replace CMD and BAT scripts on your Windows boxes, and can also be written to run scripts on your (L)inux boxes, too. It's a great, flexible language and can handle many tasks you throw at it.
That being said, PowerShell is an amazingly versatile tool for administering all manner of Windows boxes; it has all the power of .NET, with many more interfaces into MS products such as Exchange and Active Directory, which are a timesaver. Depending on your situation, you may get more use of of PS than other scripting languages just because of the interfaces available to MS products, and I know MS seems to have made a commitment to providing those APIs in a lot of products. PowerShell comes installed on all current versions of Windows (Windows 7+, Windows Server 2008+), and is fairly easily installed on older versions.
To address your edit that your scripts will be used to launch other processes, I think in that case either of the tools fit the bill. I would recommend PS if you plan on adding any admin-ish tasks to the scripts rather than just service calls, but if you stick to what you described, Python is good.
We would like to standardize our scripting and are currently using bat and cmd files as the standard.
It sounds like Windows is your predominate environment.
If so, PowerShell would be much better than Python.
PowerShell is included with Windows
Server 2008. No need to
deploy/install Python runtime on
every new server that rolls in.
The entire Microsoft server related software (Exchange, Systems Center, etc) is transitioning to PowerShell cmdlets for functionality and extensions
3rd party vendors (e.g. SCOM plugins) will also use PowerShell scripts/cmdlets to expose functionality
I have more experience with Python than PowerShell but the writing is on the wall as far as the Microsoft ecosystem is concerned: go with PowerShell. Otherwise, you'll just be going against the grain and constantly interop-ing between Python and everyone else's cmdlets.
Just because you can code import win32com.client in Python does not put it on equal footing with PowerShell in the Windows environment.
IronPython has access to all of the same .NET assemblies as any other .NET language for writing system dependent scripts on Windows. But the same knowledge of Python can be used to write similar scripts on Linux, Solaris, BSD, or OS/X. If you use the standard C Python on Windows, then you can access any COM objects and it is straightforward to translate VBA examples into Python code. The SPAMBayes Outlook plugin is a good example of how far you can go with that. http://spambayes.sourceforge.net/
Python's best feature is the "batteries included" standard library, and even though this is not distributed with IronPython, much of it will work if you just point IronPython to the installed library folder from CPython. In fact, most pure Python libraries, i.e. no compiled C or C++ modules, will work fine with IronPython. On Windows, you also have the choice of installing Python through Cygwin.com which then allows you to use a lot of modules that are normally considered UNIX-only. This can be of use if you have to maintain cross-platform scripts and you prefer consistency rather than special case coding for each OS.
And if you should need to leverage some Java classes, then Jython allows you to use the same Python language that you know to leverage this. Combine this with a nice message queuing system like RabbitMQ, and you can have Python, Jython and IronPython scripts on multiple machines all cooperating in getting the job done.
There is also a huge selection of 3rd party Python modules out there and you could spend several months trawling through delicious.com before you run out of new discoveries. This means that when you need something not part of standard Python libraries, a bit of Googling often comes up with a solution.
Some useful Python modules for scripting to replace bash, CMD and BAT files are PEXPECT http://www.noah.org/wiki/Pexpect and Python WMI http://timgolden.me.uk/python/wmi/index.html
But, in the end, Python also works just fine for simple straightforward scripts that don't need any special features... yet!
If all you do is spawning a lot of system specific programs with no or little programming logic behind then OS specific shell might be a better choice than a full general purpose programming language.
One advantage to Python is the availability of third-party libraries and an extensive built-in standard library. You can do a lot of powerful operations quickly and easily with Python on a variety of operating systems and environments. That's one reason we use Python here at the office not only as a scripting language but for all of our database backend applications as well.
We also use it for XML and HTML scraping, using ElementTree and BeautifulSoup, which are very powerful and flexible Python-specific libraries for this sort of work.
I find it sad no one yet mentioend good ol' Perl.
The questions is kind of vague, but Python is much more portable than PowerShell; however, Python isn't that prevalent on Windows. But on the other hand, I don't believe PowerShell scripts will work on a Windows machine that doesn't have PowerShell. Meaning they may not work in the old fashioned cmd shell. I think you'll find more documentation and libraries for Python as well.
PowerShell is more like Bash than it is a programming language like Python.
Maybe you could explain what you want to do with your scripts and you'll probably get better answers.
If you are working with web based scripting, then ActiveState's ActivePython seems to have a lot of support for Windows specific API's that would suit you and it has tons of great portable libraries for doing web based work.
Closed. This question is opinion-based. It is not currently accepting answers.
Want to improve this question? Update the question so it can be answered with facts and citations by editing this post.
Closed 4 years ago.
Improve this question
Currently I'm using a Windows batch file to build my software. It does things like running MSBuild, copying files, creating a ZIP file, running some tests, including the subversion revision number, etc.
But the problem is, batch files are evil. So I would like to change to something better. I was planning to recreate my build script in Python. Is that a smart choice? What about all those build systems, like Ant, SCons, Maven, Rake, etc. Would using any of those be a better choice?
Note: I'm not planning to replace my Visual Studio solution/project files. I only want to script everything else that's required to create a release of my software.
Edit: I have good reasons to move away from batch, that's not what my question is about. I would like to know (for example) what SCons gives me, over a normal Python script.
For a tool that is scripted with Python, I happen to think Paver is a more easily-managed and more flexible build automator than SCons. Unlike SCons, Paver is designed for the plethora of not-compiling-programs tasks that go along with managing and distributing a software project.
Batch files aren't evil - they've actually come quite a long way from the brain-dead days of command.com. The command language can be pretty expressive nowadays, it just requires a bit of effort on your part to learn it.
Unless there's an actual problem with your build script that you can't fix (and, if that's the case, that's the question you should be asking rather than some wishy-washy "What's the best replacement?" :-), my approach would be to stick with what you've got.
A vague feeling of evilness would not be reason for me to waste effort 'fixing' something that isn't broken. And it would be wasted effort unless there's a clear advantage to changing ("less evil" is not something I'd consider a clear advantage).
As you're mentioning Python and SCons, I'd say go for SCons. It is Python after all. And yes, any of the above would be a better choice than hand-rolled build scripts.
I've seen python scripts used for building releases elsewhere so it can't be bad. Actually, I've personally used perl scripts to automate release building. I guess any scripting language could easily automate that procedure. If it's gonna be easy to do (and probably better than batch scripts), why not try it?
I would suggest using NAnt for your build script instead of python.
My reasons for this are:
It has the tasks defined already, all you need to do is write the XML and point it to the right places. If you are working with people who do not know python, XML may be a little less scary than learning a new language.
NAnt is designed to work in the windows .Net environment, so it can already do MSBuild and NUnit tasks.
If you are already writing in C#, if you need to extend NAnt to do new tasks you are not adding another language to the mix of your project.
You can hook into Cruise Control .Net (for continuous builds). Which I think is the main reason why you would use NAnt.
Why should you use python? If your build script isn't broke don't fix it. If your having issues updating it to deal with new aditions to the project then you may want to look at rewriting it. I wouldn't use Python though tools like NANT or MSBuild do the job. I don't see the point in using a general purpis programming language to do something that tools have already been written to do unless you have a lot of obscure requirements existing tools can't deal with. Second what happens if you get hit by a bus or win the lotto? If you are determined to script everything I'd use powershell or some other Microsoft specific technology since your already wedded to Microsoft. If you leave will there be enough Python programmers to maintain the build scripts?
I would strongly suggest to take a look at waf. It's kind of what you want: "a Python-based framework for configuring, compiling and installing applications"
Personally I would use scripting as a last resort given that
With a bit of work you can get MSBuild to do all those things for you by extending it with additional components
There are third party equivalents to MSBuild like NANT that can do the same thing
There are entire tools like FinalBuilder that also do the same thing, and are easier to configure and extend
However, if I had to go the scripting route I would use Powershell for a couple of reasons:
Complete access to file system
You can easily access .NET objects
You can easily access COM objects
You can create custom makefiles for Microsoft nmake tool which you already have installed. Using a tool like that (SCons, Maven, etc. fall into the same category) gives you much more than regular scripts.
The main benefit is that dependencies between files are tracked and also the timestamps of changes. For example, you can make your .zip file depend on some other files, so .zip only gets repacked if some of those files have changed in the meantime. Just like with source code and its compiled form.
Python is very portable. SCons is field tested and reliable. Given what you know (from what you explained), why even ask the question?
If your maintaining something, its not just about getting it to build, its also about explaining to the user why it can NOT build, which saves you a ton of very frustrating questions while helping users to help themselves.
I can not think of a modern, production operating system that lacks Python, unless you get into the embedded / research arena.
So, I'm answering to say, you answered your own question :)
It depends on what technology your software uses. If you're building C++ programs, I'd probably say go for scons without question (unless you have weird requirements scons can't meet). On the other hand, consider the instructions for building C#: CSharpBuilder.
I would like to know (for example) what SCons gives me, over a normal Python script.
Think of scons as being more of a library than a program. It provides you with code that will prevent a lot of tedium that you will have to deal with without it. In my opinion, vanilla Python isn't the best option for any kind of shell scripting stuff (not that it can't do it).
But the problem is, batch files are evil.
Lastly, batch files are evil if they're used for a project they're not suited to handle. For the one or two file project, batch files do just fine.
It does things like running MSBuild, copying files, creating a ZIP file, running some tests, including the subversion revision number, etc.
MSBuild and PowerShell can easily do all of this with reasonably clean succinct code. You're then sticking to purely M$ products which managers tend to like. Otherwise I would suggest you could look into Rake if not only for its large community. It has a nice syntax and iron ruby support (irake).
To be honest all but the last task you have mentioned are easily done in MSBuild alone. I would suggest learning the tools you have before going elsewhere.
Check out http://msbuildtasks.tigris.org/ for some good add ons to MSBuild