We Keep Coding

creating a soft ware with UI using python scripts [closed]

Closed. This question needs to be more focused. It is not currently accepting answers.
Want to improve this question? Update the question so it focuses on one problem only by editing this post.
Closed 7 years ago.
Improve this question
I am planning on creating a software that will be used to analyse biological data, some of us biologists would not have much programming skills yet we do alot of repetitive tasks and are required to write more scripts or programs that we always use using.
i have decided to narrow it down to only one very important task, its a searching script, i.e to query a database of genes-(data), there is already a program that does that called ncbi-blast, usually one needs to be familiar with command line and also requires ubuntu or windows, but its a hassle if you have to analyse the huge quantities of results generated.
so i have want to package some of my python scripts into a software so that others can find it convenient to analyse their work
Open vs. Closed Source (Licensing)
i intend to make some of the functionalities freely available to the user in the software, although at a later stage, i intend to incorpaorate pipelines that would require an affordable license
testing
so with this one function, it makes testing alot simpler-i dont know yet, and i would not mind starting with linux-ubuntu and windows as the platforms.
I appreciate your advice on choosing just one language, and i will go with python, how ever i would like tyhe software to atleast support other free programs created in other languages like R, as in plugins that a client installs when they need them.
I hope this shades more light to my already complex situation
Thank you

Sounds like you're at the beginning of an adventure and some new learning curves. I will applaud your willingness to create. Here are some things to consider as you get started on your journey.
fyi: "software" is typically a single word, no space.
Based on the context of "a soft ware" it seems like you mean "application".
I mention "software" and "application" not to give you a hard time about it, but because I think 1) it will help you in future searches when you look for ways to get something done (e.g. you'll get more useful hits on "python application installer" than "python soft ware installer", and 2) you can more clearly explain to people what you are attempting.
So. Your application(s) will need a user interface for your collection of scripts. Do you want a graphical user interface (GUI)? Or a command line interface (CLI)?
GUI: more complex than CLI:
For a GUI it will be more complex to get something that runs on both Windows & Linux.
So far as I know there isn't a trivial way to create a single GUI that runs on both Windows & Linux without adding another tool. Maybe python has easy GUI user-interface stuff in it and you could use that.
For perl I will point you here: "Perl GUI programming on Windows" Perl GUI programming on Windows
This lists some Perl approaches to consider, you will have to research them and see which ones are also able to run on Linux. Be careful about including additional libraries you'll need to package (or document how to install) with you application.
CLI: more simple than a GUI:
Since they are just scripts today, I would suggest starting with a CLI which probably means cleaning up your argument handling ( #ARGV in Perl, I can't comment on Python).
Read through your code and find literals that you change when you run a script for something new, those things will become your arguments. And if it starts to seem like you are creating too many arguments maybe you want to look into a configuration file (properties file of some kind).
It will be easier for you to get started with a CLI and you can always come back and add a GUI option in a future release. I find that designing a well thought out CLI makes it easier to focus on what is important for a GUI so it should help your eventual GUI be even better.
Packaging
Packaging your application is going to be a challenge.
Do your scripts need any libraries that aren't part of default with python & perl installs? If so you need to work out a way to supply those (e.g. include them with your distribution or include documentation that your users can follow to download and install the libs).
I can't comment on pyinstaller.
For Perl I will point you to this question on distributing a Perl Application : Distributing a Perl Application
User Skill
How much skill do you expect your users to have?
Will they be ok with installing a python and perl interpreters if necessary? Not every user has that technical skill.
Do you want to make sure your scripts verify the minimum versions of python & perl they need?
Documentation
See User Skill, above. You will need to make documentation available.
This will be driven by who is going to use your scripts. What is your target user like? Can you write a "5 line" summary that would make me want to download them and try them? That will be a helpful exercise to help you focus on how you want to present the scripts.
Open vs. Closed Source (Licensing)
This also ties into licensing. Are you aiming for a commercial product? Giving the world another gift of open source? Those are both fine things to do, but you'll want to choose a suitable license for your application. You also have some work to do if you want to avoid distributing your script's source code (I'm not suggesting you pursue one approach or the other, just that you make the decision before you put it on the internet at large).
Testing
Testing your application is going to be a challenge.
If you don't have automated tests for your scripts you will go crazy trying to verify it runs everywhere you want it to run.
Do you have unit tests & test suites for your scripts so you can verify they run correctly on different versions of Windows and various Linux distributions?
If your scripts are pretty simple maybe it doesn't matter if they're running on 32 bit vs. 64 bit operating systems.
Maybe it doesn't matter if they run on Windows 10, Windows 8.x, Windows 7.x, maybe various versions of Windows Server (2012, 2008, ...).
Maybe it doesn't matter if they run on RedHat, Suse, Ubuntu, Mint, ...
You probably want some sanity checks to verify that your install program worked correctly and the environment is suitable.
Without knowing what your scripts do it is kind of hard to say how much testing they benefit from.
Free Advice: choose just one scripting language and run with that
I will end with some completely free advice (worth what you're paying for it :-) )
Think really hard about just choosing one scripting language and writing everything in that.
You are going to have a LOT of additional complexity from supporting two scripting systems, if I were doing something like you describe I would Seriously Consider consolidating into a single language and just to Python or just do Perl. The time and energy it takes you to rewrite your Python scripts as Perl (or the other way around) may very well be easier compared to learning about creating installation guides and packaging your application using two different languages.

deploying python applications

Is it possible to deploy python applications such that you don't release the source code and you don't have to be sure the customer has python installed?
I'm thinking maybe there is some installation process that can run a python app from just the .pyc files and a shared library containing the interpreter or something like that?
Basically I'm keen to get the development benefits of a language like Python - high productivity etc. but can't quite see how you could deploy it professionally to a customer where you don't know how there machine is set up and you definitely can't deliver the source.
How do professional software houses developing in python do it (or maybe the answer is that they don't) ?

You protect your source code legally, not technologically. Distributing py files really isn't a big deal. The only technological solution here is not to ship your program (which is really becoming more popular these days, as software is provided over the internet rather than fully installed locally more often.)
If you don't want the user to have to have Python installed but want to run Python programs, you'll have to bundle Python. Your resistance to doing so seems quite odd to me. Java programs have to either bundle or anticipate the JVM's presence. C programs have to either bundle or anticipate libc's presence (usually the latter), etc. There's nothing hacky about using what you need.
Professional Python desktop software bundles Python, either through something like py2exe/cx_Freeze/some in-house thing that does the same thing or through embedding Python (in which case Python comes along as a library rather than an executable). The former approach is usually a lot more powerful and robust.

Yes, it is possible to make installation packages. Look for py2exe, cx_freeze and others.
No, it is not possible to keep the source code completely safe. There are always ways to decompile.
Original source code can trivially be obtained from .pyc files if someone wants to do it. Code obfuscation would make it more difficult to do something with the code.

I am surprised no one mentioned this before now, but Cython seems like a viable solution to this problem. It will take your Python code and transpile it into CPython compatible C code. You also get a small speed boost (~25% last I checked) since it will be compiled to native machine code instead of just Python byte code. You still need to be sure the user has Python installed (either by making it a pre-requisite pushed off onto the user to deal with, or bundling it as part of the installer process). Also, you do need to have at least one small part of your application in pure Python: the hook into the main function.
So you would need something basic like this:
import cython_compiled_module
if __name__ == '__main__':
cython_compiled_module.main()
But this effectively leaks no implementation details. I think using Cython should meet the criteria in the question, but it also introduces the added complexity of compiling in C, which loses some of Python's easy cross-platform nature. Whether that is worth it or not is up to you.
As others stated, even the resulting compiled C code could be decompiled with a little effort, but it is likely much more close to the type of obfuscation you were initially hoping for.

Well, it depends what you want to do. If by "not releasing the source code" you mean "the customer should not be able to access the source code in any way", well, you're fighting a losing battle. Even programs written in C can be reverse engineered, after all. If you're afraid someone will steal from you, make them sign a contract and sue them if there's trouble.
But if you mean "the customer should not care about python files, and not be able to casually access them", you can use a solution like cx_Freeze to turn your Python application into an executable.

Build a web application in python. Then the world can use it via a browser with zero install.

Real-world Jython applications

I recently started learning Python. Not yet ventured into coding.
During one of my learning sessions, i came accross the term Jython.
I googled it & got some information.
I would like to know if anyone has implemented any real-world program using Jython.

Most of the time, Jython isn't used directly to write full read-world programs, but a lot of programs actually embed Jython to use it as a scripting language.
The official Jython website gives a list of projects, some written in Jython, others using Jython for scripting:
http://wiki.python.org/jython/JythonUsers

I am writing a full application in Jython at the moment, and would highly recommend it. Having all of the Java libraries at your disposal is very handy, and the Python syntax and language features actually make using some of them easier than it is in Java (I'm mostly talking about Swing here).
Check out the chapter on GUI Applications from the Jython book. It does a lot of comparisons like 'Look at all this Java code, and now look at it reduced to Python code of half the length!'.
The only caveats I've found are:
Jython development tends to run slightly behind Python, which can be annoying if you find a cool way of doing something in Python, only to discover it's not supported in the current Jython version.
Occasionally you might have hiccups with the interface between Python and Java (I have a couple of unsolved problems here and here, although there are always workarounds for this kind of thing).
Distribution is not as simple as it could be, although once you figure out how to do it, it's fairly painless. I recommend following the method here. It essentially consists of:
Exploding jython.jar and adding your own modules into it.
Writing and compiling a small Java class that creates a Python interpreter and loads up your Python modules.
Creating an executable .jar file consisting of the jython.jar modules, your own Python modules, and the Java class.

Jython really shines for dependency injection.
You know those pesky variables you have to give your program, like
file system paths
server names
ports
Jython provides a really nice way of injecting those variables by putting them in a script. It works equally well for injecting java dependencies, as well.

WebSphere and WebLogic use it as their default scripting engine for administrative purposes.
A lot of other Oracle products ship it as part of their "oracle_commons" module (Oracle Universal Installer, Oracle HTTP Server etc). It's mostly version 2.2 being deployed though, which is a bit old and clunky.

There is a list of application that uses jython at http://wiki.python.org/jython/JythonUsers

What is the importance of an IDE when programming in Python?

I'm a beginning Python programmer, just getting my feet wet in the language and its tools and native practices. In the past, I've used languages that were tightly integrated into IDEs, and indeed I had never before considered that it was even possible to program outside of such a tool.
However, much of the documentation and tutorials for Python eschew any sort of IDE, relying instead on powerful editors and interactive interpreters for writing and teaching the language.
How important is an IDE to normal Python development?
Are there good IDEs available for the language?
If you do use an IDE for Python, how do you use it effectively?

IDEs arent very useful in Python; powerful editors such as Emacs and Vim seem very popular among Python programmers.
This may confuse e.g. Java programmers, because in Java each file generally requires boilerplate code, such as a package statement, getters and setters.
Python is much more lightweight in comparison.
If you're looking for an equivalent to Visual Studio or Eclipse, there is... Eclipse, with Pydev.
Emacs and Vim are very powerful and general, but have a steep learning curve.
If you want to use Emacs, I highly recommend python mode; it's much better than the default Python mode.

A matter of habit and personal preferences. Me, I use vim (I have to admit emacs is at least as powerful, but my fingers are deeply trained by over 30 years of vi, and any other editor gives me the jitters, especially when it tries to imitate vi and never really manages to get it 100% right;-), occasionally an interactive environment (python itself, sometimes ipython), and on even rarer occasions a debugger (pdb). A good editor gives me all I need in term of word completion, lookup, &c.
I've tried Eclipse, its plugins, eric, and Kommodo, but I just don't like them -- Wing, I think I could get used to, and I have to admit its debugger is absolutely out of this world... but, I very rarely use (or need!) advanced debugging functionality, so after every rare occasion I'd forget, and have to learn it all over again a few months later when the need arose again... nah!-)

How important is an IDE to normal Python development?
Not very, IMHO. It's a lightweight language with much less boilerplate and simpler idioms than in some other languages, so there's less need for an IDE for that part.
The standard interactive interpreter provides help and introspection functionality and a reasonable debugger (pdb). When I want a graphical look at my class hierarchies, I use epydoc to generate it.
The only IDE-like functionality I sometimes wish I had is something that would help automate refactoring.
Are there good IDEs available for the language?
So I hear. Some of my coworkers use Wing.
If you do use an IDE for Python, how do you use it effectively?
N/A. I tried using Wing a few times but found that it interfered with my normal development process rather than supporting it.

The IDE you use is a personal and subjective thing, but it definitely matters. Personally, for writing short scripts or working with python interactively, I use PyDee available at http://pydee.googlecode.com/ . It is well done, fairly lightweight, but with good introspection capabilities.
For larger projects involving multiple components, I prefer Eclipse with appropriate plugins. It has very sophisticated management and introspection capabilities. You can download it separately or get it as part of Python (X,Y) at http://www.pythonxy.com/ .

In contrast to the other answers i think that IDE's are very important especially for script languages. Almost all code is bad documentated and an IDE with a good debugger gives you much insides about what is really going on what datatypes are assigned to this values. Is this a hash of lists of hashes or a list of hashs of hashs.
And the easy documentation lookup will save you time.
But this is only important for people who need to count there time, this normally excludes beginners or hobbyists.

(1) IDEs are less important than for other languages, but if you find one that is useful, it still makes things easier. Without IDEs -- what are doing? Always running Python from command line?
(2-3) On my Mac there's included IDLE which I keep always open for its Python shell (it's colored unlike the one in Terminal) and I use free Komodo Edit which I consider to be well-suited for Python as it doesn't go into the language deeply but rather focuses on coloring, tab management, parsing Python output, running frequent commands etc.

Would Python make a good substitute for the Windows command-line/batch scripts?

I've got some experience with Bash, which I don't mind, but now that I'm doing a lot of Windows development I'm needing to do basic stuff/write basic scripts using
the Windows command-line language. For some reason said language really irritates me, so I was considering learning Python and using that instead.
Is Python suitable for such things? Moving files around, creating scripts to do things like unzipping a backup and restoring a SQL database, etc.

Python is well suited for these tasks, and I would guess much easier to develop in and debug than Windows batch files.
The question is, I think, how easy and painless it is to ensure that all the computers that you have to run these scripts on, have Python installed.

Summary
Windows: no need to think, use Python.
Unix: quick or run-it-once scripts are for Bash, serious and/or long life time scripts are for Python.
The big talk
In a Windows environment, Python is definitely the best choice since cmd is crappy and PowerShell has not really settled yet. What's more Python can run on several platform so it's a better investment. Finally, Python has a huge set of library so you will almost never hit the "god-I-can't-do-that" wall. This is not true for cmd and PowerShell.
In a Linux environment, this is a bit different. A lot of one liners are shorter, faster, more efficient and often more readable in pure Bash. But if you know your quick and dirty script is going to stay around for a while or will need to be improved, go for Python since it's far easier to maintain and extend and you will be able to do most of the task you can do with GNU tools with the standard library. And if you can't, you can still call the command-line from a Python script.
And of course you can call Python from the shell using -c option:
python -c "for line in open('/etc/fstab') : print line"
Some more literature about Python used for system administration tasks:
The IBM lab point of view.
A nice example to compare bash and python to script report.
The basics.
The must-have book.

Sure, python is a pretty good choice for those tasks (I'm sure many will recommend PowerShell instead).
Here is a fine introduction from that point of view:
http://www.redhatmagazine.com/2008/02/07/python-for-bash-scripters-a-well-kept-secret/
EDIT: About gnud's concern: http://www.portablepython.com/

Are you aware of PowerShell?

Anything is a good replacement for the Batch file system in windows. Perl, Python, Powershell are all good choices.

#BKB definitely has a valid concern. Here's a couple links you'll want to check if you run into any issues that can't be solved with the standard library:
Pywin32 is a package for working with low-level win32 APIs (advanced file system modifications, COM interfaces, etc.)
Tim Golden's Python page: he maintains a WMI wrapper package that builds off of Pywin32, but be sure to also check out his "Win32 How Do I" page for details on how to accomplish typical Windows tasks in Python.

Python is certainly well suited to that. If you're going down that road, you might also want to investigate SCons which is a build system itself built with Python. The cool thing is the build scripts are actually full-blown Python scripts themselves, so you can do anything in the build script that you could otherwise do in Python. It makes make look pretty anemic in comparison.
Upon rereading your question, I should note that SCons is more suited to building software projects than to writing system maintenance scripts. But I wouldn't hesitate to recommend Python to you in any case.

As a follow up, after some experimentation the thing I've found Python most useful for is any situation involving text manipulation (yourStringHere.replace(), regexes for more complex stuff) or testing some basic concept really quickly, which it is excellent for.
For stuff like SQL DB restore scripts I find I still usually just resort to batch files, as it's usually either something short enough that it actually takes more Python code to make the appropriate system calls or I can reuse snippets of code from other people reducing the writing time to just enough to tweak existing code to fit my needs.
As an addendum I would highly recommend IPython as a great interactive shell complete with tab completion and easy docstring access.

I've done a decent amount of scripting in both Linux/Unix and Windows environments, in Python, Perl, batch files, Bash, etc. My advice is that if it's possible, install Cygwin and use Bash (it sounds from your description like installing a scripting language or env isn't a problem?). You'll be more comfortable with that since the transition is minimal.
If that's not an option, then here's my take. Batch files are very kludgy and limited, but make a lot of sense for simple tasks like 'copy some files' or 'restart this service'. Python will be cleaner, easier to maintain, and much more powerful. However, the downside is that either you end up calling external applications from Python with subprocess, popen or similar. Otherwise, you end up writing a bunch more code to do things that are comparatively simple in batch files, like copying a folder full of files. A lot of this depends on what your scripts are doing. Text/string processing is going to be much cleaner in Python, for example.
Lastly, it's probably not an attractive alternative, but you might also consider VBScript as an alternative. I don't enjoy working with it as a language personally, but if portability is any kind of concern then it wins out by virtue of being available out of the box in any copy of Windows. Because of this I've found myself writing scripts that were unwieldy as batch files in VBScript instead, since I can't usually depend on Python or Perl or Bash being available on Windows.

Python, along with Pywin32, would be fine for Windows automation. However, VBScript or JScript used with the Windows Scripting Host works just as well, and requires nothing additional to install.

I've been using a lot of Windows Script Files lately. More powerful than batch scripts, and since it uses Windows scripting, there's nothing to install.

As much as I love python, I don't think it a good choice to replace basic windows batch scripts.
I can't see see someone having to import modules like sys, os or getopt to do basic things you can do with shell like call a program, check environment variable or an argument.
Also, in my experience, goto is much easier to understand to most sysadmins than a function call.