Quite-probably a silly question, as I don't know much about Java/Jython/JRuby/bytecode, but..
I stumbled across _why's unholy again today.. It allows you to output Python bytecode from Ruby code.. Basically allowing them to produce the same bytecode..
Jython outputs Java bytecode, as does JRuby.. Since these both compile to the same bytecode, does this mean you could potentially use any Python library from Ruby, and Ruby libraries from Python?
No, that won't work. At least not the way you think it would.
Interoperability between Jython and JRuby works the same way as between CPython and YARV: they both run on the same platform, so they can communicate with each other using that platform.
In the case of CPython and YARV, that platform is C/POSIX, so they can communicate with each other using C structs, ints, char*s and C function calls. In the case of Jython and JRuby, that platform is the JVM, so they can communicate with each other using JVM objects, JVM classes, JVM interfaces, JVM types and JVM methods.
In both cases, those platform primitives look nothing like Python or Ruby objects.
To JRuby, Jython is just yet another Java program. To Jython, JRuby is just another Java program.
For example: in Ruby, you can add, remove and redefine methods dynamically at any moment. On the JVM, the smallest unit of code that can be dynamically added and removed is a class. So, a Ruby method is actually not represented as a Java method. It is represented as a Java class. And logically, a Ruby object with a couple of methods is represented as a Java object with no methods, just a Dictionary<String, RubyMethod> field. IOW: it's totally unusable from Java, and, since from JRuby's point of view Jython is just Java, it's also unusable from Jython.
Now, there are ways to make this a little bit better. You could use actual Java types to communicate between the two – both implementations have great interoperability with Java. So, instead of passing a Ruby hash to Python or a Python dictionary to Ruby, you would use a Java Map from both Ruby and Python. But note that this requires that both your Ruby and Python code are specifically written to work on the JVM. IOW: you cannot just use any Python or Ruby library you find on the web, which is what you are asking about.
Another possibility is the one mentioned by #duncan in his answer: embed Jython or JRuby as a scripting engine into your Ruby or Python application. But again, this doesn't really answer your question about using arbitrary Python libraries from Ruby or vice versa.
So, what is the problem here?
The problem is that in order for the two runtimes to communicate, they need to speak the same "language". And in this particular case, the only language that the two runtimes have in common, is Java, or rather a severely crippled subset of Java.
So, we need to find a common language. One way to define such a language would be for both runtimes to understand each other's Meta-Object Protocol (MOP).
A MOP is basically an object model for the language's object model. Um, that's confusing because we use the word "object model" to mean two different things. Let me rephrase that:
A MOP is basically a domain model for the language's object system. Just like a domain model for a banking system contains objects that represent real-world customers, accounts, balances, ledgers and so on, and methods that represent real-world actions like money transfers, withdrawals and so on, a MOP contains objects that represent language classes, methods, variables, objects and methods that represent language actions like looking up a variable, calling a method, inheriting from a class, constructing an instance of a class.
Normally, every runtime keeps its MOP private, and every runtime has its own MOP.
If JRuby and Jython exposed their MOPs to each other and understood each other's MOPs (or, even better yet: they exposed their MOPs to the JVM and both used the same MOP), then you could pass one of those crazy JRuby method bags to Jython, and it would know how to find the methods that belong to that object and how to call them, because it can just ask JRuby's MOP how to do it.
There is actually a project to create just such a MOP for the JVM: the dynalang MOP is a project for a shared, standardized MOP for dynamic languages running on the JVM. It was created by Attila Szegedi, the maintainer of the Mozilla Rhino ECMAScript engine. At the moment, none of the big language implementations uses it, but there is collaboration going on between at least Rhino, JRuby, Jython and Groovy to make sure that dynalang is generic enough that it can support all of the different language's object models.
If you want a sneak peek at what a world with such a shared MOP would look like, you can take a look at Microsoft's Dynamic Language Runtime (DLR). The DLR contains just such a MOP and all runtimes which support the DLR (which, in addition to the usual suspects such as IronRuby, IronPython, IronJS and IronScheme now also includes C# 4 and Visual Basic.NET 10) can almost seamlessly interoperate with each other.
Another similar platform is the Parrot Virtual Machine, which was specifically designed to allow multiple dynamic languages to interoperate on the same runtime platform. There are implementations of Python (Pynie) and Ruby (Cardinal) available, but especially Cardinal is still very far from being even a remotely complete Ruby implementation.
There are two ways to do it. Both offer the ability to statically compile code and produce a real Java class from the script. Jython AFAIK in this case generates Java source code and then calls javac, via an jythonc script. But this requires compilation.
For both interpreters, you can call Java code from scripts, and you can embed the interpreter in a Java application.
For example, to call Java from Python:
>>> from java.util import Random
>>> r = Random()
>>> r.nextInt()
501203849
To embed JRuby interpreter in Java, you can do (note, there is a JSR223 based way too, this is the core one):
package vanilla;
import org.jruby.embed.ScriptingContainer;
public class HelloWorld {
private HelloWorld() {
ScriptingContainer container = new ScriptingContainer();
container.runScriptlet("puts Hello world");
}
public static void main(String[] args) {
new HelloWorld();
}
You could do the same from Jyton (I guess you would need to give the jruby paths correctly):
import org.jruby.embed.ScriptingContainer
container = ScriptingContainer()
container.runScriptlet("puts Hello world")
The same can be done the other way around.
You won't get the whole ruby stdlib exported to the python interpreter by doing an import. You would need to precompile ruby's stdlib to bytecode in advance.
However with the technique described above, and adding a couple of helper scripts and defined interfaces, you can bridge specific functionality from one language to the other.
Related
We have a sizable code base in Perl. For the forseeable future, our codebase will remain in Perl. However, we're looking into adding a GUI-based dashboard utility. We are considering writing the dashboard in Python (using tkinter or wx). The problem, however, is that we would like to leverage our existing Perl codebase in the Python GUI.
So... any suggestions on how achieve this? We are considering a few options:
Write executables (in Perl) that mimic function calls; invoke those Perl executables in python as system calls.
Write Perl executables on-the-fly inside the Python dashboard, and invoke the (temporary) Perl executable.
Find some kind of Perl-to-Python converter or binding.
Any other ideas? I'd love to hear if other people have confronted this problem. Unfortunately, it's not an option to convert the codebase itself to Python at this time.
I hate to be another one in the chorus, but...
Avoid the use of an alternate language
Use Wx so it's native look and feel makes the application look "real" to non-technical audiences.
Download the Padre source code and see how it does Wx Perl code, then steal rampantly from it's best tricks or maybe just gut it and use the application skeleton (using the Artistic half of the Perl dual license to make it legal).
Build your own Strawberry Perl subclass to package the application as an MSI installer and push it out across the corporate Active Directory domain.
Of course, I only say all this because you said "Dashboard" which I read as "Corporate", which then makes me assume a Microsoft AD network...
You can spawn a child process and use an IPC mechanism like sockets or STDIO, or even embed one interpreter in the other.
But why switch languages when Perl offers several Tk (Tk, Tkx, and Tcl::Tk) bindings and a very capable Wx binding?
I have written and distributed GUI projects with Perl's Tk and Wx libraries.
If you are need the ability to create stand-alone executables, check out PAR::Packer, ActiveState's PerlApp, and Cava Pacakger.
Try the CPAN distribution Python (pyperl) for interfacing with python code.
Well, if you really want to write the GUI in another language (which, seriously, is just a bad idea, since it will cost you more than it could ever benefit you), the thing you should do is the following:
Document your Perl app in terms of the services it provides. You should do it with XML Schema Definition - XSD - for the data types and Web Service Description Language - WSDL - for the actual service.
Implement the services in Perl, possibly using Catalyst::Controller::SOAP, or just XML::Compile::SOAP.
Consume the services from your whatever-language GUI interface.
Profit.
But honestly, I really suggest you taking a look at the Perl GTK2 binding, it is awesome, including features such as implementing a Gtk class entirely in Perl and using it as argument to a function written in C - for instance, you can write a model class for a gtk tree entirely in Perl.
I'd avoid inter-language calls if possible; fragility and massive increases in dependencies await you down this road. However, there is...
Inline::Python
If python must be used, the Inline::* series of modules have been generally well received. This lets you write python inside a perl script. You still have to write perl but it would let you use python libraries inside perl scripts. It will make things more difficult to debug, though.
I'd think the major criterion for any qualified answer would involve the details of the existing codebase. How is this Perl code called and how does it return it's results?
A collection of command line utilities returning results through reasonably good textual output ("good" as in "amenable to further machine parsing" or "pipeline friendly") ... should be reasonably easy to call from any programming language (and Python's excellent subprocess and multiprocessing modules in particular). A collection of web CGI or other modules layered between Apache and some DBMS system could still be accessed with things like urlopen2 or mechanize -- but it might be better to bypass the Perl code and write Python to query the underlying (presumably canonical) model (data store).
If the majority of the codebase is a set of libraries or modules ... and the functionality that your proposed dashboard requires isn't already exposed via some higher level mechanism (some command line interface, networking protocol, etc) ... then it's basically insane to consider interfacing to it through any language other than Perl. (Unless, by some strange and extremely unlikely twist of fate, your existing codebase and your intended implementation target are both already stable under Parrot).
Let's ask a different, broader, question: What interface to you intend to use between your dashboard and your existing code base?
This question is paramount regardless of your choice of implementation language. If you write the dashboard in Perl it still needs to call into your existing code base in some way. You probably need to fix-up your code base to implement support for whatever your going to use for your dashboard. At the point where your codebase supports the necessary API (has command line or IPC protocol calls into the desired functionality which return results over any reasonable IPC mechanism) ... then your choice of dashboard implementation language will be essentially arbitrary.
Interesting project: I would opt for loose-coupling and consider an XML-RPC or JSON based approach.
Find some kind of Perl-to-Python converter
Try out my pythonizer - it does a pretty good job at that!
There is a project I'm about to build in Smalltalk (Pharo). And there is a python library which I intend to use for the same. Now, there are 3 options:
Smalltalk wrapper for those python libraries
Porting the python library to Smalltalk
Write the library from scratch (in Smalltalk) for use in my project
The following are my queries:
What are 'basic' differences in porting/wrapping (No satisfactory explanation found anywhere yet)
How to know when to use which (of all the three)?
Any resources or pointers where I can get further help/some kick-start into my project.
Thanks!
Wrapper
Write functions in the native language whose sole purpose is to call the functions in the external library. The goal is to do as little as possible in the native language. For example, translating data types from the native language to the external library language, etc.
Wrappers make sense when the external library is:
written in a more efficient language than the native code (eg, a C++ library called from Python)
large/complex and would be time-consuming or error prone to translate
regularly updated; in a well-maintained library the interfaces (what your wrapper is concerned with) will change less often than the implementation of the features; so if you have wrappers around the functionality, updating to a new version of the library should be fairly straightforward
Porting
A port is simply a translation from one language to another. In general, the same logic is maintained as much as possible.
Porting makes sense when:
the native language is more efficient than the external library
the library is simple and one wants to save on the overhead involved with wrapping
one intends to make and maintain changes to the ported code in the native language
there are no plans to use the external library in its own language
one wants to learn one or both of the languages involved
Re-Write
Think of a Re-Write as a Port with a lot of refactoring. The goal is to take advantage of features of the native language to improve the library in some way (efficiency, readability, etc.)
Re-Writing makes sense in all of the same scenarios as porting. Deciding whether to do a simple port or a full re-write usually comes down to one question:
Is there a better way to implement the features of the external library in the native language?
Is there a language based on S-expressions with powerful macros that allows as seamless integration with Python as Clojure with JVM?
I want to try using such syntax and features while having access to all usual python libraries (including PyQt).
I've been working a project to do this: psil. I have a series of blog posts talking about what I've done. Here's the short manifesto:
Psil is a new general-purpose programming language in the Lisp family of languages. Psil is implemented on top of Python, allowing easy access to existing Python libraries.
Best features from Lisp and Scheme
Complete language in its own right
Built upon the Python standard libraries
Strong interoperability with Python code
The reality hasn't quite caught up to the vision; for example I don't think there is a way to declare new classes in Psil code that can be used from Python. But at least for functions, it's mostly there.
Note that Psil is built completely on Python 3, and there is no Python 2 version. I don't know whether there is a PyQt for Python 3.
While these aren't exactly what you're looking for, check:
CLPython - an implementation of Python in Common Lisp
(An ((Even Better) Lisp) Interpreter (in Python))
Check out Boo; it's a python-inspired language that runs on the CLR, with built-in support for full macros. If that's what you're missing from Lisp, give it a shot. A friend swears by it.
It's difficult to tell what is being asked here. This question is ambiguous, vague, incomplete, overly broad, or rhetorical and cannot be reasonably answered in its current form. For help clarifying this question so that it can be reopened, visit the help center.
Closed 12 years ago.
so I asked here few days ago about C# and its principles. Now, if I may, I have some additional general questions about some languages, because for novice like me, it seems a bit confusing. To be exact I want to ask more about language functions capabilities than syntax and so.
To be honest, its just these special functions that bothers me and make me so confused. For example, C has its printf(), Pascal has writeln() and so. I know in basic the output in assembler of these functions would be similar, every language has more or less its special functions. For console output, for file manipulation, etc. But all these functions are de-facto part of its OS API, so why is for example in C distinguished between C standard library functions and (on Windows) WinAPI functions when even printf() has to use some Windows feature, call some of its function to actually show desired text on console window, becouse the actual "showing" is done by OS. Where is the line between language functions and system API?
Now languages I don't quite understand - Python, Ruby and similar. To be more specific, I know they are similar to java and C# in term they are compiled into bytecode. But, I do not unerstand what are its capabilities in term of building GUI applications. I saw tutorial for using Ruby to program GUI applications on Linux and Windows. But isn´t that just some kind of upgrade? I mean fram other tutorials It seemed like these languages was first intended for small scripts than building big applications.
I hope you understand why I am confused. If you do, please help me sort it out a bit, I have no one to ask.
At the bottom you have the OS kernel itself - code that runs in a special CPU mode that allows direct access to otherwise protected resources. You will never have to deal with this unless you're an OS developer.
Then comes a do-not-cross line seperating this "kernel space" from "user space". Everything you do as "normal" developer is done in user space.
The OS kernel exports a limited number of very basic functions into user space, dubbed "system calls". Open a file, read / write a number of bytes, closing the file, for example.
Because these system calls usually require some Assembler code developers don't want to be bothered with, they are "wrapped" in (usually) C code functions: open(), read(), write(), close().
Now come two sets of APIs available to the developer: The OS API, and the standard language API.
The standard language API provides functions that can be used on any platform supporting the language: fopen(), fputc(), fgetc(), fclose(). It will also provide higher-level functions that make life easier: fprintf(), for example.
The OS API provides its own set of functions. These are not portable to a different operating system, but might be more intuitive to use, or more powerful, or merely different. OpenFile(), ReadFile(), WriteFile(), CloseFile(). Again, higher-level functions might be available, i.e. PrintLn().
The two sets of functions might partially rely on each other, or make system calls directly, but that shouldn't bother you too much. You should, however, decide beforehand which set of functions you will want to use for your project, because mixing the two sets - while not a mistake in itself - opens a whole new can of worms (i.e., potential errors).
C is portable. That means that on different systems the assembler output for printf will be different... this is something the compiler does based on what your target system is. Write C code and compile as a Linux app and the output will be different than as a Win32 app, and also different than if you compile the exact same code for an iPhone or something like that.
Internally, the C standard libraries might wrap a call to Win32 API when you call printf, but that's not really your concern in most cases. The C standard library (like printf and other I/O for files and stuff) wraps the low-level OS or hardware code needed to do what you want.
It's worth noting the same effect happens in Java, but in a different way. At a broad level: In Java, the code you write always compiles to the same byte-code. But then when the JVM runs this byte-code, the JRE translates it to machine-specific instructions at run-time, rather than at compile-time on C.
So.
For your first question, the interface between the C API and the OS API is the C runtime. On Windows this is some incarnation of MSVCRT.DLL, whereas on Linux this is glibc.
For the second, the native language for most GUI toolkits is either C or C++. Higher-level languages seeking to use them require bindings which translate back and forth between the language and the C/C++ API.
For the third, these high-level languages only appear to be used for "small scripts". The simple fact is that they are far more expressive than C or C++, which means that they have equal or more capabilities than a C or C++ program while being written in fewer lines of code.
If I assume this is your central question:
Where is the line between language functions and system API?
Then imagine if you will this analogy:
OS API system calls are like lego bricks and lego components.
Programming 'functions' are merely an arrangement of many lego bricks. Such that the combination results in a tool.
Thus different languages may 'arrange' and create the tool in different ways.
If I asked you to create a car with lego's, you could come up with many different designs.
C's printf() is a wrapper. You can use it and compile your code under any OS, but the resulting machine code will be different. In Windows, it might call some function inside the Windows API. In Linux, it will use the Linux API. You ask why is the Windows API distinguished. That's because, if you're programming for Windows, you can use it to do some OS-specific things like create GUIs, manipulate console text instead of just printing, asking for OS resources, and stuff like that. An API like that exists for Linux and Mac (and I guess all the other OS's) too, and they let you do more or less the same things. Unlike printf(), though, they are not portable.
You ask what is the line between language functions and the system API. The language functions simply call the OS's API. You can call these yourself, but then you won't be able to compile your code on different systems.
Python and Ruby (and some others) are interpreted. They are compiled to bytecode behind the scenes, but all the user sees is that double-clicking the source file will run it. No need to compile. That means, obviously, that they're slower than compiled languages. However, their dynamic nature makes for faster development, since you usually need less code to do the same thing (I said usually).
That doesn't mean these languages can't be used for "big" applications: There are GUI libraries for them. That's because these are general purpose languages, unlike some others like Bash.
To be more specific, I know they are similiar to java and C# in term they are compiled into
bytecode.
Ruby and Python are both interpreted languages, http://en.wikipedia.org/wiki/Interpreted_language, and their code is not translated into bytecode prior the execution.
A lot of useful features in Python are somewhat "hidden" inside modules. Named tuples (new in Python 2.6), for instance, are found in the collections module.
The Library Documentation page will give you all the modules in the language, but newcomers to Python are likely to find themselves saying "Oh, I didn't know I could have done it this way using Python!" unless the important features in the language are pointed out by the experienced developers.
I'm not specifically looking for new modules in Python 2.6, but modules that can be found in this latest release.
The most impressive new module is probably the multiprocessing module. First because it lets you execute functions in new processes just as easily and with roughly the same API as you would with the threading module. But more importantly because it introduces a lot of great classes for communicating between processes, such as a Queue class and a Lock class which are each used just like those objects would be in multithreaded code, as well as some other classes for sharing memory between processes.
You can find the documentation at http://docs.python.org/library/multiprocessing.html
The new json module is a real boon to web programmers!! (It was known as simplejson before being merged into the standard library.)
It's ridiculously easy to use: json.dumps(obj) encodes a built-in-type Python object to a JSON string, while json.loads(string) decodes a JSON string into a Python object.
Really really handy.
May be PEP 0631 and What's new in 2.6 can provide elements of answer. This last article explains the new features in Python 2.6, released on October 1 2008.
Essential Libraries
The main challenge for an experienced programmer coming from another language to Python is figuring out how one language maps to another. Here are a few essential libraries and how they relate to Java equivalents.
os, os.path
Has functionality like in java.io.File, java.lang.Process, and others. But cleaner and more sophisticated, with a Unix flavor. Use os.path instead of os for higher-level functionality.
sys
Manipulate the sys.path (which is like the classpath), register exit handlers (like in java Runtime object), and access the standard I/O streams, as in java.lang.System.
unittest
Very similar (and based on) jUnit, with test fixtures and runnable harnesses.
logging
Functionality almost identical to log4j with loglevels and loggers. ( logging is also in the standard java.util.Logging library)
datetime
Allows parsing and formatting dates and times, like in java.text.DateFormat, java.util.Date and related.
ConfigParser
Allows persistant configuration as in a java Properties file (but also allows nesting). Use this when you don't want the complexity of XML or a database backend.
socket, urllib
Similar functionality to what is in java.net, for working with either sockets, or retrieving content via URLs/URIs.
Also, keep in mind that a lot of basic functionality, such as reading files, and working with collections, is in the core python language, whereas in Java it lives in packages.