I have a some old Fortran77 codes, which does some calculations. Now I build a website hosted by Google App Engine, and need to call those models' calculation results. Since I am new to both GAE and PiCloud, my basic questions are:
Should I first compile those Fortran77 code using a windows compiler?
Then, publish those models to PiCloud
Call from GAE?
Does my approach make sense? or Does PiCloud has Fortran77 environment, which can directly do the calculation without compiling first? If so, is there any example about this topic?
Thanks!
Picloud claim "install any library or binary written in any language", so it's safe to say you can run fortran programs on it. In fact, their homepage even says:
You can deploy any software written in any programming language
including C, C++, Java, R, Fortran, Ruby, etc.
You shouldn't compile it with a windows compiler, because picloud runs linux - compile it using a linux compiler, such as GCC.
Regarding using it from App Engine, see this page, where it says:
[...] use PiCloud from Google AppEngine, since our cloud client library is not supported on GAE.
Related
I am new to Google Cloud Platform and in my whole I have been working on Python 3. I am trying to find out which version of Python is more complete for Google App Engine: Python 2.7 or Python 3.
As I'm starting to work with Google App Engine I have realised that continuing using Python 3 seems too painful as basic tools like dev_appserver.py are written for Python 2 only. Now I am hitting the opposite problem: cloudstorage module seems to exist only for python3. Again, when I install it, seems the only way I can test read/write to google bucket locally is by authenticating with google.appengine.ext, which in turn only works within dev_appserver.py or remotely. This leaves me confused which environment to chose.
What is a general agreement / what is the focus of Google App Engine: Python 2 or Python 3?
In App Engine, you have to options: the Standard environment and the Flexible environment.
Python 2.7 is available in both Standard and Flexible, while Python 3.6 is only available in Flexible.
Also, the choice between Standard and Flexible depends on what you want to do/what libraries you need:
There are some third-party libraries already built-in in the Standard Environment, and you can include other libraries, but, those libraries can't include C extensions, they must be written in pure Python. If you need libraries with C extensions, you will have to move to Flexible.
In Standard, you can use propietary libraries (like google.appengine.ext, as you mentioned) to do tasks like accessing databases, while in Flexible you can use other libraries (like the client you mentioned).
There are also another important differences, like pricing, scaling, etc. The choice will depend, as I said, in your needs for your application.
EDIT
dev_appserver.py is only used when developing in Standard. There is a tutorial in here, with Flask. If you are in Flexible, you can test the app locally as if you were running as usual a python file, like in this other example.
You can use buckets in both Standard and Flexible
The python3-only cloudstorage support assumption based on the SO post you referenced is not correct:
the import appears to be done in a regular python shell or as a standalone script, not from a standard environment GAE app - different things, see import cloudstorage, ImportError: No module named google.appengine.api.
it is not specified where that library comes from
GCS is definitely supported in the standard env GAE (i.e. on python 2), you just need to follow the steps from the official documentation: Setting Up Google Cloud Storage and Reading and Writing to Google Cloud Storage.
Both were good. But the question is what kind of environment do you want? Standard environment or Flexible environment.
Find your answer in this document: https://cloud.google.com/appengine/docs/python/
It kind of depends on what you're using it for. If you're doing data science, for example, I'm seeing a few notices of Python libraries that are (finally) dropping support for Python 2. numpy is one that is dropping support.
Generally speaking, I would recommend Python 3 over Python 2. Why spend time developing in an aging version when its replacement has matured nicely and is more consistent?
I am planning on writing some software for a web server that uses machine learning to process large amounts of data. This will be real estate data from a MySQL server. I will be using the CUDA framework from Nvidia with python/caffe or the c++ library. I will be using a Tesla P100. Although python is more widely used for machine learning I presume it is hard to write a server app in python without sacrificing performance. Is this true? Is c++ well supported for machine learning? Will anything be sacrificed by writing a professional server app in python (ex: connecting to MySQL database)?
Python is a language that performs worse than c++ in terms of runtime for several reasons:
First and foremost, Python is a scripting language that runs with an interpreter as opposed to c++ which compiled into machine code before running.
Secondly: python runs in the background a garbage collector system while in c++ the memory management is done manually by the programmer.
In your case, I recommend that you work with Python for several reasons:
Writing in Python in CUDA allows you to compile the code even though it is Python (CUDA provides JIT - Just In Time compiler, as well as a compiler and other effective tools), Which greatly improves performance
Python provides many, rich varied libraries that will help you a lot in the project, especially in the field of machine learning.
The development time and code length will be significantly shorter in Python.
From my experience with working at CUDA in the Python language I recommend you use numba and numbapro libraries, they are comfortable to work with and provide support for many libraries like numpy.
Best of luck.
Both C++ and Python are perfectly reasonable languages for implementing web servers. Python is actually quite common, and there are many frameworks for writing web servers in Python such as flask, bottle, django, etc. Architecturally, I wonder whether you really need the machine learning (which I imagine would be a data processing pipeline) and the web server to be the same process / binary; however, even if you do them in the same server, I suspect that either language would be perfectly reasonable; moreover, if you ever came to the point where you needed to run a piece of computation in C++ for performance, using SWIG to call C++ from Python or using some form of message passing from Python to a C++ helper process (such as via gRPC) are options.
Let me explain what I'm trying to achieve. In the past while working on Java platform, I used to write Java codes(say, to push or pull data from MySQL database etc.) then create a war file which essentially bundles all the class files, supporting files etc and put it under a servlet container like Tomcat and this becomes a web service and can be invoked from any platform.
In my current scenario, I've majority of work being done in Java, however the Natural Language Processing(NLP)/Machine Learning(ML) part is being done in Python using the NLTK, Scipy, Numpy etc libraries. I'm trying to use the services of this Python engine in existing Java code. Integrating the Python code to Java through something like Jython is not that straight-forward(as Jython does not support calling any python module which has C based extensions, as far as I know), So I thought the next option would be to make it a web service, similar to what I had done with Java web services in the past. Now comes the actual crux of the question, how do I run the ML engine as a web service and call the same from any platform, in my current scenario this happens to be Java. I tried looking in the web, for various options to achieve this and found things like CherryPy, Werkzeug etc but not able to find the right approach or any sample code or anything that shows how to invoke a NLTK-Python script and serve the result through web, and eventually replicating the functionality Java web service provides. In the Python-NLTK code, the ML engine does a data-training on a large corpus(this takes 3-4 minutes) and we don't want the Python code to go through this step every time a method is invoked. If I make it a web service, the data-training will happen only once, when the service starts and then the service is ready to be invoked and use the already trained engine.
Now coming back to the problem, I'm pretty new to this web service things in Python and would appreciate any pointers on how to achieve this .Also, any pointers on achieving the goal of calling NLTK based python scripts from Java, without using web services approach and which can deployed on production servers to give good performance would also be helpful and appreciable. Thanks in advance.
Just for a note, I'm currently running all my code on a Linux machine with Python 2.6, JDK 1.6 installed on it.
One method is to build an XML-RPC server, but you may wish to fork a new process for each connection to prevent the server from seizing up. I have written a detailed tutorial on how to go about this: https://speakerdeck.com/timclicks/case-studies-of-python-in-parallel?slide=68.
NLTK based system tends to be slow at response per request, but good throughput can be achieved given enough RAM.
I am basically a PHP guy. now moving towards python. I am starting to learn python.
How do I install it and start working it, and develop websites .
I got totally confused with the alternative implementations in the download section of the Python site. Can you tell me what "alternative implementations" means?.
I mean to say: I can create a .php file in my server and then access it from browser like http://example.com/index.php, so I was wondering whether I can do the same with python, like creating a .py file and accessing from browser http://example.com/index.py.
Just as a disclaimer, I interpret you saying "run Python in a browser" as "making a website with Python."
If you want to start writing web applications in Python, you can either use CGI or use one of its many web app frameworks. Python is not like PHP in the sense that you can't just embed it in HTML. Many of those frameworks come with development servers that you can use to test your web app (by looking at it in a browser).
A particularly good Python web framework is Django.
I really do recommend that you do the Python tutorial before you dive into any of those frameworks, though. Python is not only for writing web applications, so you'll have to get some fundamentals down first before any of that makes sense to you.
As for installing Python, I recommend you take the version you got with your OS if you use Mac or Linux, or installing the Python 2.7.1 32-bit binary from python.org if you use Windows. The alternative implementations include a Python that runs on the Java virtual machine and one that runs on the .NET Common Language Runtime, but for your purposes the reference implementation, CPython, should work fine.
Python by default is a general purpose scripting language and is not meant specifically for web application development (like PHP is primarily). So, you'll first need to download and install Python (choose the version that's appropriate for your OS) from www.python.org I'd recommend v2.7.1 but you can try out v3.2 if you'd like.
The other versions (IronPython, Jython, etc) are Python implementations on other platforms (.NET and JVM respectively), and in all likelihood you don't need to bother with them unless you really want to.
To start working with Python and create web applications you will also need to download and install a Python-based web framework. There are many of them, too many in fact to list here. However, there is a page on the Python Wiki that has a list of useful frameworks for web development.
You don't "run it from a browser". You don't run PHP from a browser, either.
The way to use Python for web development is to use a framework like Django or Pyramid.
Sage or CodeNode let you run Python in a browser.
You can get a limited Python experience by pointing your browser to http://shell.appspot.com/ . You will be running Python 2.5.2 on Google's site.
This link says that Android support Python, Lua and BeanShell Scripts, subsequently for Perl too. If it is so, is it possible for developers to write python scripts and call them in their standard Java based android applications?
I remember reading about this awhile back as well.
It's not on the android dev site.
It's a separate project, android-scripting.
Python API:
API Reference
SL4A API Help
I think I have read somewhere that ASE with Python was a huge library ( several Mo), and so was completely unpractical for a public application.
But you can still use it for development...