For security reasons I need to encrypt a column before putting the data in the repository.
How can I do this using python or some processor.
I tried to use the cryptography library but without success.
If you can guide me which is the best way I would be very grateful.
Take a look at ScriptedTransformRecord, it should allow you to do arbitrary operations on whichever field/columns you like. If you're using a Python library, remember that the script engine is Jython and cannot import modules that aren't pure Python (CPython modules for example). Java has crypto libraries so you could achieve this with Groovy instead.
There is a Jira to cover an EncryptAttributes processor, but perhaps there should be one for Expression Language and/or RecordPath functions as well.
Related
I want to write golang bindings for an existing (third party) Python module.
The purpose is that I want to use the API that the Python module provides in Golang.
I already found golang bindings for Python's C API (go-python3 for py3 and go-python for py2), but I still haven't figured out how to translate a relatively complex Python module into Golang (i.e. how to deal with type safety in go for unsafe inputs and returns in python, etc).
What would be a good approach? Are there any pre-existing tools in that space? Are there any good examples for Golang bindings for Python code? (I couldn't find many tbh).
I want to use the API that the Python module provides in Golang.
Calling Python from Go is detailed recently in "Python and Go : Part I - gRPC" by Miki Tebeka.
You can see an example in ardanlabs/python-go/grpc
But, as shown in their next two articles, you can also:
compiled Go code to a shared library and used it from the Python interactive shell.
use a Python module that hides the low level details of working with a shared library and then package this code as a Python package.
Full example: ardanlabs/python-go/pyext.
I want to use numpy in brython.
But I do not know how to import extra modules in brython.
If you have similar experience or question,
please tell me way to solve
You can't.
Brython is a transpiler from a Python-like syntax to Javascript code. While it holds good conformance to latest Python 3 syntax, and amazing compatibility, it does not have any features to support running binary code.
And numpy, along with a lot of third and first party Python libraries is compiled from C and other languages to native CPU code. There is no support, no way, and not even a roadmap, or prevision of prevision to allow that.
It may be possible to develop Brython front end apps that use those libraries running on the backend, though, performing all calculations on the server side, and just passing serialized data to the brython counterpart. One would habe to manually write this, though (and it might be possible to use a pure-python striped down version of numpy arrays just for deserializing, displaying, and submitting data to the backend)
Is there any good way to call python from clojure as a means of doing data science with scipy, numpy, scikit-learn, etc.
I know about implementations of clojure which run on python instead of java, but this doeesn't work for me, as I also need to call java libraries in my project. I also know about Jython, but I don't know of a clean way to use this with Clojure.
I want to use Clojure in my projects because I prefer it as a language, but I can't deny that Python has an incredible community, and some of the most beautiful, well-designed libraries around.
Instead of trying to get Jython to play well with both Clojure and numpy/scipy, you can use Hy. It is hosted on Python and it somewhat resembles Clojure.
If I really wanted to use numpy/scipy, I would write the backend in Python (or Hy), run it as a separate service. And if I really like ring for instance, or can't live without Instaparse, I would write a frontend in Clojure.
As an aside Python has EDN libs. It would be an interesting project to integrate one of them in Hy, or write one from scratch.
Give the toolz library a try, it's a functional standard library for Python that was designed to generally adhere to the API of the Clojure standard library.
Apart from that, I'd encourage you to find the seams between your computations, and write individual tools in the Unix way in either Clojure or Python depending on which seems to fit that use case best. Serialize data between the tools, either as text/JSON through pipes or using a binary serialization format like Protobuf, which has standard APIs for both Java and Python.
If you had a gun to my head and told me to build Clojure/Python interop, I'd start with py4j and bridge the two languages through Java interfaces, using implements members in a Python class and reify on the Clojure side.
You could use Graal VM now. Although some large companies are using it in production, it's still early days. Here's an example of using Python from Clojure:
(.eval context "python" "
import time;
time.clock()
")
http://gigasquidsoftware.com/blog/2017/10/22/embedded-interop-between-clojure-r-and-python-with-graalvm/
During developing one of my applications, I've come to a point where I'd like to give the users a more powerful filter. Therefore, I'd like to provide a simple scripting interface to the users. The scripting language would be Python.
For obvious reasons, I'd like to tighten the scope of the language to match my particular purposes (I don't want the users to touch the server's HDD files etc.). I also don't want to write a Python interpreter myself (which would be reinventing the wheel and the "new" wheel would be rectangular in the end). However, I haven't found any suitable library or module for this purpose.
Groovy's approach with its Compilation Customizers and Compiler Configuration would be exactly what I want, does something similar exist for Python?
What you're looking for is called a "sandbox" or "restricted execution." This wiki page discusses some of the details.
In a nutshell, there have been several efforts by Python geeks and gurus to build a sandbox on top of Python but they all failed eventually.
The main reason is that Python offers so many paths to do something that the sandbox would either have to forbid common use cases (rendering a lot of the library and Python code useless) or it would have to have holes in the sandbox which would make the concept useless.
So while it looks like a good and simple idea, so far, there is no solution. AFAIK, there are no hooks in Python to tweak the byte code compiler to achieve something like Groovy Sandbox.
Related:
How can I sandbox Python in pure Python?
Is there a "safe" subset of Python for use as an embedded scripting language?
I have a C extension module for Python and I want to make it available to Rubyists.
The source has a number of C modules, with only one being Python-dependent. The rest depend only on each other and the standard library. I can build it with python setup.py build in the usual way.
I've been experimenting with adding Ruby support using newgem and I can build a version of the extension with rake gem. However, the combined source has an ugly directory layout (mixing Gem-style and Setuptools-style structures) and the build process is a kludge.
I can't just keep all the sources in the same directory because mkmf automatically picks up the Python-dependent module and tries to build that, and users shouldn't have to install Python to compile a module that won't be used. My current hack is for extconf.rb to copy the Python-independent source-files into the same directory as the Ruby-dependent extension module.
Is there a saner way to make the code available to both languages? Should I just duplicate the Python-independent code in a separate Gem? Should I release the independent code as a separate lib built with autotools? Is there a version of mkmf that can skip the unwanted module?
One way to solve it is to create three different projects:
The library itself, independent on python & ruby
Python bindings
Ruby bindings
That's probably the cleanest solution, albeit it requires a bit more work when doing releases, but it has the advantage that you can release a new version of the Ruby bindings without having to ship a new library/python bindings version.
Complementing on what Johan said, I've used a couple c/c++ support libraries in Python thanks to swig. You write your code in c/c++ then make an intermediary template for each language that you want to support. Its rather painless for Python, but some considerations must be made for Ruby... namely I don't think pthread support is to happy with ruby or vice versa.
http://www.swig.org/
It's got a somewhat steep learning curve so it might be best to find an example project out there that demonstrates how to use the wrapper for your target languages.
This is definitely a useful tool as it makes your code a lot cleaner while still providing robust bindings to multiple languages (PHP, Python, Ruby, and I believe c#)