I have a program written in Python (.py) which requires to import and use a function written in Golang (.go) package located in Github repo.
How can I import the .go package module in python code and use the function. The function should accept args from my python code and return a value after doing some operations.
Note: I want to achieve this in Python2.7 version.
go_file.go
// Utility function to get string formed from input list of strings.
func NewFromStrings(inputs []string) string {
// do something
return "abc"
}
python_file.py
# This is just a pseudo code to make problem statement more clear.
import github.com/path_to_package/go_file
str_list = ['abc', 'def']
result = go_file.NewFromStrings(str_list)
print(result)
Thanks in advance :)
You have a few options, but it isn't possible to directly import Go code from python without a little work.
To do the inverse of the question you linked in your comment, you can create a small go cli to expose your Go function. You can then execute this Go cli from python using subprocess.run. The result will then be accessible from stdout. See https://docs.python.org/3/library/subprocess.html#subprocess.run
Alternatively, you can communicate using IPC, like sockets, which will involve creating a little server in Go.
The third option I can think of is exporting your Go function in a native library and calling it from python using c bindings. This is probably the most complicated route, but will get you closest to being able to import go code from python.
Check out this blog post with a much more thorough break down of some of your options: https://www.ardanlabs.com/blog/2020/06/python-go-grpc.html
Related
Is there a way to save and load an entire namespace using pickle allowing me to perform something like this:
import pickle
import numpy as np
a = 1
# other arbitrary code
pickle.dump(namespace,open('my_namespace.p','wb')
Allowing to open a new python prompt and performing the following code:
import pickle
namespace.update(pickle.load(open('my_namespace.p','rb'))
print(np.array(a)+1)
>>2
# using the rest of the namespace
If your intention is to 'freeze' a particular code version of the modules you use I suggest taking a look at Python Virtual Environment (venv). Otherwise, as jasonharper suggested - it is very unlikely to work well (particularly with modules like NumPy which are complex amalgamates of Python code and binary libraries).
venv would help you deal with that particular problem too.
I think that I get the question. You want to be able to restore all variables and interact with a shell in a different interpreter or at a different time. The easiest way to do this is probably to use a notebook like jupyter or ipython. These allow you to save all commands run and reconstitute them. Sorry for it not being a direct answer, but I think it may "equivalently accomplish" what you specified.
I have a Python 3 file. I want to use an open-source tool on the internet (nltk), but unfortunately it only supports Python 2. There is no way for me to convert it to Python 3, nor can I convert my Python 3 file to Python 2.
If the user does not give a certain argument (on argparse) then I do something in my file. If the user does give a certain argument, however, I need to use nltk.
Writing a Python 2 script that uses nltk and then executing script that in my Python 3 script
My current idea is to write a script in Python 2 that does what I want with nltk and then run that from my current Python 3 script. However, I don't actually know how to do this.
I found this code: os.system(command) and so I will modify it to be os.system("python py2.py") (where py2.py is my newly written Python 2 file).
I'm not sure if that will work.
I also don't know if that is the most efficient way to solve my problem. I cannot find any information about it on the internet.
The data transferred will probably be quite large. Currently, my test data is about 6600 lines, utf-8. Functionality is more important than how long it takes (to a certain extent) in my case.
Also, how would I pass values from my Python 2 script to my Python 3 script?
Thanks
Is there any other way to do this?
Well, if you're sure you can't convert your script to Python 2, then having one script call the other by running the Python interpreter probably is the best way. (And, this being Python, the best way is, or at least should be, the only way.)
But are you sure? Between the six module, the 3to2 tool, and __future__ statements, it may not be as hard as you think.
Anyway, if you do need to have one script call the other, you should almost never use os.system. As the docs for that function say:
The subprocess module provides more powerful facilities for spawning new processes and retrieving their results; using that module is preferable to using this function. See the Replacing Older Functions with the subprocess Module section in the subprocess documentation for some helpful recipes.
The simplest version is this:
subprocess.check_call(["python", "py2.py"])
This runs your script, waits for it to finish, and raises an exception if the script returns failure—basically, what you wanted to do with os.system, but better. (For example, it doesn't spawn an unnecessary extra shell, it takes care of error handling, etc.)
That assumes whatever other data you need to share is being shared in some implicit, external way (e.g., by accessing files with the same name). You might be better off passing data to py2.py as command-line arguments and/or stdin, passing data back as via stdout, or even opening an explicit pipe or socket to pass things over. Without knowing more about exactly what you need to do, it's hard to suggest anything, but the docs, especially the section Replacing Older Functions with the subprocess Module have lots of discussion on the options.
To give you an idea, here's a simple example: to pass one of your filename arguments to py2.py, and then get data back from py2.py to py3.py, just have py3.py do this:
py2output = subprocess.check_output(["python", "py2.py", my_args[0]])
And then in py2.py, just print whatever you want to send back.
The Anyone hear when NLTK 3.0 will be out? here in SO points out that...
There's a Python 3 branch:
https://github.com/nltk/nltk/tree/nltk-py3k
The answer is from July 2011. It could be improved since then.
I have just looked at https://github.com/nltk/nltk. There is at least the document that talks about Python 3 port related things https://github.com/nltk/nltk/blob/2and3/web/dev/python3porting.rst.
Here is a longer discussion on NLTK and Python 3 that you may be interested in.
And the Grants to Assist Kivy, NLTK in Porting to Python 3 (published 3 days ago) is directly related to the problem.
I have used the Python's C-API to call some Python code in my c code and now I want to profile my python code for bottlenecks. I came across the PyEval_SetProfile API and am not sure how to use it. Do I need to write my own profiling function?
I will be very thankful if you can provide an example or point me to an example.
If you only need to know the amount of time spent in the Python code, and not (for example), where in the Python code the most time is spent, then the Python profiling tools are not what you want. I would write some simple C code that sampled the time before and after the Python interpreter invocation, and use that. Or, C-level profiling tools to measure the Python interpreter as a C function call.
If you need to profile within the Python code, I wouldn't recommend writing your own profile function. All it does is provide you with raw data, you'd still have to aggregate and analyze it. Instead, write a Python wrapper around your Python code that invokes the cProfile module to capture data that you can then examine.
I'm trying to get results of db.stats() mongo shell command in my python code (for monitoring purposes).
But unlike for example serverStatus I can't do db.command('stats'). I was not able to find any API equivalent in mongodb docs. I've also tried variations with db.$cmd but none of that worked.
So,
Small question: how can I get results of db.stats() (number of connections/objects, size of data & indexes, etc) in my python code?
Bigger question: can anyone explain why some of shell commands are easily accessible from API, while others are not? It's very annoying: some admin-related tools are accessible via db.$cmd.sys, some via db.command, some via ...? Is there some standard or explanation of this situation?
PS: mongodb 2.0.2, pymongo 2.1.0, python 2.7
The Javascript shell's stats command helper actually invokes a command named dbstats, which you can run from PyMongo using the Database.command method. The easiest way to find out what command a shell helper will run is to invoke the shell helper without parentheses -- this will print out the Javascript code it runs:
> db.stats
function (scale) {
return this.runCommand({dbstats:1, scale:scale});
}
As for why some commands have helpers and others do not, it's largely a question of preference, time, and perceived frequency of use by the driver authors. You can run any command by name with Database.command, which is just a convenience wrapper around db.$cmd.find_one. You can find a full list of commands at List of Database Commands. You can also submit a patch against PyMongo to add a helper method for commands you find that you need to invoke frequently but aren't supported by PyMongo yet.
I'm developing a system that operates on (arbitrary) data from databases. The data may need some preprocessing before the system can work with it. To allow the user the specify possibly complex rules I though of giving the user the possibility to input Python code which is used to do this task. The system is pure Python.
My plan is to introduce the tables and columns as variables and let the user to anything Python can do (including access to the standard libs). Now to my problem:
How do I take a string (the user entered), compile it to Python (after adding code to provide the input data) and get the output. I think the easiest way would be to use the user-entered data a the body of a method and take the return value of that function a my new data.
Is this possible? If yes, how? It's unimportant that the user may enter malicious code since the worst thing that could happen is, that he screws up his own system, which is thankfully not my problem ;)
Python provides an exec() statement which should do what you want. You will want to pass in the variables that you want available as the second and/or third arguments to the function (globals and locals respectively) as those control the environment that the exec is run in.
For example:
env = {'somevar': 'somevalue'}
exec(code, env)
Alternatively, execfile() can be used in a similar way, if the code that you want executed is stored in its own file.
If you only have a single expression that you want to execute, you can also use eval.
Is this possible?
If it doesn't involve time travel, anti-gravity or perpetual motion the answer to this question is always "YES". You don't need to ask that.
The right way to proceed is as follows.
You build a framework with some handy libraries and packages.
You build a few sample applications that implement this requirement: "The data may need some preprocessing before the system can work with it."
You write documentation about how that application imports and uses modules from your framework.
You turn the framework, the sample applications and the documentation over to users to let them build these applications.
Don't waste time on "take a string (the user entered), compile it to Python (after adding code to provide the input data) and get the output".
The user should write applications like this.
from your_framework import the_file_loop
def their_function( one_line_as_dict ):
one_line_as_dict['field']= some stuff
the_file_loop( their_function )
That can actually be the entire program.
You'll have to write the_file_loop, which will look something like this.
def the_file_loop( some_function ):
with open('input') as source:
with open('output') as target:
for some_line in source:
the_data = make_a_dictionary( some_line )
some_function( the_data )
target.write( make_a_line( the_data ) )
By creating a framework, and allowing users to write their own programs, you'll be a lot happier with the results. Less magic.
2 choices:
You take his input and put it in a file, then you execute it.
You use exec()
If you just want to set some local values and then provide a python shell, check out the code module.
You can start an instance of a shell that is similar to the python shell, as well as initialize it with whatever local variables you want. This would assume that whatever functionality you want to use the resulting values is built into the classes you are passing in as locals.
Example:
shell = code.InteractiveConsole({'foo': myVar1, 'bar': myVar2})
What you actually want is exec, since eval is limited to taking an expression and returning a value. With exec, you can have code blocks (statements) and work on arbitrarily complex data, passed in as the globals and locals of the code.
The result is then returned by the code via some convention (like binding it to result).
well, you're describing compile()
But... I think I'd still implement this using regular python source files. Add a special location to the path, say '~/.myapp/plugins', and just __import__ everything there. Probably you'll want to provide some convenient base classes that expose the interface you're trying to offer, so that your users can inherit from them.