I have a GUI program that I have developed in Python 3.2 to extract various geospatial data products. I need to call a module I have developed in Python 2.7.
I am looking for a way to be able to call the Python 2.7 code using a Python 2.7 interpreter inside of Python 3.2 program. I cannot port the 2.7 to Python 3.2 as it uses a Python version installed with ESRI ArcMap and relies on the arcpy module which is not available for Python 3. My only idea right now would be to use subprocess to call the module as a batch process however this is a bit messy and I would prefer the two programs had some relationship.
Thank you in advance.
You could spawn the python 2.7 process as a server handling RPC requests from your GUI running on 3.2. That will work either over the network, or local pipes, or shared memory, or your system's message bus, or many other ways. You just need to translate your library's API into some kind of serialised messages.
Let's say your library has a function: (super simplified example)
def add(a, b):
return a+b
You'd wrap this in some server, let's say a flask app, which does:
#app.route("/add", methods=["POST"])
def handle_add():
data = request.get_json()
ret = your_lib.add(data['a'], data['b'])
return jsonify(ret)
and on the client side, send and unpack the values using something like requests
You could even make it fairly transparent by implementing a translator module with methods named the same as the library itself and doing import your_http_wrapper as your_library_name.
The trick now is to make sure all your parameters can be serialised and that you can realistically send all the arguments/return values in a reasonable time on each call. Also, you lose the ability to change the contents of the variables you pass to the wrapper, because the server will modify only the local copy (unless you implement serialising all those modifications as well)
Try using subprocess.check_output(['C:\\python27\\python.exe', 'yourModule.py'])
If you want to call a specific function within the 27 file, you can use more system arguments. The call would look like:
subprocess.check_output(['C:\\python27\\python.exe', 'yourModule.py', 'funcName'])
And in the 27 file you can add:
import sys
if __name__=='__main__':
if 'funcName' in sys.argv:
funcName()
else:
#... execute normally
Somewhat late but in case someone stumbles over this thread:
I've written a module that transparently runs parts of a program in other python interpreters. Basically it provides decorators and base classes that replace functions and objects with proxies that interface with other interpreters.
It's intended for compatibility, e.g. running python2 only code in python3, or accessing C modules from pypy.
In the following example, the loop runs 4x as fast via using pypy as it would with plain python.
#!/usr/local/bin/python
from cpy2py import TwinMaster, twinfunction
import sys
import time
import math
# loops in PyPy
#twinfunction('pypy')
def prime_sieve(max_val):
start_time = time.time()
primes = [1] * 2 + [0] * (max_val - 1)
for value, factors in enumerate(primes):
if factors == 0:
for multiple in xrange(value*value, max_val + 1, value):
primes[multiple] += 1
return {'xy': [
[primes[idx] == 0 for idx in range(minidx, minidx + int(math.sqrt(max_val)))]
for minidx in range(0, max_val, int(math.sqrt(max_val)))
], 'info': '%s in %.1fs' % (sys.executable, time.time() - start_time)}
# matplotlib in CPython
#twinfunction('python')
def draw(xy, info='<None>'):
from matplotlib import pyplot
pyplot.copper()
pyplot.matshow(xy)
pyplot.xlabel(info, color="red")
pyplot.show()
if __name__ == '__main__':
twins = [TwinMaster('python'), TwinMaster('pypy')]
for twin in twins:
twin.start()
data = prime_sieve(int(1E6))
draw(**data)
Related
I use a software library that has different function names in different versions of the library.
I try to use the following code:
some_variable = module.old_name_of_function()
But this code only works with the old version of the program library.
I plan to use the code on different computers, with different installed versions of the software library.
Some computers have a new version of the program library installed, and the following code should be used there:
some_variable = module.new_name_of_function()
And if I use old_name_of_function() in the new version of the library, I will get an error.
How to solve this issue?
I suppose you could do
try:
my_func = module.old_name_of_function
except AttributeError:
my_func = module.new_name_of_function
some_variable = my_func()
You can use pkg_resources module for it (example for numpy):
import pkg_resources
pkg_resources.get_distribution("numpy").version
will return:
'1.15.2'
Then you can use cases, ifs or something else to run a function you need.
For example:
import pkg_resources
version = pkg_resources.get_distribution("numpy").version
v = version.split('.')
if int(v[0]) == 1 and int(v[1]) < 17:
print('WAKA')
else:
print('NEW WAKA')
will print 'WAKA' for every 1.X version of numpy, where X < 17.
The basic question is this: Let's say I was writing R functions which called python via rPython, and I want to integrate this into a package. That's simple---it's irrelevant that the R function wraps around Python, and you proceed as usual. e.g.
# trivial example
# library(rPython)
add <- function(x, y) {
python.assign("x", x)
python.assign("y", y)
python.exec("result = x+y")
result <- python.get("result")
return(result)
}
But what if the python code with R functions require users to import Python libraries first? e.g.
# python code, not R
import numpy as np
print(np.sin(np.deg2rad(90)))
# R function that call Python via rPython
# *this function will not run without first executing `import numpy as np`
print_sin <- function(degree){
python.assign("degree", degree)
python.exec('result = np.sin(np.deg2rad(degree))')
result <- python.get('result')
return(result)
}
If you run this without importing the library numpy, you will get an error.
How do you import a Python library in an R package? How do you comment it with roxygen2?
It appears the R standard is this:
# R function that call Python via rPython
# *this function will not run without first executing `import numpy as np`
print_sin <- function(degree){
python.assign("degree", degree)
python.exec('import numpy as np')
python.exec('result = np.sin(np.deg2rad(degree))')
result <- python.get('result')
return(result)
}
Each time you run an R function, you will import an entire Python library.
As #Spacedman and #DirkEddelbuettel suggest you could add a .onLoad/.onAttach function to your package that calls python.exec to import the modules that will typically always be required by users of your package.
You could also test whether the module has already been imported before importing it, but (a) that gets you into a bit of a regression problem because you need to import sys in order to perform the test, (b) the answers to that question suggest that at least in terms of performance, it shouldn't matter, e.g.
If you want to optimize by not importing things twice, save yourself the hassle because Python already takes care of this.
(although admittedly there is some quibblingdiscussion elsewhere on that page about possible scenarios where there could be a performance cost).
But maybe your concern is stylistic rather than performance-oriented ...
How do I get the start/base address of a process? Per example Solitaire.exe (solitaire.exe+BAFA8)
#-*- coding: utf-8 -*-
import ctypes, win32ui, win32process
PROCESS_ALL_ACCESS = 0x1F0FFF
HWND = win32ui.FindWindow(None,u"Solitär").GetSafeHwnd()
PID = win32process.GetWindowThreadProcessId(HWND)[1]
PROCESS = ctypes.windll.kernel32.OpenProcess(PROCESS_ALL_ACCESS,False,PID)
print PID, HWND,PROCESS
I would like to calculate a memory address and for this way I need the base address of solitaire.exe.
Here's a picture of what I mean:
I think the handle returned by GetModuleHandle is actually the base address of the given module. You get the handle of the exe by passing NULL.
Install pydbg
Source: https://github.com/OpenRCE/pydbg
Unofficial binaries here: http://www.lfd.uci.edu/~gohlke/pythonlibs/#pydbg
from pydbg import *
from pydbg.defines import *
import struct
dbg = pydbg()
path_exe = "C:\\windows\\system32\\calc.exe"
dbg.load(path_exe, "-u amir")
dbg.debug_event_loop()
parameter_addr = dbg.context.Esp #(+ 0x8)
print 'ESP (address) ',parameter_addr
#attach not working under Win7 for me
#pid = raw_input("Enter PID:")
#print 'PID entered %i'%int(pid)
#dbg.attach(int(pid)) #attaching to running process not working
You might want to have a look at PaiMei, although it's not very active right now https://github.com/OpenRCE/paimei
I couldn't get attach() to work and used load instead. Pydbg has loads of functionality, such as read_proccess_memory, write_process_memory etc.
Note that you can't randomly change memory, because an operating system protects memory of other processes from your process (protected mode). Before the x86 processors there were some which allowed all processors to run in real mode, i.e. the full access of memory for every programm. Non-malicious software usually (always?) doesn't read/write other processes' memory.
The HMDOULE value of GetModuleHandle is the base address of the loaded module and is probably the address you need to compute the offset.
If not, that address is the start of the header of the module (DLL/EXE), which can be displayed with the dumpbin utility that comes with Visual Studio or you can interpret it yourself using the Microsoft PE and COFF Specification to determine the AddressOfEntryPoint and BaseOfCode as offsets from the base address. If the base address of the module isn't what you need, one of these two is another option.
Example:
>>> BaseAddress = win32api.GetModuleHandle(None) + 0xBAFA8
>>> print '{:08X}'.format(BaseAddress)
1D0BAFA8
If The AddressOfEntryPoint or BaseOfCode is needed, you'll have to use ctypes to call ReadProcessMemory following the PE specification to locate the offsets, or just use dumpbin /headers solitaire.exe to learn the offsets.
You can use frida to easy do that.
It is very useful to make hack and do some memory operation just like make address offset, read memory, write something to special memory etc...
https://github.com/frida/frida
2021.08.01 update:
Thanks for #Simas Joneliunas reminding
There some step using frida(windows):
Install frida by pip
pip install frida-tools # CLI tools
pip install frida # Python bindings
Using frida api
session = frida.attach(processName)
script = session.create_script("""yourScript""")
script.load()
sys.stdin.read() #make program always alive
session.detach()
Edit your scrip(using JavaScrip)
var baseAddr = Module.findBaseAddress('solitaire.exe');
var firstPointer = baseAddr.add(0xBAFA8).readPointer();
var secondPointer = firstPointer.add(0x50).readPointer();
var thirdPointer = secondPointer.add(0x14).readPointer();
#if your target pointer points to a Ansi String, you can use #thirdPointer.readAnsiString() to read
The official site https://frida.re/
I'm working on a python application that runs on 2 different platforms, namely regular desktop linux and Maemo 4. We use PyGTK on both platforms but on Maemo there are a bunch of little tweaks to make it look nice which are implemented as follows:
if util.platform.MAEMO:
# do something fancy for maemo
else:
# regular pygtk
There are roughly 15 of these if statements needed to get the UI looking and working nice on Maemo 4.
This has been very manageable for all this time. The problem is that a while ago there was a new version of Maemo released (5, aka fremantle) and it has some big differences compared to Maemo 4. I don't want to add a bunch of checks throughout the GUI code in order to get all 3 platforms working nicely with the same codebase because that would get messy. I also don't want to create a copy of the original GUI code for each platform and simply modify it for the specific platform (I'd like to re-use as much code as possible).
So, what are ways to have slightly different UIs for different platforms which are based on the same core UI code? I don't think this is a python or Maemo-specific question, I'd just like to know how this is done.
You could wind up much of this in a factory:
def createSpec():
if util.platform.MAEMO: return Maemo4Spec()
elif util.platform.MAEMO5: return Maemo5Spec()
return StandardPyGTKSpec()
Then, somewhere early in your code, you just call that factory:
spec = createSpec()
Now, everywhere else you had conditions, you just call the necessary function:
spec.drawComboBox()
As long as drawComboBox(), handles anything specific to the platform, you should be in good shape.
You could isolate the platform specific stuff you need to do into small consistently named functions inside a platform module, create the right function name using the platform you're running on and then getattr the right one and call it. The if/else boilerplate would disappear then.
I've made a separate module to handle all of my specializing between normal Linux, Maemo 4.1, and Maemo 5. It detects what features are available and allows the program to gracefully degrade.
For example
def _fremantle_hildonize_window(app, window):
oldWindow = window
newWindow = hildon.StackableWindow()
oldWindow.get_child().reparent(newWindow)
app.add_window(newWindow)
return newWindow
def _hildon_hildonize_window(app, window):
oldWindow = window
newWindow = hildon.Window()
oldWindow.get_child().reparent(newWindow)
app.add_window(newWindow)
return newWindow
def _null_hildonize_window(app, window):
return window
try:
hildon.StackableWindow
hildonize_window = _fremantle_hildonize_window
except AttributeError:
try:
hildon.Window
hildonize_window = _hildon_hildonize_window
except AttributeError:
hildonize_window = _null_hildonize_window
For more, see
Dialcentral, Gonert, ejpi, or Quicknote's source code for a file called hildonize.py
https://garage.maemo.org/plugins/ggit/browse.php/?p=gc-dialer;a=blob;f=src/hildonize.py;
Another example from The One Ring's GObject Utils (go_utils.py)
def _old_timeout_add_seconds(timeout, callback):
return gobject.timeout_add(timeout * 1000, callback)
def _timeout_add_seconds(timeout, callback):
return gobject.timeout_add_seconds(timeout, callback)
try:
gobject.timeout_add_seconds
timeout_add_seconds = _timeout_add_seconds
except AttributeError:
timeout_add_seconds = _old_timeout_add_seconds
Is there a way to dynamically call an Objective C function from Python?
For example, On the mac I would like to call this Objective C function
[NSSpeechSynthesizer availableVoices]
without having to precompile any special Python wrapper module.
As others have mentioned, PyObjC is the way to go. But, for completeness' sake, here's how you can do it with ctypes, in case you need it to work on versions of OS X prior to 10.5 that do not have PyObjC installed:
import ctypes
import ctypes.util
# Need to do this to load the NSSpeechSynthesizer class, which is in AppKit.framework
appkit = ctypes.cdll.LoadLibrary(ctypes.util.find_library('AppKit'))
objc = ctypes.cdll.LoadLibrary(ctypes.util.find_library('objc'))
objc.objc_getClass.restype = ctypes.c_void_p
objc.sel_registerName.restype = ctypes.c_void_p
objc.objc_msgSend.restype = ctypes.c_void_p
objc.objc_msgSend.argtypes = [ctypes.c_void_p, ctypes.c_void_p]
# Without this, it will still work, but it'll leak memory
NSAutoreleasePool = objc.objc_getClass('NSAutoreleasePool')
pool = objc.objc_msgSend(NSAutoreleasePool, objc.sel_registerName('alloc'))
pool = objc.objc_msgSend(pool, objc.sel_registerName('init'))
NSSpeechSynthesizer = objc.objc_getClass('NSSpeechSynthesizer')
availableVoices = objc.objc_msgSend(NSSpeechSynthesizer, objc.sel_registerName('availableVoices'))
count = objc.objc_msgSend(availableVoices, objc.sel_registerName('count'))
voiceNames = [
ctypes.string_at(
objc.objc_msgSend(
objc.objc_msgSend(availableVoices, objc.sel_registerName('objectAtIndex:'), i),
objc.sel_registerName('UTF8String')))
for i in range(count)]
print voiceNames
objc.objc_msgSend(pool, objc.sel_registerName('release'))
It ain't pretty, but it gets the job done. The final list of available names is stored in the voiceNames variable above.
2012-4-28 Update: Fixed to work in 64-bit Python builds by making sure all parameters and return types are passed as pointers instead of 32-bit integers.
Since OS X 10.5, OS X has shipped with the PyObjC bridge, a Python-Objective-C bridge. It uses the BridgeSupport framework to map Objective-C frameworks to Python. Unlike, MacRuby, PyObjC is a classical bridge--there is a proxy object on the python side for each ObjC object and visa versa. The bridge is pretty seamless, however, and its possible to write entire apps in PyObjC (Xcode has some basic PyObjC support, and you can download the app and file templates for Xcode from the PyObjC SVN at the above link). Many folks use it for utilities or for app-scripting/plugins. Apple's developer site also has an introduction to developing Cocoa applications with Python via PyObjC which is slightly out of date, but may be a good overview for you.
In your case, the following code will call [NSSpeechSynthesizer availableVoices]:
from AppKit import NSSpeechSynthesizer
NSSpeechSynthesizer.availableVoices()
which returns
(
"com.apple.speech.synthesis.voice.Agnes",
"com.apple.speech.synthesis.voice.Albert",
"com.apple.speech.synthesis.voice.Alex",
"com.apple.speech.synthesis.voice.BadNews",
"com.apple.speech.synthesis.voice.Bahh",
"com.apple.speech.synthesis.voice.Bells",
"com.apple.speech.synthesis.voice.Boing",
"com.apple.speech.synthesis.voice.Bruce",
"com.apple.speech.synthesis.voice.Bubbles",
"com.apple.speech.synthesis.voice.Cellos",
"com.apple.speech.synthesis.voice.Deranged",
"com.apple.speech.synthesis.voice.Fred",
"com.apple.speech.synthesis.voice.GoodNews",
"com.apple.speech.synthesis.voice.Hysterical",
"com.apple.speech.synthesis.voice.Junior",
"com.apple.speech.synthesis.voice.Kathy",
"com.apple.speech.synthesis.voice.Organ",
"com.apple.speech.synthesis.voice.Princess",
"com.apple.speech.synthesis.voice.Ralph",
"com.apple.speech.synthesis.voice.Trinoids",
"com.apple.speech.synthesis.voice.Vicki",
"com.apple.speech.synthesis.voice.Victoria",
"com.apple.speech.synthesis.voice.Whisper",
"com.apple.speech.synthesis.voice.Zarvox"
)
(a bridged NSCFArray) on my SL machine.
Mac OS X from 10.5 onward has shipped with Python and the objc module that will let you do what you want.
An example:
from Foundation import *
thing = NSKeyedUnarchiver.unarchiveObjectWithFile_(some_plist_file)
You can find more documentation here.
You probably want PyObjC. That said, I've never actually used it myself (I've only ever seen demos), so I'm not certain that it will do what you need.