I have a GUI program that I have developed in Python 3.2 to extract various geospatial data products. I need to call a module I have developed in Python 2.7.
I am looking for a way to be able to call the Python 2.7 code using a Python 2.7 interpreter inside of Python 3.2 program. I cannot port the 2.7 to Python 3.2 as it uses a Python version installed with ESRI ArcMap and relies on the arcpy module which is not available for Python 3. My only idea right now would be to use subprocess to call the module as a batch process however this is a bit messy and I would prefer the two programs had some relationship.
Thank you in advance.
You could spawn the python 2.7 process as a server handling RPC requests from your GUI running on 3.2. That will work either over the network, or local pipes, or shared memory, or your system's message bus, or many other ways. You just need to translate your library's API into some kind of serialised messages.
Let's say your library has a function: (super simplified example)
def add(a, b):
return a+b
You'd wrap this in some server, let's say a flask app, which does:
#app.route("/add", methods=["POST"])
def handle_add():
data = request.get_json()
ret = your_lib.add(data['a'], data['b'])
return jsonify(ret)
and on the client side, send and unpack the values using something like requests
You could even make it fairly transparent by implementing a translator module with methods named the same as the library itself and doing import your_http_wrapper as your_library_name.
The trick now is to make sure all your parameters can be serialised and that you can realistically send all the arguments/return values in a reasonable time on each call. Also, you lose the ability to change the contents of the variables you pass to the wrapper, because the server will modify only the local copy (unless you implement serialising all those modifications as well)
Try using subprocess.check_output(['C:\\python27\\python.exe', 'yourModule.py'])
If you want to call a specific function within the 27 file, you can use more system arguments. The call would look like:
subprocess.check_output(['C:\\python27\\python.exe', 'yourModule.py', 'funcName'])
And in the 27 file you can add:
import sys
if __name__=='__main__':
if 'funcName' in sys.argv:
funcName()
else:
#... execute normally
Somewhat late but in case someone stumbles over this thread:
I've written a module that transparently runs parts of a program in other python interpreters. Basically it provides decorators and base classes that replace functions and objects with proxies that interface with other interpreters.
It's intended for compatibility, e.g. running python2 only code in python3, or accessing C modules from pypy.
In the following example, the loop runs 4x as fast via using pypy as it would with plain python.
#!/usr/local/bin/python
from cpy2py import TwinMaster, twinfunction
import sys
import time
import math
# loops in PyPy
#twinfunction('pypy')
def prime_sieve(max_val):
start_time = time.time()
primes = [1] * 2 + [0] * (max_val - 1)
for value, factors in enumerate(primes):
if factors == 0:
for multiple in xrange(value*value, max_val + 1, value):
primes[multiple] += 1
return {'xy': [
[primes[idx] == 0 for idx in range(minidx, minidx + int(math.sqrt(max_val)))]
for minidx in range(0, max_val, int(math.sqrt(max_val)))
], 'info': '%s in %.1fs' % (sys.executable, time.time() - start_time)}
# matplotlib in CPython
#twinfunction('python')
def draw(xy, info='<None>'):
from matplotlib import pyplot
pyplot.copper()
pyplot.matshow(xy)
pyplot.xlabel(info, color="red")
pyplot.show()
if __name__ == '__main__':
twins = [TwinMaster('python'), TwinMaster('pypy')]
for twin in twins:
twin.start()
data = prime_sieve(int(1E6))
draw(**data)
I'm trying to add cross-references to external API into my documentation but I'm facing three different behaviors.
I am using sphinx(1.3.1) with Python(2.7.3) and my intersphinx mapping is configured as:
{
'python': ('https://docs.python.org/2.7', None),
'numpy': ('http://docs.scipy.org/doc/numpy/', None),
'cv2' : ('http://docs.opencv.org/2.4/', None),
'h5py' : ('http://docs.h5py.org/en/latest/', None)
}
I have no trouble writing a cross-reference to numpy API with :class:`numpy.ndarray` or :func:`numpy.array` which gives me, as expected, something like numpy.ndarray.
However, with h5py, the only way I can have a link generated is if I omit the module name. For example, :class:`Group` (or :class:`h5py:Group`) gives me Group but :class:`h5py.Group` fails to generate a link.
Finally, I cannot find a way to write a working cross-reference to OpenCV API, none of these seems to work:
:func:`cv2.convertScaleAbs`
:func:`cv2:cv2.convertScaleAbs`
:func:`cv2:convertScaleAbs`
:func:`convertScaleAbs`
How to properly write cross-references to external API, or configure intersphinx, to have a generated link as in the numpy case?
In addition to the detailed answer from #gall, I've discovered that intersphinx can also be run as a module:
python -m sphinx.ext.intersphinx 'http://python-eve.org/objects.inv'
This outputs nicely formatted info. For reference: https://github.com/sphinx-doc/sphinx/blob/master/sphinx/ext/intersphinx.py#L390
I gave another try on trying to understand the content of an objects.inv file and hopefully this time I inspected numpy and h5py instead of only OpenCV's one.
How to read an intersphinx inventory file
Despite the fact that I couldn't find anything useful about reading the content of an object.inv file, it is actually very simple with the intersphinx module.
from sphinx.ext import intersphinx
import warnings
def fetch_inventory(uri):
"""Read a Sphinx inventory file into a dictionary."""
class MockConfig(object):
intersphinx_timeout = None # type: int
tls_verify = False
class MockApp(object):
srcdir = ''
config = MockConfig()
def warn(self, msg):
warnings.warn(msg)
return intersphinx.fetch_inventory(MockApp(), '', uri)
uri = 'http://docs.python.org/2.7/objects.inv'
# Read inventory into a dictionary
inv = fetch_inventory(uri)
# Or just print it
intersphinx.debug(['', uri])
File structure (numpy)
After inspecting numpy's one, you can see that keys are domains:
[u'np-c:function',
u'std:label',
u'c:member',
u'np:classmethod',
u'np:data',
u'py:class',
u'np-c:member',
u'c:var',
u'np:class',
u'np:function',
u'py:module',
u'np-c:macro',
u'np:exception',
u'py:method',
u'np:method',
u'np-c:var',
u'py:exception',
u'np:staticmethod',
u'py:staticmethod',
u'c:type',
u'np-c:type',
u'c:macro',
u'c:function',
u'np:module',
u'py:data',
u'np:attribute',
u'std:term',
u'py:function',
u'py:classmethod',
u'py:attribute']
You can see how you can write your cross-reference when you look at the content of a specific domain. For example, py:class:
{u'numpy.DataSource': (u'NumPy',
u'1.9',
u'http://docs.scipy.org/doc/numpy/reference/generated/numpy.DataSource.html#numpy.DataSource',
u'-'),
u'numpy.MachAr': (u'NumPy',
u'1.9',
u'http://docs.scipy.org/doc/numpy/reference/generated/numpy.MachAr.html#numpy.MachAr',
u'-'),
u'numpy.broadcast': (u'NumPy',
u'1.9',
u'http://docs.scipy.org/doc/numpy/reference/generated/numpy.broadcast.html#numpy.broadcast',
u'-'),
...}
So here, :class:`numpy.DataSource` will work as expected.
h5py
In the case of h5py, the domains are:
[u'py:attribute', u'std:label', u'py:method', u'py:function', u'py:class']
and if you look at the py:class domain:
{u'AttributeManager': (u'h5py',
u'2.5',
u'http://docs.h5py.org/en/latest/high/attr.html#AttributeManager',
u'-'),
u'Dataset': (u'h5py',
u'2.5',
u'http://docs.h5py.org/en/latest/high/dataset.html#Dataset',
u'-'),
u'ExternalLink': (u'h5py',
u'2.5',
u'http://docs.h5py.org/en/latest/high/group.html#ExternalLink',
u'-'),
...}
That's why I couldn't make it work as numpy references. So a good way to format them would be :class:`h5py:Dataset`.
OpenCV
OpenCV's inventory object seems malformed. Where I would expect to find domains there is actually 902 function signatures:
[u':',
u'AdjusterAdapter::create(const',
u'AdjusterAdapter::good()',
u'AdjusterAdapter::tooFew(int',
u'AdjusterAdapter::tooMany(int',
u'Algorithm::create(const',
u'Algorithm::getList(vector<string>&',
u'Algorithm::name()',
u'Algorithm::read(const',
u'Algorithm::set(const'
...]
and if we take the first one's value:
{u'Ptr<AdjusterAdapter>': (u'OpenCV',
u'2.4',
u'http://docs.opencv.org/2.4/detectorType)',
u'ocv:function 1 modules/features2d/doc/common_interfaces_of_feature_detectors.html#$ -')}
I'm pretty sure it is then impossible to write OpenCV cross-references with this file...
Conclusion
I thought intersphinx generated the objects.inv based on the content of the documentation project in an standard way, which seems not to be the case.
As a result, it seems that the proper way to write cross-references is API dependent and one should inspect a specific inventory object to actually see what's available.
An additional way to inspect the objects.inv file is with the sphobjinv module.
You can search local or even remote inventory files (with fuzzy matching). For instance with scipy:
$ sphobjinv suggest -t 90 -u https://docs.scipy.org/doc/scipy/reference/objects.inv "signal.convolve2d"
Remote inventory found.
:py:function:`scipy.signal.convolve2d`
:std:doc:`generated/scipy.signal.convolve2d`
Note that you may need to use :py:func: and not :py:function: (I'd be happy to know why).
How to use OpenCV 2.4 (cv2) intersphinx
Inspired by #Gall's answer, I wanted to compare the contents of the OpenCV & numpy inventory files. I couldn't get sphinx.ext.intersphinx.fetch_inventory to work from ipython, but the following does work:
curl http://docs.opencv.org/2.4/objects.inv | tail -n +5 | zlib-flate -uncompress > cv2.inv
curl https://docs.scipy.org/doc/numpy/objects.inv | tail -n +5 | zlib-flate -uncompress > numpy.inv
numpy.inv has lines like this:
numpy.ndarray py:class 1 reference/generated/numpy.ndarray.html#$ -
whereas cv2.inv has lines like this:
cv2.imread ocv:pyfunction 1 modules/highgui/doc/reading_and_writing_images_and_video.html#$ -
So presumably you'd link to the OpenCV docs with :ocv:pyfunction:`cv2.imread` instead of :py:function:`cv2.imread`. Sphinx doesn't like it though:
WARNING: Unknown interpreted text role "ocv:pyfunction".
A bit of Googling revealed that the OpenCV project has its own "ocv" sphinx domain: https://github.com/opencv/opencv/blob/2.4/doc/ocv.py -- presumably because they need to document C, C++ and Python APIs all at the same time.
To use it, save ocv.py next to your Sphinx conf.py, and modify your conf.py:
sys.path.insert(0, os.path.abspath('.'))
import ocv
extensions = [
'ocv',
]
intersphinx_mapping = {
'cv2': ('http://docs.opencv.org/2.4/', None),
}
In your rst files you need to say :ocv:pyfunc:`cv2.imread` (not :ocv:pyfunction:).
Sphinx prints some warnings (unparseable C++ definition: u'cv2.imread') but the generated html documentation actually looks ok with a link to http://docs.opencv.org/2.4/modules/highgui/doc/reading_and_writing_images_and_video.html#cv2.imread. You can edit ocv.py and remove the line that prints that warning.
The accepted answer no longer works in the new version (1.5.x) ...
import requests
import posixpath
from sphinx.ext.intersphinx import read_inventory
uri = 'http://docs.python.org/2.7/'
r = requests.get(uri + 'objects.inv', stream=True)
r.raise_for_status()
inv = read_inventory(r.raw, uri, posixpath.join)
Stubborn fool that I am, I used 2to3 and the Sphinx deprecated APIs chart to revive #david-röthlisberger's ocv.py-based answer so it'll work with Sphinx 2.3 on Python 3.5.
The fixed-up version is here:
https://gist.github.com/ssokolow/a230b27b7ea4a31f7fb40621e6461f9a
...and the quick version of what I did was:
Run 2to3 -w ocv.py && rm ocv.py.bak
Cycle back and forth between running Sphinx and renaming functions to their replacements in the chart. I believe these were the only changes I had to make on this step:
Directive now has to be imported from docutils.parsers.rst
Replace calls to l_(...) with calls to _(...) and remove the l_ import.
Replace calls to env.warn with calls to log.warn where log = sphinx.util.logging.getLogger(__name__).
Then, you just pair it with this intersphinx definition and you get something still new enough to be relevant for most use cases:
'cv2': ('https://docs.opencv.org/3.0-last-rst/', None)
For convenience, I made a small extension for aliasing intersphinx cross references. This is useful as sometimes the object inventory gets confused when an object from a submodule is imported from a package's __init__.py.
See also https://github.com/sphinx-doc/sphinx/issues/5603
###
# Workaround of
# Intersphinx references to objects imported at package level can"t be mapped.
#
# See https://github.com/sphinx-doc/sphinx/issues/5603
intersphinx_aliases = {
("py:class", "click.core.Group"):
("py:class", "click.Group"),
("py:class", "click.core.Command"):
("py:class", "click.Command"),
}
def add_intersphinx_aliases_to_inv(app):
from sphinx.ext.intersphinx import InventoryAdapter
inventories = InventoryAdapter(app.builder.env)
for alias, target in app.config.intersphinx_aliases.items():
alias_domain, alias_name = alias
target_domain, target_name = target
try:
found = inventories.main_inventory[target_domain][target_name]
try:
inventories.main_inventory[alias_domain][alias_name] = found
except KeyError:
print("could not add to inv")
continue
except KeyError:
print("missed :(")
continue
def setup(app):
app.add_config_value("intersphinx_aliases", {}, "env")
app.connect("builder-inited", add_intersphinx_aliases_to_inv)
To use this, I paste the above code in my conf.py and add aliases to the intersphinx_aliases dictionary.
I'm sure this is documented somewhere but i can't find it...
My code is getting a python object from another library (that i can't modify), and i need to call some win32 api functions on it.
Python returns something that isn't the os-level handle from file.fileno(), my guess is that it gives MSVCRT's fileno.
>>> ctypes.windll.kernel32.CreateFileA('test',0x80000000L,1,None,3,0,0)
1948 # <- HANDLE
>>> file('test','r').fileno()
4 # <- not a HANDLE
How do i convert it into a real win32 handle?
I found the answer:
>>> msvcrt.get_osfhandle(a.fileno())
1956 # valid HANDLE
This is actually documented on http://docs.python.org/library/msvcrt.html , no idea how i missed it.
win32file._get_osfhandle from the PyWin32 library will return what you want.
win32file._get_osfhandle(a.fileno()) returns the same thing as msvcrt.get_osfhandle(a.fileno()) in my testing.
I am writing a function which prints out detailed Windows Version informations, the output may be a tuple like this:
('32bit', 'XP', 'Professional', 'SP3', 'English')
It will be supporting Windows XP and above. And I'm stuck with getting the Windows edition, e.g., "Professional", "Home Basic", etc.
platform.win32_ver() or sys.getwindowsversion() doesn't do it for me.
win32api.GetVersionEx(1) almost hits, but looks like it doesn't tell me enough information.
Then I saw GetProductInfo(), but looks like it's not implemented in pywin32.
Any hints?
You can use ctypes to access any WinAPI function. GetProductInfo() is in windll.kernel32.GetProductInfo.
I'd found a Python version (GPL licensed, but you can see usage of the functions there) of the MSDN "Getting the System Version" example.
If ctypes doesn't work (due to 32 vs 64 bits?), this hack should:
def get_Windows_name():
import subprocess, re
o = subprocess.Popen('systeminfo', stdout=subprocess.PIPE).communicate()[0]
try: o = str(o, "latin-1") # Python 3+
except: pass
return re.search("OS Name:\s*(.*)", o).group(1).strip()
print(get_Windows_name())
Or just read the registry:
try: import winreg
except: import _winreg as winreg
with winreg.OpenKey(winreg.HKEY_LOCAL_MACHINE, r"SOFTWARE\Microsoft\Windows NT\CurrentVersion") as key:
print(winreg.QueryValueEx(key, "EditionID")[0])
Or use this:
from win32com.client import GetObject
wim = GetObject('winmgmts:')
print([o.Caption for o in wim.ExecQuery("Select * from Win32_OperatingSystem")][0])
I tried a few of the solutions above, but I was looking for something that gave me "Windows XP" or "Windows 7". There are a few more methods in platform that expose even more information.
import platform
print platform.system(),platform.release()
I'm relatively new to programming/python, so I'd appreciate any help I can get. I want to save an excel file as a specific format using Excel through COM. Here is the code:
import win32com.client as win32
def excel():
app = 'Excel'
x1 = win32.gencache.EnsureDispatch('%s.Application' % app)
ss = x1.Workbooks.Add()
sh = ss.ActiveSheet
x1.Visible = True
sh.Cells(1,1).Value = 'test write'
ss.SaveAs(Filename="temp.xls", FileFormat=56)
x1.Application.Quit()
if __name__=='__main__':
excel()
My question is how do I specify the FileFormat if I don't explicitly know the code for it? Browsing through the documentation I find the reference at about a FileFormat object. I'm clueless on how to access the XlFileFormat object and import it in a way that I can find the enumeration value for it.
Thanks!
This question is a bit stale, but for those reaching this page from Google (as I did) my solution was accessing the constants via the win32com.client.constants object instead of on the application object itself as suggested by Eric. This lets you use enum constants just like in the VBE:
>>> import win32com.client
>>> xl = win32com.client.gencache.EnsureDispatch('Excel.Application')
>>> C = win32com.client.constants
>>> C.xlWorkbookNormal
-4143
>>> C.xlCSV
6
>>> C.xlErrValue
2015
>>> C.xlThemeColorAccent1
5
Also, unless you've manually run the makepy utility, the constants may not be available if initializing the application with the regular win32com.client.Dispatch(..) method, which was another issue I was having. Using win32com.client.gencache.EnsureDispatch(..) (as the questioner does) checks for and generates the Python bindings at runtime if required.
I found this ActiveState page to be helpful.
When I used COM to access quickbooks, I could reach the constants defined under a constants member of the object. The code looked something like this (you'll be intersted in the third line):
self._session_manager.OpenConnection2("",
application_name,
QBFC8Lib.constants.ctLocalQBD)
I'm not sure if this will work, but try this:
import win32com.client as win32
def excel():
app = 'Excel'
x1 = win32.gencache.EnsureDispatch('%s.Application' % app)
ss = x1.Workbooks.Add()
sh = ss.ActiveSheet
x1.Visible = True
sh.Cells(1,1).Value = 'test write'
ss.SaveAs(Filename="temp.xls", FileFormat=x1.constants.xlWorkbookNormal)
x1.Application.Quit()
if __name__=='__main__':
excel()
Replace xlWorkbookNormal with whatever format your trying to choose in the X1FileFormat web page you posted in your question.
All of the file format constants are documented here
As a general rule I find it really useful to pre-record any code in the VBA IDE in Excel. This way you can find out all the values of constants etc that you need to use within your python code. You can also make sure stuff will work from within a more controlled environment.