We Keep Coding

torch.utils.ffi is deprecated. How to use cpp extensions instead?

I am trying to run code that is using wrap_function from torch.utils.ffi (which has deprecated). I am struggling to figure out how to use cpp extensions instead as suggested by the error message, can any anyone help please?
The code I need to replace:
from torch.utils.ffi import _wrap_function
from ._nms import lib as _lib, ffi as _ffi
__all__ = []
def _import_symbols(locals):
for symbol in dir(_lib):
fn = getattr(_lib, symbol)
if callable(fn):
locals[symbol] = _wrap_function(fn, _ffi)
else:
locals[symbol] = fn
__all__.append(symbol)
_import_symbols(locals())
I have tried running the code in both python 3.6 and python 2.7, see error message below,
File "build.py", line 3, in <module>
from torch.utils.ffi import create_extension
File "/usr/local/lib/python2.7/dist-packages/torch/utils/ffi/__init__.py", line 1, in <module>
raise ImportError("torch.utils.ffi is deprecated. Please use cpp extensions instead.")
ImportError: torch.utils.ffi is deprecated. Please use cpp extensions instead.

Check
from torch.utils.cpp import ....
#or
from torch.utils.cpp_extension import ....
#or
from torch.utils.cpp_extension_versioner import...
it looks good at my PC

Compile Python code with Openpyxl into executable using PyInstaller Error cannot import '__versions__' [duplicate]

My traceback from running pandas takes me to:
site-packages\pandas\io\excel.py line 58, in get_writer
AttributeError: 'module' object has no attribute '__version__'
I found this link to a git issue in the PyInstaller repo
https://github.com/pyinstaller/pyinstaller/issues/1890 and found my openpyxl version, manually added it into the get_writer method like so:
def get_writer(engine_name):
if engine_name == 'openpyxl':
try:
import openpyxl
#remove when conda update available
openpyxl.__version__ = '2.3.2'
# with version-less openpyxl engine
# make sure we make the intelligent choice for the user
if LooseVersion(openpyxl.__version__) < '2.0.0':
return _writers['openpyxl1']
elif LooseVersion(openpyxl.__version__) < '2.2.0':
return _writers['openpyxl20']
else:
return _writers['openpyxl22']
except ImportError:
# fall through to normal exception handling below
pass
try:
return _writers[engine_name]
except KeyError:
raise ValueError("No Excel writer '%s'" % engine_name)
Still no dice. The line number given in the error traceback doesn't even change. I then updated the openpyxl version to 2.3.5, still receiving the error. The openpyxl init file has a version variable in it:
try:
here = os.path.abspath(os.path.dirname(__file__))
src_file = os.path.join(here, ".constants.json")
with open(src_file) as src:
constants = json.load(src)
__author__ = constants['__author__']
__author_email__ = constants["__author_email__"]
__license__ = constants["__license__"]
__maintainer_email__ = constants["__maintainer_email__"]
__url__ = constants["__url__"]
__version__ = constants["__version__"]
except IOError:
# packaged
pass
Any known or potential fixes or workarounds?

Edits were not making an impact because the process was compiled into an exe that these modules were running through. Exported the sections I needed outside of my anaconda environment and now the process works without a hitch.

I will add my workaround to this discussion as I was having the same problem using openpyxl 2.4.0 and maybe a few others are stuck too.
I found that to create a .exe file you have to revert to an older version of openpyxl. To do so:
Open the command prompt and uninstall openpyxl with 'pip uninstall openpyxl'
Reinstall openpyxl using an older version 'pip install openpyxl==2.3.5'
Now you should be able to create your .exe file using py2exe, cx_freeze, etc.

This is my fix: go to your openpyxl site-package folder (for me it's: C:\Python27\Lib\site-packages\openpyxl). Copy all variables in your .constant.json file directly into the _ init _.py file, so it looks like:
import json
import os
__author__= "See AUTHORS"
__author_email__= "eric.gazoni#gmail.com"
__license__= "MIT/Expat",
__maintainer_email__= "openpyxl-users#googlegroups.com"
__url__= "http://openpyxl.readthedocs.org"
__version__= "2.4.0"
try:
here = os.path.abspath(os.path.dirname(__file__))

From *folder_name* import *variable* Python 3.4.2

File setup:
...\Project_Folder
...\Project_Folder\Project.py
...\Project_folder\Script\TestScript.py
I'm attempting to have Project.py import modules from the folder Script based on user input.
Python Version: 3.4.2
Ideally, the script would look something like
q = str(input("Input: "))
from Script import q
However, python does not recognize q as a variable when using import.
I've tried using importlib, however I cannot figure out how to import from the Script folder mentioned above.
import importlib
q = str(input("Input: "))
module = importlib.import_module(q, package=None)
I'm not certain where I would implement the file path.

Repeat of my answer originally posted at How to import a module given the full path?
as this is a Python 3.4 specific question:
This area of Python 3.4 seems to be extremely tortuous to understand, mainly because the documentation doesn't give good examples! This was my attempt using non-deprecated modules. It will import a module given the path to the .py file. I'm using it to load "plugins" at runtime.
def import_module_from_file(full_path_to_module):
"""
Import a module given the full path/filename of the .py file
Python 3.4
"""
module = None
try:
# Get module name and path from full path
module_dir, module_file = os.path.split(full_path_to_module)
module_name, module_ext = os.path.splitext(module_file)
# Get module "spec" from filename
spec = importlib.util.spec_from_file_location(module_name,full_path_to_module)
module = spec.loader.load_module()
except Exception as ec:
# Simple error printing
# Insert "sophisticated" stuff here
print(ec)
finally:
return module
# load module dynamically
path = "<enter your path here>"
module = import_module_from_file(path)
# Now use the module
# e.g. module.myFunction()

I did this by defining the entire import line as a string, formatting the string with q and then using the exec command:
imp = 'from Script import %s' %q
exec imp

keyring module is not included while packaging with py2exe

I am making an app using python 2.7 on windows and keyring-3.2.1 . In my python code on eclipse, I used
import keyring
keyring.set_password("service","jsonkey",json_res)
json_res= keyring.get_password("service","jsonkey")
is working fine as I am storing json response in keyring. But, when I converted python code into exe by using py2exe, it shows import error keyring while making dist. Please suggest how to include keyring in py2exe.
Traceback (most recent call last):
File "APP.py", line 8, in <module>
File "keyring\__init__.pyc", line 12, in <module>
File "keyring\core.pyc", line 15, in <module>
File "keyring\util\platform_.pyc", line 4, in <module>
File "keyring\util\platform.pyc", line 29, in <module>
AttributeError: 'module' object has no attribute 'system'
platform_.py code is :
from __future__ import absolute_import
import os
import platform
def _data_root_Windows():
try:
root = os.environ['LOCALAPPDATA']
except KeyError:
# Windows XP
root = os.path.join(os.environ['USERPROFILE'], 'Local Settings')
return os.path.join(root, 'Python Keyring')
def _data_root_Linux():
"""
Use freedesktop.org Base Dir Specfication to determine storage
location.
"""
fallback = os.path.expanduser('~/.local/share')
root = os.environ.get('XDG_DATA_HOME', None) or fallback
return os.path.join(root, 'python_keyring')
# by default, use Unix convention
data_root = globals().get('_data_root_' + platform.system(), _data_root_Linux)
platform.py code is:
import os
import sys
# While we support Python 2.4, use a convoluted technique to import
# platform from the stdlib.
# With Python 2.5 or later, just do "from __future__ import absolute_import"
# and "import platform"
exec('__import__("platform", globals=dict())')
platform = sys.modules['platform']
def _data_root_Windows():
try:
root = os.environ['LOCALAPPDATA']
except KeyError:
# Windows XP
root = os.path.join(os.environ['USERPROFILE'], 'Local Settings')
return os.path.join(root, 'Python Keyring')
def _data_root_Linux():
"""
Use freedesktop.org Base Dir Specfication to determine storage
location.
"""
fallback = os.path.expanduser('~/.local/share')
root = os.environ.get('XDG_DATA_HOME', None) or fallback
return os.path.join(root, 'python_keyring')
# by default, use Unix convention
data_root = globals().get('_data_root_' + platform.system(), _data_root_Linux)

The issue you're reporting is due to an environment that contains invalid modules, perhaps from an improper installation of one version of keyring over another. You will want to ensure that you've removed remnants of the older version of keyring. In particular, make sure there's no file called keyring\util\platform.* in your site-packages.
After doing that, however, you'll encounter another problem. Keyring loads its backend modules programmatically, so py2exe won't detect them.
To work around that, you'll want to add a 'packages' declaration to your py2exe options to specifically include the keyring.backends package. I invoked the following setup.py script with Python 2.7 to convert 'app.py' (which imports keyring) to an exe:
from distutils.core import setup
import py2exe
setup(
console=['app.py'],
options=dict(py2exe=dict(
packages='keyring.backends',
)),
)
The resulting app.exe will import and invoke keyring.

Packaging a Python library

I have a few Munin plugins which report stats from an Autonomy database. They all use a small library which scrapes the XML status output for the relevant numbers.
I'm trying to bundle the library and plugins into a Puppet-installable RPM. The actual RPM-building should be straightforward; once I have a distutils-produced distfile I can make it into an RPM based on a .spec file pinched from the Dag or EPEL repos [1]. It's the distutils bit I'm unsure of -- in fact I'm not even sure my library is correctly written for packaging. Here's how it works:
idol7stats.py:
import datetime
import os
import stat
import sys
import time
import urllib
import xml.sax
class IDOL7Stats:
cache_dir = '/tmp'
def __init__(self, host, port):
self.host = host
self.port = port
# ...
def collect(self):
self.data = self.__parseXML(self.__getXML())
def total_slots(self):
return self.data['Service:Documents:TotalSlots']
Plugin code:
from idol7stats import IDOL7Stats
a = IDOL7Stats('db.example.com', 23113)
a.collect()
print a.total_slots()
I guess I want idol7stats.py to wind up in /usr/lib/python2.4/site-packages/idol7stats, or something else in Python's search path. What distutils magic do I need? This:
from distutils.core import setup
setup(name = 'idol7stats',
author = 'Me',
author_email = 'me#example.com',
version = '0.1',
py_modules = ['idol7stats'])
almost works, except the code goes in /usr/lib/python2.4/site-packages/idol7stats.py, not a subdirectory. I expect this is down to my not understanding the difference between modules/packages/other containers in Python.
So, what's the rub?
[1] Yeah, I could just plonk the library in /usr/lib/python2.4/site-packages using RPM but I want to know how to package Python code.

You need to create a package to do what you want. You'd need a directory named idol7stats containing a file called __init__.py and any other library modules to package. Also, this will affect your scripts' imports; if you put idol7stats.py in a package called idol7stats, then your scripts need to "import idol7stats.idol7stats".
To avoid that, you could just rename idol7stats.py to idol7stats/__init__.py, or you could put this line into idol7stats/__init__.py to "massage" the imports into the way you expect them:
from idol7stats.idol7stats import *