We Keep Coding

NameError: name 'sv' is not defined while importing python module

I am facing a problem in python. Tho the error is quite common, but since i am bit new to python, unable to comprehend the source hence asking you all. There are 2 modules: session.py and objects.py.
session.py
import copy
import pymongo
import spacy
import tweepy
import objects
objects.py:
import re
def refresh (sv = sv, obj = ''):
return 0
now, in python shell, i am getting the error before even executing objects.py:
$ python
Python 2.7.13 (default, Sep 26 2018, 18:42:22)
[GCC 6.3.0 20170516] on linux2
Type "help", "copyright", "credits" or "license" for more information.
>>> import session
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
File "session.py", line 6, in <module>
import objects
File "objects.py", line 3, in <module>
def refresh (sv = sv, obj = ''):
NameError: name 'sv' is not defined
>>>
I came from perl background to maybe missing some very common thing, but still i am able to do this:
>>> def ff(t): print t
...
In above, whitout defining t, it is working while in objects.py, how can i define sv without starting execution?

Windows / ctypes issue with custom build

EDIT (begin)
After spending time with #eryksun we dug really deep and found the underlying issue. These two examples fail1.py and fail2.py have the same effect as the original lengthier example I posted.
It turns out the issue has to do with copying 4-byte C ints into an 8-byte stack location without zero'ing out the memory first, potentially leaving garbage in the upper 4 bytes. This was confirmed using a debugger (again props to #eryksun).
This was one of those weird bugs (heisenbug) where you can add some printf statements then the bug doesn't exist any more.
the fix
At the top ffi_prep_args on the line before argp = stack;, add a call
to memset(stack, 0, ecif->cif->bytes);. This will zero the entire
stack.
This is the location to fix:
https://github.com/python/cpython/blob/v2.7.13/Modules/_ctypes/libffi_msvc/ffi.c#L47
fail1.py
import ctypes
kernel32 = ctypes.WinDLL('kernel32')
hStdout = kernel32.GetStdHandle(-11)
written = ctypes.c_ulong()
kernel32.WriteFile(hStdout, b'spam\n', 5, ctypes.byref(written), None)
fail2.py
import ctypes
import msvcrt
import sys
kernel32 = ctypes.WinDLL('kernel32')
hStdout = msvcrt.get_osfhandle(sys.stdout.fileno())
written = ctypes.c_ulong()
kernel32.WriteFile(hStdout, b'spam\n', 5, ctypes.byref(written), None)
EDIT (end)
I built my own Python 2.7.13 for Windows because I'm using bindings I created to a 3rd party application which must be built with a specific build of Visual Studio 2012.
I started to code up some stuff using the "click" module and found an error when using click.echo with any kind of unicode echo.
Python 2.7.13 (default, Mar 27 2017, 11:11:01) [MSC v.1700 64 bit (AMD64)] on win32
Type "help", "copyright", "credits" or "license" for more information.
>>> import click
>>> click.echo('Hello World')
Hello World
>>> click.echo(u'Hello World')
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
File "C:\Users\eric\my_env\lib\site-packages\click\utils.py", line 259, in echo
file.write(message)
File "C:\Users\eric\my_env\lib\site-packages\click\_winconsole.py", line 180, in write
return self._text_stream.write(x)
File "C:\Users\eric\my_env\lib\site-packages\click\_compat.py", line 63, in write
return io.TextIOWrapper.write(self, x)
File "C:\Users\eric\my_env\lib\site-packages\click\_winconsole.py", line 164, in write
raise OSError(self._get_error_message(GetLastError()))
OSError: Windows error 6
If I download and install the Python 2.7.13 64-bit installer I don't get this issue. It echo's just fine.
I have looked into this a lot and am at a loss right now.
I'm not too familiar with Windows, Visual Studio, or ctypes.
I spent some time looking at the code path to produce the smallest file (without click) which demonstrates this problem (see below)
It produces the same "Windows error 6"... again, this works fine with the python installed from the 2.7.13 64 bit MSI installer.
Can someone share the process used to create the Windows installers? Is this a manual process or is it automated?
Maybe I'm missing some important switch to msbuild or something. Any help or ideas are appreciated.
I cannot use a downloaded copy of Python... it needs to be built with a specific version, update, patch, etc of Visual Studio.
All I did was
clone cpython from github and checkout 2.7.13
edit some xp stuff out of tk stuff to get it to compile on Windows Server 2003
In externals\tk-8.5.15.0\win\Makefile.in remove ttkWinXPTheme.$(OBJEXT) line
In externals\tk-8.5.15.0\win\makefile.vc remove $(TMP_DIR)\ttkWinXPTheme.obj line
In externals\tk-8.5.15.0\win\ttkWinMonitor.c remove 2 TtkXPTheme_Init lines
In PCbuild\tcltk.props change VC9 to VC11 at the bottom
PCbuild\build.bat -e -p x64 "/p:PlatformToolset=v110"
After that I created an "install" by copying .exe, .pyd, .dll files, ran get-pip.py, then python -m pip install virtualenv, then virtualenv my_env, then activated it, then did a pip install click.
But with this stripped down version you don't need pip, virtualenv or click... just ctypes.
You could probably even build it without the -e switch to build.bat.
from ctypes import byref, POINTER, py_object, pythonapi, Structure, windll
from ctypes import c_char, c_char_p, c_int, c_ssize_t, c_ulong, c_void_p
c_ssize_p = POINTER(c_ssize_t)
kernel32 = windll.kernel32
STDOUT_HANDLE = kernel32.GetStdHandle(-11)
PyBUF_SIMPLE = 0
MAX_BYTES_WRITTEN = 32767
class Py_buffer(Structure):
_fields_ = [
('buf', c_void_p),
('obj', py_object),
('len', c_ssize_t),
('itemsize', c_ssize_t),
('readonly', c_int),
('ndim', c_int),
('format', c_char_p),
('shape', c_ssize_p),
('strides', c_ssize_p),
('suboffsets', c_ssize_p),
('internal', c_void_p)
]
_fields_.insert(-1, ('smalltable', c_ssize_t * 2))
bites = u"Hello World".encode('utf-16-le')
bytes_to_be_written = len(bites)
buf = Py_buffer()
pythonapi.PyObject_GetBuffer(py_object(bites), byref(buf), PyBUF_SIMPLE)
buffer_type = c_char * buf.len
buf = buffer_type.from_address(buf.buf)
code_units_to_be_written = min(bytes_to_be_written, MAX_BYTES_WRITTEN) // 2
code_units_written = c_ulong()
kernel32.WriteConsoleW(STDOUT_HANDLE, buf, code_units_to_be_written, byref(code_units_written), None)
bytes_written = 2 * code_units_written.value
if bytes_written == 0 and bytes_to_be_written > 0:
raise OSError('Windows error %s' % kernel32.GetLastError())

How to store the results of a command to a variable?

I'm trying to build a context menu that will show a tree structure of all my packages and their resources. Something like this terrible MS Paint rendering I drew:
I have PackageResourceViewer installed. It offers commands that show in a window. But I would like to use them to populate these sub-context menues:
get_packages_list that I think will populate the first sub-menu (packages)
list_package_files that should populate each subsequent sub-menu
However, I'm not sure how to get the output of these into a context menu. I've been looking at python (writing a sublime plugin for this).
How do I get the output of these commands into a variable?
My Code
import sublime
import sublime_plugin
class ExampleCommand(sublime_plugin.TextCommand):
def run(self, edit):
self.view.insert(edit, 0, package_resource_viewer)
But it gives me this error:
>>> view.window().run_command("example")
Traceback (most recent call last):
File "C:\Program Files\Sublime Text 3\sublime_plugin.py", line 818, in run_
return self.run(edit)
File "C:\Users\heete\AppData\Roaming\Sublime Text 3\Packages\User\hello_world.py", line 7, in run
self.view.insert(edit, 0, package_resource_viewer)
NameError: global name 'package_resource_viewer' is not defined
Obviously I haven't gotten far, but I can't even get the output of this to show in the buffer.

You can import and access those commands. I made a minimal example, which prints the packages to the current view.
However output the to a context/sidebar menu seems to be harder, because they are usually static and you cannot just create one dynamically. (You may try to create a static menu file via a command.)
import sublime_plugin
from PackageResourceViewer.package_resources import (
get_packages_list, list_package_files
)
class ExampleListPackagesCommand(sublime_plugin.TextCommand):
def run(self, edit):
packages = get_packages_list()
insert_sb = []
insert_sb.append("Installed Packages:")
insert_sb.extend(packages)
for package in packages:
insert_sb.append("")
insert_sb.append(package)
package_files = list_package_files(package)
insert_sb.extend("\t" + pf for pf in package_files)
self.view.insert(edit, 0, "\n".join(insert_sb))

Python os.environ throws key error?

I'm accessing an environment variable in a script with os.environ.get and it's throwing a KeyError. It doesn't throw the error from the Python prompt. This is running on OS X 10.11.6, and is Python 2.7.10.
What is going on?
$ python score.py
Traceback (most recent call last):
File "score.py", line 4, in <module>
setup_logging()
File "/score/log.py", line 29, in setup_logging
config = get_config()
File "/score/log.py", line 11, in get_config
environment = os.environ.get('NODE_ENV')
File "/System/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/UserDict.py", line 23, in __getitem__
raise KeyError(key)
KeyError: 'NODE_ENV'
$ python -c "import os; os.environ.get('NODE_ENV')"
$
As requested, here's the source code for score.py
from __future__ import print_function
from log import get_logger, setup_logging
setup_logging()
log = get_logger('score')
And here's log.py
import json
import os
import sys
from iron_worker import IronWorker
from logbook import Logger, Processor, NestedSetup, StderrHandler, SyslogHandler
IRON_IO_TASK_ID = IronWorker.task_id()
def get_config():
environment = os.environ.get('NODE_ENV')
if environment == 'production':
filename = '../config/config-production.json'
elif environment == 'integration':
filename = '../config/config-integration.json'
else:
filename = '../config/config-dev.json'
with open(filename) as f:
return json.load(f)
def setup_logging():
# This defines a remote Syslog handler
# This will include the TASK ID, if defined
app_name = 'scoreworker'
if IRON_IO_TASK_ID:
app_name += '-' + IRON_IO_TASK_ID
config = get_config()
default_log_handler = NestedSetup([
StderrHandler(),
SyslogHandler(
app_name,
address = (config['host'], config['port']),
level = 'ERROR',
bubble = True
)
])
default_log_handler.push_application()
def get_logger(name):
return Logger(name)

Try running:
find . -name \*.pyc -delete
To delete your .pyc files.
Researching your problem I came across this question, where a user was experiencing the same thing: .get() seemingly raising a KeyError. In that case, it was caused, according to this accepted answer, by a .pyc file which contained code where a dict value was being accessed by key (i.e., mydict['potentially_nonexistent_key']), while the traceback was showing the code from the updated .py file where .get() was used. I have never heard of this happening, where the traceback references current code from a .py file, but shows an error raised by an outdated .pyc file, but it seems to have happened at least once in the history of Python...
It is a long shot, but worth a try I thought.

I encountered a similar error when I set the environment variable without exporting it. So if you do this:
me#host:/# NODE_ENV=foo
You will get this:
me#host:/# python3
Python 3.8.2 (default, Apr 27 2020, 15:53:34)
[GCC 9.3.0] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> import os
>>> node_env = os.environ['NODE_ENV']
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
File "/usr/lib/python3.8/os.py", line 675, in __getitem__
raise KeyError(key) from None
KeyError: 'NODE_ENV'
>>>
But if you do this:
me#host:/# NODE_ENV=foo
me#host:/# export NODE_ENV
It works:
me#host:/# python3
Python 3.8.2 (default, Apr 27 2020, 15:53:34)
[GCC 9.3.0] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> import os
>>> node_env = os.environ['NODE_ENV']
>>> print(node_env)
foo
>>>

Command for windows to delete the .pyc files:
del /S *.pyc

I had the same problem. I solved that by making some corrections on the .env file:
Before:
Key = Value
After my correction:
Key=Value
without blank spaces and worked!

I was getting this error while trying to source from a .env file.
I didn't explicitly export the env vars so I had to change this.
ENVIRONMENT=DEV
to this
export ENVIRONMENT=DEV

Use export a=10 instead of a=10 while setting env variable. Add the same in ~./bashrc to reload the env var wherever you login.
Doing this resolved the issue

I'd recommend you start debugging os.py, for instance, on windows it's being used this implementation:
def get(self, key, failobj=None):
print self.data.__class__
print key
return self.data.get(key.upper(), failobj)
And if I test it with this:
import os
try:
os.environ.get('NODE_ENV')
except Exception as e:
print("-->{0}".format(e.__class__))
os.environ['NODE_ENV'] = "foobar"
try:
os.environ.get('NODE_ENV')
except Exception as e:
print("{0}".format(e.__class__))
The output will be:
<type 'dict'>
PYTHONUSERBASE
<type 'dict'>
APPDATA
<type 'dict'>
NODE_ENV
<type 'dict'>
NODE_ENV
So it makes sense the exception is not spawned reading dict.get docs.
In any case, if you don't want to mess up or debugging the python modules, try cleaning up the *.pyc files, try to set up properly NODE_ENV. And if all that don't work, restart your terminal to clear up.

PyCuda: Can import module, then I can't... (PyCUDA Samples)

Example code:
import pycuda.autoinit
import pycuda.driver as drv
import numpy
from pycuda.compiler import SourceModule
mod = SourceModule("""
__global__ void multiply_them(float *dest, float *a, float *b)
{
const int i = threadIdx.x;
dest[i] = a[i] * b[i];
}
""")
multiply_them = mod.get_function("multiply_them")
a = numpy.random.randn(400).astype(numpy.float32)
b = numpy.random.randn(400).astype(numpy.float32)
dest = numpy.zeros_like(a)
multiply_them(
drv.Out(dest), drv.In(a), drv.In(b),
block=(400,1,1), grid=(1,1))
print dest-a*b
Results:
Traceback (most recent call last):
File "test.py", line 12, in <module>
""")
File "build/bdist.linux-x86_64/egg/pycuda/compiler.py", line 238, in __init__
File "build/bdist.linux-x86_64/egg/pycuda/compiler.py", line 223, in compile
File "build/bdist.linux-x86_64/egg/pycuda/compiler.py", line 149, in _find_pycuda_include_path
ImportError: No module named pycuda
Sounds simple enough, so lets test this.
Python 2.7.1 (r271:86832, Feb 17 2011, 14:13:40)
[GCC 4.3.4] on linux2
Type "help", "copyright", "credits" or "license" for more information.
>>> import pycuda
>>> pycuda
<module 'pycuda' from '/home/abolster/lib/python2.7/site-packages/pycuda-0.94.2-py2.7-linux-x86_64.egg/pycuda/__init__.pyc'>
>>>
Ok, thats weird...
Long story short, even stepping through the file line by line into the python console, nothing goes wrong until the actual execution of the mod=SourceModule() line.
(Final Traceback, I promise)
/home/abolster/lib/python2.7/site-packages/pycuda-0.94.2-py2.7-linux-x86_64.egg/pycuda/compiler.pyc in _find_pycuda_include_path()
147 def _find_pycuda_include_path():
148 from imp import find_module
--> 149 file, pathname, descr = find_module("pycuda")
150
151 # Who knew Python installation is so uniform and predictable?
ImportError: No module named pycuda
So it looks like pycuda is getting different include dirs than runtime python, which shouldn't happen (as i understand it)
Any ideas? (Sorry for the long question)
Talonmies borought up a point about nvcc not being found; unless python is getting its envars from somewhere I can't think of, there's no reason it shouldn't :
[bolster#dellgpu src]$ which nvcc
~/cuda/bin/nvcc

Changing to Python 2.6 and reinstalling relevant modules fixed the problem for the OP.

There is nothing wrong with the code you are trying to run - it should work. My guess is that nvcc cannot be found. Make sure that the path to the nvcc executable is set in your environment before you try using pycuda.compiler.

I think you did not install the CUDA toolkit from nvidia and added the
/usr/local/cuda/lib/
to
LD_LIBRARY_PATH
find the the .so of the pycuda module and give us the output of:
>lld pycuda.so