I wrote a Python module which is just two methods, help and check. Check just takes a string file name and does something with it. When I import the module, there are no methods in it. Only __name__ and the like, but neither check nor help appears in the dir.
I am just importing the file. lyricCheck.py
Here's my code in lyricCheck.py:
#!/usr/bin/python
#python lyric checker
#Jim Boulter
#January 19, 2015
#Copyright 2015
import sys
import urllib2
import fileinput
from decimal import *
from re import *
from pygoogle import pygoogle
def help():
print 'usage: python check.py filename.txt\n'
print 'input line structure: artist name; song title\n'
def check(filename):
if(str(filename).lower == "help" or str(filename).lower == "-h"):
help()
return
#do lots of other stuff
If you are creating your package like this, and I believe you are, you need to import your module from your package:
~/tmp$ mkdir lyricCheck
~/tmp$ cd lyricCheck/
~/tmp/lyricCheck$ touch __init__.py
~/tmp/lyricCheck$ cat > lyricCheck.py
#!/usr/bin/python
#python lyric checker
#Jim Boulter
#January 19, 2015
#Copyright 2015
import sys
import urllib2
import fileinput
from decimal import *
from re import *
from pygoogle import pygoogle
def help():
print 'usage: python check.py filename.txt\n'
print 'input line structure: artist name; song title\n'
def check(filename):
if(str(filename).lower == "help" or str(filename).lower == "-h"):
help()
return
#do lots of other stuff
~/tmp/lyricCheck$ cd ..
~/tmp$ python
Python 2.7.6 (default, Mar 22 2014, 22:59:56)
[GCC 4.8.2] on linux2
Type "help", "copyright", "credits" or "license" for more information.
>>> import lyricCheck
>>> dir(lyricCheck)
['__builtins__', '__doc__', '__file__', '__name__', '__package__', '__path__']
Here's my payoff, and this should have worked if I had pygoogle, so this is how I know I found the issue:
>>> from lyricCheck import lyricCheck
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
File "lyricCheck/lyricCheck.py", line 12, in <module>
from pygoogle import pygoogle
ImportError: No module named pygoogle
You can put this in your __init__.py file to import the functions from the lower level module and make them available right at the package level:
from lyricCheck import help, check
Also note, when you do this:
from decimal import *
from re import *
You dump all the names in those modules into your module's namespace. It's generally considered better to declare them individually.
Related
I am facing a problem in python. Tho the error is quite common, but since i am bit new to python, unable to comprehend the source hence asking you all. There are 2 modules: session.py and objects.py.
session.py
import copy
import pymongo
import spacy
import tweepy
import objects
objects.py:
import re
def refresh (sv = sv, obj = ''):
return 0
now, in python shell, i am getting the error before even executing objects.py:
$ python
Python 2.7.13 (default, Sep 26 2018, 18:42:22)
[GCC 6.3.0 20170516] on linux2
Type "help", "copyright", "credits" or "license" for more information.
>>> import session
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
File "session.py", line 6, in <module>
import objects
File "objects.py", line 3, in <module>
def refresh (sv = sv, obj = ''):
NameError: name 'sv' is not defined
>>>
I came from perl background to maybe missing some very common thing, but still i am able to do this:
>>> def ff(t): print t
...
In above, whitout defining t, it is working while in objects.py, how can i define sv without starting execution?
To do
import itertools
dynamically, I can do the following
import importlib
importlib.import_module('itertools')
But what should I do to do the following dynamically
import itertools as iters
Problem Context:
I need to import a different version (0.10) of a module('pika' in my case), installed in a separate directory instead of default version(0.9).
Python 2.7.6 (default, Jun 22 2015, 17:58:13)
[GCC 4.8.2] on linux2
Type "help", "copyright", "credits" or "license" for more information.
Welcome to Python !!
>>> import importlib
>>> import pika
>>> pika.__version__
'0.9.14'
>>> import scale.lib.hypervisor.esx65.pika_3_5 as pika35
>>> pika35.__version__
'0.10.0'
>>> importlib.import_module('scale.lib.hypervisor.esx65.pika_3_5')
<module 'scale.lib.hypervisor.esx65.pika_3_5' from 'scale/lib/hypervisor/esx65/pika_3_5/__init__.pyc'>
As we can see regular imports are working fine. However when importing dynamically, importing relative to the location is causing issues. As per importlib.import_module documentaion , following should work but it doesn't.
>>> importlib.import_module('pika_3_5', 'scale.lib.hypervisor.esx65')
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
File "/usr/lib/python2.7/importlib/__init__.py", line 37, in import_module
__import__(name)
ImportError: No module named pika_3_5
And when trying to import 'pika' from a relative path, which should fail as there is no module pika under the relative path, it is still importing it from default module.
>>> importlib.import_module('pika', 'scale.lib.hypervisor.esx65.pika_3_5')
<module 'pika' from '/usr/local/lib/python2.7/dist-packages/pika/__init__.pyc'>
>>>
What is it that I am missing ? I mainly want to do the following dynamically.
import scale.lib.hypervisor.esx65.pika_3_5 as pika
To do
import itertools
dynamically, I can do the following
import importlib
importlib.import_module('itertools')
No, you do the following:
import importlib
itertools = importlib.import_module('itertools')
Similarly, to replicate import itertools as iters, you do
import importlib
iters = importlib.import_module('itertools')
importlib.import_module doesn't care what you call the module. as is not and cannot be part of importlib.import_module's functionality. It just gives you the module object; what you call that object is up to you.
As for your context, you've misunderstood what a relative import is. The second argument to importlib.import_module has nothing to do with from imports, and importlib.import_module('thing', 'whatever') is not supposed to be equivalent to from whatever import thing.
If you want to do
import scale.lib.hypervisor.esx65.pika_3_5 as pika
that's pika = importlib.import_module('scale.lib.hypervisor.esx65.pika_3_5'). The second argument doesn't enter the picture. You seem to think this is somehow not dynamic, but it's as dynamic as any other importlib call.
I'm accessing an environment variable in a script with os.environ.get and it's throwing a KeyError. It doesn't throw the error from the Python prompt. This is running on OS X 10.11.6, and is Python 2.7.10.
What is going on?
$ python score.py
Traceback (most recent call last):
File "score.py", line 4, in <module>
setup_logging()
File "/score/log.py", line 29, in setup_logging
config = get_config()
File "/score/log.py", line 11, in get_config
environment = os.environ.get('NODE_ENV')
File "/System/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/UserDict.py", line 23, in __getitem__
raise KeyError(key)
KeyError: 'NODE_ENV'
$ python -c "import os; os.environ.get('NODE_ENV')"
$
As requested, here's the source code for score.py
from __future__ import print_function
from log import get_logger, setup_logging
setup_logging()
log = get_logger('score')
And here's log.py
import json
import os
import sys
from iron_worker import IronWorker
from logbook import Logger, Processor, NestedSetup, StderrHandler, SyslogHandler
IRON_IO_TASK_ID = IronWorker.task_id()
def get_config():
environment = os.environ.get('NODE_ENV')
if environment == 'production':
filename = '../config/config-production.json'
elif environment == 'integration':
filename = '../config/config-integration.json'
else:
filename = '../config/config-dev.json'
with open(filename) as f:
return json.load(f)
def setup_logging():
# This defines a remote Syslog handler
# This will include the TASK ID, if defined
app_name = 'scoreworker'
if IRON_IO_TASK_ID:
app_name += '-' + IRON_IO_TASK_ID
config = get_config()
default_log_handler = NestedSetup([
StderrHandler(),
SyslogHandler(
app_name,
address = (config['host'], config['port']),
level = 'ERROR',
bubble = True
)
])
default_log_handler.push_application()
def get_logger(name):
return Logger(name)
Try running:
find . -name \*.pyc -delete
To delete your .pyc files.
Researching your problem I came across this question, where a user was experiencing the same thing: .get() seemingly raising a KeyError. In that case, it was caused, according to this accepted answer, by a .pyc file which contained code where a dict value was being accessed by key (i.e., mydict['potentially_nonexistent_key']), while the traceback was showing the code from the updated .py file where .get() was used. I have never heard of this happening, where the traceback references current code from a .py file, but shows an error raised by an outdated .pyc file, but it seems to have happened at least once in the history of Python...
It is a long shot, but worth a try I thought.
I encountered a similar error when I set the environment variable without exporting it. So if you do this:
me#host:/# NODE_ENV=foo
You will get this:
me#host:/# python3
Python 3.8.2 (default, Apr 27 2020, 15:53:34)
[GCC 9.3.0] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> import os
>>> node_env = os.environ['NODE_ENV']
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
File "/usr/lib/python3.8/os.py", line 675, in __getitem__
raise KeyError(key) from None
KeyError: 'NODE_ENV'
>>>
But if you do this:
me#host:/# NODE_ENV=foo
me#host:/# export NODE_ENV
It works:
me#host:/# python3
Python 3.8.2 (default, Apr 27 2020, 15:53:34)
[GCC 9.3.0] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> import os
>>> node_env = os.environ['NODE_ENV']
>>> print(node_env)
foo
>>>
Command for windows to delete the .pyc files:
del /S *.pyc
I had the same problem. I solved that by making some corrections on the .env file:
Before:
Key = Value
After my correction:
Key=Value
without blank spaces and worked!
I was getting this error while trying to source from a .env file.
I didn't explicitly export the env vars so I had to change this.
ENVIRONMENT=DEV
to this
export ENVIRONMENT=DEV
Use export a=10 instead of a=10 while setting env variable. Add the same in ~./bashrc to reload the env var wherever you login.
Doing this resolved the issue
I'd recommend you start debugging os.py, for instance, on windows it's being used this implementation:
def get(self, key, failobj=None):
print self.data.__class__
print key
return self.data.get(key.upper(), failobj)
And if I test it with this:
import os
try:
os.environ.get('NODE_ENV')
except Exception as e:
print("-->{0}".format(e.__class__))
os.environ['NODE_ENV'] = "foobar"
try:
os.environ.get('NODE_ENV')
except Exception as e:
print("{0}".format(e.__class__))
The output will be:
<type 'dict'>
PYTHONUSERBASE
<type 'dict'>
APPDATA
<type 'dict'>
NODE_ENV
<type 'dict'>
NODE_ENV
So it makes sense the exception is not spawned reading dict.get docs.
In any case, if you don't want to mess up or debugging the python modules, try cleaning up the *.pyc files, try to set up properly NODE_ENV. And if all that don't work, restart your terminal to clear up.
I have a package with three files:
testimport
├── __init__.py
├── logging.py
└── util.py
__init__.py contains:
from __future__ import ( absolute_import, division, print_function, unicode_literals )
import logging # imports standard library module (because absolute_import is activated)
_logging_file = logging.__file__
from .util import testlog
if _logging_file != logging.__file__:
# at this point, `logging` no longer points to the standard
# library module, but the local logging module instead(!)
raise AssertionError('`logging` overwritten; {!r} is not equal to {!r}'.format(_logging_file, logging.__file__))
LOGGER = logging.getLogger(__name__)
logging.py contains:
import sys
__all__ = ()
SILENT = -(sys.maxsize) - 1
util.py contains:
from __future__ import ( absolute_import, division, print_function, unicode_literals )
import logging # imports standard library module (because absolute_import is activated)
from .logging import SILENT # this is (perversely) where the importing module's `logging` gets overridden
__all__ = ( 'testlog' )
_LOGGER = logging.getLogger(__name__)
def testlog(log_lvl=SILENT):
_LOGGER.log(log_lvl, 'Hello!')
The AssertionError is raised when importing testimport:
% python
Python 2.7.10 (default, Sep 24 2015, 10:13:45)
[GCC 4.2.1 Compatible Apple LLVM 6.0 (clang-600.0.57)] on darwin
Type "help", "copyright", "credits" or "license" for more information.
>>> import testimport
<function testlog at 0x10e86e1b8>
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
File "testimport/__init__.py", line ..., in <module>
raise AssertionError('`logging` overwritten; {!r} is not equal to {!r}'.format(_logging_file, logging.__file__))
AssertionError: `logging` overwritten; '/.../lib/python2.7/logging/__init__.pyc' is not equal to 'testimport/logging.pyc'
Why on earth is that happening?
Test repo is here. Travis builds are here.
Update, when stepping through this with pdb, it appears that the offending instruction is from .logging import SILENT in util.py, but I have no idea why. This is an abbreviated session from the repo version:
% echo 'import testimport' >|testme.py
% python -m pdb testme.py
(Pdb) s
--Call--
> /.../testimport/testimport/__init__.py(1)<module>()
-> from __future__ import (
(Pdb) b 12
Breakpoint 1 at /.../testimport/testimport/__init__.py:12
(Pdb) c
> /.../testimport/testimport/__init__.py(12)<module>()
-> from testimport.util import testlog
(Pdb) s
--Call--
> /.../testimport/testimport/util.py(1)<module>()
-> from __future__ import (
(Pdb) b 5
Breakpoint 2 at /.../testimport/testimport/util.py:5
(Pdb) c
> /.../testimport/testimport/util.py(5)<module>()
-> from .logging import SILENT
(Pdb) u
> /.../testimport/testimport/__init__.py(12)<module>()
-> from testimport.util import testlog
(Pdb) p logging
<module 'logging' from '/.../lib/python2.7/logging/__init__.pyc'>
(Pdb) d
> /.../testimport/testimport/util.py(5)<module>()
-> from .logging import SILENT
(Pdb) s
--Call--
> /.../testimport/testimport/logging.py(1)<module>()
-> from __future__ import (
(Pdb) b 6
Breakpoint 3 at /.../testimport/testimport/logging.py:6
(Pdb) c
> /.../testimport/testimport/logging.py(6)<module>()
-> SILENT = -(sys.maxsize) - 1
(Pdb) u
> /.../testimport/testimport/util.py(5)<module>()
-> from .logging import SILENT
(Pdb) u
> /.../testimport/testimport/__init__.py(12)<module>()
-> from testimport.util import testlog
(Pdb) p logging
<module 'logging' from '/.../lib/python2.7/logging/__init__.pyc'>
(Pdb) s
> /.../testimport/testimport/util.py(7)<module>()
-> 'testlog',
(Pdb) u
> /.../testimport/testimport/__init__.py(12)<module>()
-> from testimport.util import testlog
(Pdb) p logging
<module 'testimport.logging' from 'testimport/logging.pyc'>
After some more research, I understand the above to be the correct behavior. I am embarrassed to say that I did not see this before. It makes total sense once one understands what's happening. From the docs:
Submodules
When a submodule is loaded using any mechanism (e.g. importlib APIs, the import or import-from statements, or built-in __import__()) a binding is placed in the parent module’s namespace to the submodule object.
...
Given Python’s familiar name binding rules this might seem surprising, but it’s actually a fundamental feature of the import system.
This explanation seems to have been absent before Python 3.4, but affects all versions of Python. See Python issue #24029.
In the example above, testimport.logging necessarily refers to the local module. Importing it (from anywhere) installs it in testimport (as should be expected). It should come as no surprise that this necessarily replaces any existing logging member of testimport.
I have a package named 'package'within that i have modules module1.py and module2.py i imported the package as
import package
from package import module1
In module1 i have a function named funcwhenever i import that function as
from module1 import func
and use it, the function as
module1.func(x)
it doesn't work
What is the problem and what should be done??
You can do either:
from module1 import func
func(x)
OR
module1.func(x)
Real world example which should demonstrate how things work:
>>> import os
>>> os.path.abspath("C:/Documents")
'C:\\Documents'
>>>
>>> from os import path
>>> path.abspath("C:/documents")
'C:\\documents'
>>>
>>> from path import abspath
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
ImportError: No module named path
>>>
>>> from os.path import abspath
>>> abspath("C:/documents")
'C:\\documents'
You can either import as:
from foo import bar
bar(baz)
or:
import foo
foo.bar(baz)
In certain cases, it may also be helpful to:
from foo import bar as qux
qux(baz
There is an extensive tutorial on handling imports available as well.
2 options:
from package.module1 import func
func(x)
2nd option:
from package import module1
module1.func(x)