I am running an app on Google app engine that uses urlfetch in production, but uses requests locally. For example:
try:
import urlfetch
except:
import requests
The two modules act differently, so it's not as easy as just aliasing it, like json vs simplejson. How would I tell which module I have imported? For example, something like:
if 'urlfetch' loaded:
urlfetch(method='post', url='url', ...)
else:
requests.post(url)
You can use sys.modules to see which modules have been imported. For example:
>>> import sys; reload(sys)
>>> 'urlfetch' in sys.modules
True
Related
I'm trying to use a third-party lib (docutils) on Google App Engine and have a problem with this code (in docutils):
try:
import pwd
do stuff
except ImportError:
do other stuff
I want the import to fail, as it will on the actual GAE server, but the problem is that it doesn't fail on my development box (ubuntu). How to make it fail, given that the import is not in my own code?
Even easier than messing with __import__ is just inserting None in the sys.modules dict:
>>> import sys
>>> sys.modules['pwd'] = None
>>> import pwd
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
ImportError: No module named pwd
In your testing framework, before you cause docutils to be imported, you can perform this setup task:
import __builtin__
self.savimport = __builtin__.__import__
def myimport(name, *a):
if name=='pwd': raise ImportError
return self.savimport(name, *a)
__builtin__.__import__ = myimport
and of course in teardown put things back to normal:
__builtin__.__import__ = self.savimport
Explanation: all import operations go through __builtin__.__import__, and you can reassign that name to have such operations use your own code (alternatives such as import hooks are better for such purposes as performing import from non-filesystem sources, but for purposes such as yours, overriding __builtin__.__import__, as you see above, affords truly simple code).
I have two files 'mod1.py' and 'mod2.py'.
mod1 requires request module to function. But I have not imported them in the mod1 instead I have imported both request and mod1 module in mod2.
But
I get an error 'name 'requests' is not defined'. I know it works if i Import 'request' module in mod1 directly it works fine. But I have other modules I want to use that requires 'request' module. So how do I import the module once and make in accessible to all the other modules ?.
mod1.py
class getUrl():
def __init__(self, url):
self.url = url
def grab_html(self):
html = requests.get(self.url).text
return html
mod2.py
import requests
import mod1
module1 = mod1.getUrl('https://www.wikipedia.org/')
HTML = module1.grab_html()
Edit: Complete error
Traceback (most recent call last):
File "C:\Users\camel\Desktop\test\mod2.py", line 5, in <module>
HTML = module1.grab_html()
File "C:\Users\camel\Desktop\test\mod1.py", line 6, in grab_html
html = requests.get(self.url).text
NameError: name 'requests' is not defined
[Finished in 0.5s with exit code 1]
[shell_cmd: python -u "C:\Users\guru\Desktop\test\mod2.py"]
When you import something, it becomes a named thing in the module that imported it. Requests is not being used driectly by mod2.py, but is by mod1.py so that's where you should import it.
You can do this, for example.
mod1.py
import requests
class getUrl():
def __init__(self, url):
self.url = url
def grab_html(self):
html = requests.get(self.url).text
return html
mod2.py
import mod1
module1 = mod1.getUrl('https://www.wikipedia.org/')
HTML = module1.grab_html()
# And also access requests via mod1
indirectly = mod1.requests.get('https://www.wikipedia.org/').text
import requests, should be in mod1.py, because it is used in the methods of the class defined in mod1.py. you could import it in both places if it was needed in mod2.py as well.
As you are not using requests in mod2.py, you could just do import requests in mod1.py
If you are worried about the memory, it will take the same amount as you are going to use it in just one script. But if you are using if you are planning to use it in mod2.py as well, then you have to include there also.
You need to create an __init__.py file (it can be empty) file so that the folder containing mod1 is recognized as a module.
Then you can do from mod1 import *, or from path.to.mod1 import * and it will carry over all the imports over to mod2. Check out this relative answer. In my opinion this is a sensible way of doing things, as you can keep all your dependencies in a centralized location.
As you're concerned about memory utilization, take a look at another conversation on the same matter.
I have pip installed watson-developer-cloud, on python v3.5
I am simply trying to run one of the example codes: alchemy_data_news_v1.py
Link:https://github.com/watson-developer-cloud/python-sdk/tree/master/examples
import json
from watson_developer_cloud import AlchemyLanguageV1
alchemy_data_news = AlchemyDataNewsV1(api_key='api-key')
results = alchemy_data_news.get_news_documents(start='now-7d', end='now',
time_slice='12h')
print(json.dumps(results, indent=2))
results = alchemy_data_news.get_news_documents(
start='1453334400',
end='1454022000',
return_fields=['enriched.url.title',
'enriched.url.url',
'enriched.url.author',
'enriched.url.publicationDate'],
query_fields={
'q.enriched.url.enrichedTitle.entities.entity':
'|text=IBM,type=company|'})
print(json.dumps(results, indent=2))
I have also tried utilizing my own personal api-key and the result is the same:
File "c:\users\Joseph Sansevero\desktop\test.py", line 2, in
watson_developer_cloud import AlchemyLanguageV1 ImportError: No module
named watson_developer_cloud
Change your import statement to
from watson_developer_cloud import AlchemyLanguageV1
Alchemy language is a different api than AlchemyNews.
Head over to https://www.ibm.com/watson/developercloud/alchemydata-news/api/v1/?python#methods and you'll see the example has AlchemyNews imported.
Also make sure you install these packages using before running your code.
I'm trying to use a variable as a module to import from in Python.
Using ImportLib I have been successfully able to find the test...
sys.path.insert(0, sys.path[0] + '\\tests')
tool_name = selected_tool.split(".")[0]
selected_module = importlib.import_module("script1")
print(selected_module)
... and by printing the select_module I can see that it succesfully finds the script:
<module 'script1' from 'C:\\Users\\.....">
However, when I try to use this variable in the code to import a module from it:
from selected_module import run
run(1337)
The program quits with the following error:
ImportError: No module named 'selected_module'
I have tried to add a init.py file to the main directory and the /test directory where the scripts are, but to no avail. I'm sure it's just something stupidly small I'm missing - does anyone know?
Import statements are not sensitive to variables! Their content are treated as literals
An example:
urllib = "foo"
from urllib import parse # loads "urllib.parse", not "foo.parse"
print(parse)
Note that from my_module import my_func will simply bind my_module.my_func to the local name my_func. If you have already imported the module via importlib.import_module, you can just do this yourself:
# ... your code here
run = selected_module.run # bind module function to local name
I have a few Munin plugins which report stats from an Autonomy database. They all use a small library which scrapes the XML status output for the relevant numbers.
I'm trying to bundle the library and plugins into a Puppet-installable RPM. The actual RPM-building should be straightforward; once I have a distutils-produced distfile I can make it into an RPM based on a .spec file pinched from the Dag or EPEL repos [1]. It's the distutils bit I'm unsure of -- in fact I'm not even sure my library is correctly written for packaging. Here's how it works:
idol7stats.py:
import datetime
import os
import stat
import sys
import time
import urllib
import xml.sax
class IDOL7Stats:
cache_dir = '/tmp'
def __init__(self, host, port):
self.host = host
self.port = port
# ...
def collect(self):
self.data = self.__parseXML(self.__getXML())
def total_slots(self):
return self.data['Service:Documents:TotalSlots']
Plugin code:
from idol7stats import IDOL7Stats
a = IDOL7Stats('db.example.com', 23113)
a.collect()
print a.total_slots()
I guess I want idol7stats.py to wind up in /usr/lib/python2.4/site-packages/idol7stats, or something else in Python's search path. What distutils magic do I need? This:
from distutils.core import setup
setup(name = 'idol7stats',
author = 'Me',
author_email = 'me#example.com',
version = '0.1',
py_modules = ['idol7stats'])
almost works, except the code goes in /usr/lib/python2.4/site-packages/idol7stats.py, not a subdirectory. I expect this is down to my not understanding the difference between modules/packages/other containers in Python.
So, what's the rub?
[1] Yeah, I could just plonk the library in /usr/lib/python2.4/site-packages using RPM but I want to know how to package Python code.
You need to create a package to do what you want. You'd need a directory named idol7stats containing a file called __init__.py and any other library modules to package. Also, this will affect your scripts' imports; if you put idol7stats.py in a package called idol7stats, then your scripts need to "import idol7stats.idol7stats".
To avoid that, you could just rename idol7stats.py to idol7stats/__init__.py, or you could put this line into idol7stats/__init__.py to "massage" the imports into the way you expect them:
from idol7stats.idol7stats import *