I'm using a pretty standard boilerplate undetected_chromedriver installation:
LOCAL_CHRPATH = '/opt/google/chrome/chromedriver'
driver = uc.Chrome(
driver_executable_path=LOCAL_CHRPATH, headless=True
)
However, driver.get(url) simply fails silently, exiting the program. I have looked through the documentation, and found no way to get diagnostics pre or post get. Any idea of what to do or where to check?
You can pretty much ignore the specifics about the path and so on; the real question here is to understand why it fails.
Update: if I set debug to True, I get the same error that's indicated in this SO question. However, don't see a workaround for that. As an indication, this was working correctly with Python 3.10, stopped working when downgrading to 3.9
LOCAL_CHRPATH = '/opt/google/chrome/chromedriver'
This is local variable, not environmental. To set envvar you could:
os.environ["LOCAL_CHRPATH"] = '/opt/google/chrome/chromedriver'
Or just don't use envvars, or set default value:
driver = uc.Chrome(
driver_executable_path=os.getenv('LOCAL_CHRPATH', LOCAL_CHRPATH),
headless=True
)
Using flask, flask-assets, webassets
I am trying to pass a filter specific configuration for a particular filter in webassets, but haven't been able to do so, so far.
Have read the documentation multiple times to no avail. Have also gone through the source code, but that hasn't helped either.
Specifics:
The filter I'm trying to use is cleancss
And the particular configuration I'm trying to pass is --skip-rebase
According to the link to cleancss docs above, I should be able to pass the --skip-rebase option in CLEANCSS_EXTRA_ARGS. But where do I put that?
What I've tried so far, passing as flask config variables:
CLEANCSS_EXTRA_ARGS = ['--skip-rebase'] # actually works, see answer!
CLEANCSS_EXTRA_ARGS = ['skip-rebase'] # doesn't work!
ASSETS_CLEANCSS_EXTRA_ARGS = ['--skip-rebase'] # doesn't work!
FLASK_ASSETS_CLEANCSS_EXTRA_ARGS = ['--skip-rebase'] # doesn't work!
ASSETS_CLEANCSS_EXTRA_ARGS = ['skip-rebase'] # doesn't work!
FLASK_ASSETS_CLEANCSS_EXTRA_ARGS = ['skip-rebase'] # doesn't work!
ps: I also tried modifying the source to check if cleancss parameters are being correctly passed to the command line, and it is, so the problem is only in specifying the option correctly in the app configuration.
On retrying with the first option in question:
CLEANCSS_EXTRA_ARGS = ['--skip-rebase']
It worked!
More info available in the flask assets repo issues.
In my hurry I must have tried with:
CLEANCSS_EXTRA_ARGS = ['skip-rebase']
which didn't work and hence the question.
This feature should be documented though. Will probably submit a pull request for doc improvement.
Also note that this requires version 0.11 (webassets) and above.
I'm trying to add cross-references to external API into my documentation but I'm facing three different behaviors.
I am using sphinx(1.3.1) with Python(2.7.3) and my intersphinx mapping is configured as:
{
'python': ('https://docs.python.org/2.7', None),
'numpy': ('http://docs.scipy.org/doc/numpy/', None),
'cv2' : ('http://docs.opencv.org/2.4/', None),
'h5py' : ('http://docs.h5py.org/en/latest/', None)
}
I have no trouble writing a cross-reference to numpy API with :class:`numpy.ndarray` or :func:`numpy.array` which gives me, as expected, something like numpy.ndarray.
However, with h5py, the only way I can have a link generated is if I omit the module name. For example, :class:`Group` (or :class:`h5py:Group`) gives me Group but :class:`h5py.Group` fails to generate a link.
Finally, I cannot find a way to write a working cross-reference to OpenCV API, none of these seems to work:
:func:`cv2.convertScaleAbs`
:func:`cv2:cv2.convertScaleAbs`
:func:`cv2:convertScaleAbs`
:func:`convertScaleAbs`
How to properly write cross-references to external API, or configure intersphinx, to have a generated link as in the numpy case?
In addition to the detailed answer from #gall, I've discovered that intersphinx can also be run as a module:
python -m sphinx.ext.intersphinx 'http://python-eve.org/objects.inv'
This outputs nicely formatted info. For reference: https://github.com/sphinx-doc/sphinx/blob/master/sphinx/ext/intersphinx.py#L390
I gave another try on trying to understand the content of an objects.inv file and hopefully this time I inspected numpy and h5py instead of only OpenCV's one.
How to read an intersphinx inventory file
Despite the fact that I couldn't find anything useful about reading the content of an object.inv file, it is actually very simple with the intersphinx module.
from sphinx.ext import intersphinx
import warnings
def fetch_inventory(uri):
"""Read a Sphinx inventory file into a dictionary."""
class MockConfig(object):
intersphinx_timeout = None # type: int
tls_verify = False
class MockApp(object):
srcdir = ''
config = MockConfig()
def warn(self, msg):
warnings.warn(msg)
return intersphinx.fetch_inventory(MockApp(), '', uri)
uri = 'http://docs.python.org/2.7/objects.inv'
# Read inventory into a dictionary
inv = fetch_inventory(uri)
# Or just print it
intersphinx.debug(['', uri])
File structure (numpy)
After inspecting numpy's one, you can see that keys are domains:
[u'np-c:function',
u'std:label',
u'c:member',
u'np:classmethod',
u'np:data',
u'py:class',
u'np-c:member',
u'c:var',
u'np:class',
u'np:function',
u'py:module',
u'np-c:macro',
u'np:exception',
u'py:method',
u'np:method',
u'np-c:var',
u'py:exception',
u'np:staticmethod',
u'py:staticmethod',
u'c:type',
u'np-c:type',
u'c:macro',
u'c:function',
u'np:module',
u'py:data',
u'np:attribute',
u'std:term',
u'py:function',
u'py:classmethod',
u'py:attribute']
You can see how you can write your cross-reference when you look at the content of a specific domain. For example, py:class:
{u'numpy.DataSource': (u'NumPy',
u'1.9',
u'http://docs.scipy.org/doc/numpy/reference/generated/numpy.DataSource.html#numpy.DataSource',
u'-'),
u'numpy.MachAr': (u'NumPy',
u'1.9',
u'http://docs.scipy.org/doc/numpy/reference/generated/numpy.MachAr.html#numpy.MachAr',
u'-'),
u'numpy.broadcast': (u'NumPy',
u'1.9',
u'http://docs.scipy.org/doc/numpy/reference/generated/numpy.broadcast.html#numpy.broadcast',
u'-'),
...}
So here, :class:`numpy.DataSource` will work as expected.
h5py
In the case of h5py, the domains are:
[u'py:attribute', u'std:label', u'py:method', u'py:function', u'py:class']
and if you look at the py:class domain:
{u'AttributeManager': (u'h5py',
u'2.5',
u'http://docs.h5py.org/en/latest/high/attr.html#AttributeManager',
u'-'),
u'Dataset': (u'h5py',
u'2.5',
u'http://docs.h5py.org/en/latest/high/dataset.html#Dataset',
u'-'),
u'ExternalLink': (u'h5py',
u'2.5',
u'http://docs.h5py.org/en/latest/high/group.html#ExternalLink',
u'-'),
...}
That's why I couldn't make it work as numpy references. So a good way to format them would be :class:`h5py:Dataset`.
OpenCV
OpenCV's inventory object seems malformed. Where I would expect to find domains there is actually 902 function signatures:
[u':',
u'AdjusterAdapter::create(const',
u'AdjusterAdapter::good()',
u'AdjusterAdapter::tooFew(int',
u'AdjusterAdapter::tooMany(int',
u'Algorithm::create(const',
u'Algorithm::getList(vector<string>&',
u'Algorithm::name()',
u'Algorithm::read(const',
u'Algorithm::set(const'
...]
and if we take the first one's value:
{u'Ptr<AdjusterAdapter>': (u'OpenCV',
u'2.4',
u'http://docs.opencv.org/2.4/detectorType)',
u'ocv:function 1 modules/features2d/doc/common_interfaces_of_feature_detectors.html#$ -')}
I'm pretty sure it is then impossible to write OpenCV cross-references with this file...
Conclusion
I thought intersphinx generated the objects.inv based on the content of the documentation project in an standard way, which seems not to be the case.
As a result, it seems that the proper way to write cross-references is API dependent and one should inspect a specific inventory object to actually see what's available.
An additional way to inspect the objects.inv file is with the sphobjinv module.
You can search local or even remote inventory files (with fuzzy matching). For instance with scipy:
$ sphobjinv suggest -t 90 -u https://docs.scipy.org/doc/scipy/reference/objects.inv "signal.convolve2d"
Remote inventory found.
:py:function:`scipy.signal.convolve2d`
:std:doc:`generated/scipy.signal.convolve2d`
Note that you may need to use :py:func: and not :py:function: (I'd be happy to know why).
How to use OpenCV 2.4 (cv2) intersphinx
Inspired by #Gall's answer, I wanted to compare the contents of the OpenCV & numpy inventory files. I couldn't get sphinx.ext.intersphinx.fetch_inventory to work from ipython, but the following does work:
curl http://docs.opencv.org/2.4/objects.inv | tail -n +5 | zlib-flate -uncompress > cv2.inv
curl https://docs.scipy.org/doc/numpy/objects.inv | tail -n +5 | zlib-flate -uncompress > numpy.inv
numpy.inv has lines like this:
numpy.ndarray py:class 1 reference/generated/numpy.ndarray.html#$ -
whereas cv2.inv has lines like this:
cv2.imread ocv:pyfunction 1 modules/highgui/doc/reading_and_writing_images_and_video.html#$ -
So presumably you'd link to the OpenCV docs with :ocv:pyfunction:`cv2.imread` instead of :py:function:`cv2.imread`. Sphinx doesn't like it though:
WARNING: Unknown interpreted text role "ocv:pyfunction".
A bit of Googling revealed that the OpenCV project has its own "ocv" sphinx domain: https://github.com/opencv/opencv/blob/2.4/doc/ocv.py -- presumably because they need to document C, C++ and Python APIs all at the same time.
To use it, save ocv.py next to your Sphinx conf.py, and modify your conf.py:
sys.path.insert(0, os.path.abspath('.'))
import ocv
extensions = [
'ocv',
]
intersphinx_mapping = {
'cv2': ('http://docs.opencv.org/2.4/', None),
}
In your rst files you need to say :ocv:pyfunc:`cv2.imread` (not :ocv:pyfunction:).
Sphinx prints some warnings (unparseable C++ definition: u'cv2.imread') but the generated html documentation actually looks ok with a link to http://docs.opencv.org/2.4/modules/highgui/doc/reading_and_writing_images_and_video.html#cv2.imread. You can edit ocv.py and remove the line that prints that warning.
The accepted answer no longer works in the new version (1.5.x) ...
import requests
import posixpath
from sphinx.ext.intersphinx import read_inventory
uri = 'http://docs.python.org/2.7/'
r = requests.get(uri + 'objects.inv', stream=True)
r.raise_for_status()
inv = read_inventory(r.raw, uri, posixpath.join)
Stubborn fool that I am, I used 2to3 and the Sphinx deprecated APIs chart to revive #david-röthlisberger's ocv.py-based answer so it'll work with Sphinx 2.3 on Python 3.5.
The fixed-up version is here:
https://gist.github.com/ssokolow/a230b27b7ea4a31f7fb40621e6461f9a
...and the quick version of what I did was:
Run 2to3 -w ocv.py && rm ocv.py.bak
Cycle back and forth between running Sphinx and renaming functions to their replacements in the chart. I believe these were the only changes I had to make on this step:
Directive now has to be imported from docutils.parsers.rst
Replace calls to l_(...) with calls to _(...) and remove the l_ import.
Replace calls to env.warn with calls to log.warn where log = sphinx.util.logging.getLogger(__name__).
Then, you just pair it with this intersphinx definition and you get something still new enough to be relevant for most use cases:
'cv2': ('https://docs.opencv.org/3.0-last-rst/', None)
For convenience, I made a small extension for aliasing intersphinx cross references. This is useful as sometimes the object inventory gets confused when an object from a submodule is imported from a package's __init__.py.
See also https://github.com/sphinx-doc/sphinx/issues/5603
###
# Workaround of
# Intersphinx references to objects imported at package level can"t be mapped.
#
# See https://github.com/sphinx-doc/sphinx/issues/5603
intersphinx_aliases = {
("py:class", "click.core.Group"):
("py:class", "click.Group"),
("py:class", "click.core.Command"):
("py:class", "click.Command"),
}
def add_intersphinx_aliases_to_inv(app):
from sphinx.ext.intersphinx import InventoryAdapter
inventories = InventoryAdapter(app.builder.env)
for alias, target in app.config.intersphinx_aliases.items():
alias_domain, alias_name = alias
target_domain, target_name = target
try:
found = inventories.main_inventory[target_domain][target_name]
try:
inventories.main_inventory[alias_domain][alias_name] = found
except KeyError:
print("could not add to inv")
continue
except KeyError:
print("missed :(")
continue
def setup(app):
app.add_config_value("intersphinx_aliases", {}, "env")
app.connect("builder-inited", add_intersphinx_aliases_to_inv)
To use this, I paste the above code in my conf.py and add aliases to the intersphinx_aliases dictionary.
I am trying to use Pubnub Access Management with Python by I always get an error message saying that the 'Signature Does not Match'. I know their documentation says that this means that the Secret/Publish/Subscribe keys may be wrong but I have triple checked that they are correct. Also checked in Pubnub developer console and it works fine from there.
Here is the method that I am using:
def get_pubnub_token(request, connector_id):
pubnub = Pubnub(settings.PUBKEY, settings.SUBKEY, secret_key=settings.PUB_SECRET_KEY)
authkey = str(uuid.uuid4())
result = pubnub.grant(channel=connector_id+"-pnpres", auth_key=authkey, read=True, write=True, ttl=60)
if result['error']:
return HttpResponse(status=result['status'])
result = pubnub.grant(channel=connector_id, auth_key=authkey, read=True, write=True, ttl=60)
if result['error']:
return HttpResponse(status=result['status'])
result = pubnub.grant(channel=connector_id+"-stream", auth_key=authkey, read=True, write=True, ttl=60)
if result['error']:
return HttpResponse(status=result['status'])
jsonstr = get_json(authkey)
return HttpResponse(jsonstr, content_type='application/json')
I wonder if anyone has done this before and if they could give me any tips as to why I could be seeing this error.
Which version of the SDK are you on?
Also, this may also happen because your system clock is not set to NTP. Can you verify its set to NTP, and try again?
If it persists, please contact us at support#pubnub.com and we'd be happy to help trace the issue down with you.
geremy
There were problems using version 3.7.1 of the SDK in python 3.4.3. They were addressed by Pubnub's developers some time last night and it seems to have been resolved in their development code. I guess they will put it into the production code some time in the near future.
I develop a critical application used by a multi-national company. Users in offices all around the globe need to be able to install this application.
The application is actually a plugin to Excel and we have an automatic installer based on Setuptools' easy_install that ensures that all a project's dependancies are automatically installed or updated any time a user switches on their Excel. It all works very elegantly as users are seldom aware of all the installation which occurs entirely in the background.
Unfortunately we are expanding and opening new offices which all have different proxy settings. These settings seem to change from day to day so we cannot keep up with the outsourced security guys who change stuff without telling us. It sucks but we just have to work around it.
I want to programatically detect the system-wide proxy settings on the Windows workstations our users run:
Everybody in the organisazation runs Windows XP and Internet Explorer. I've verified that everybody can download our stuff from IE without problems regardless of where they are int the world.
So all I need to do is detect what proxy settings IE is using and make Setuptools use those settings. Theoretically all of this information should be in the Registry.. but is there a better way to find it that is guaranteed not to change with people upgrade IE? For example is there a Windows API call I can use to discover the proxy settings?
In summary:
We use Python 2.4.4 on Windows XP
We need to detect the Internet Explorer proxy settings (e.g. host, port and Proxy type)
I'm going to use this information to dynamically re-configure easy_install so that it can download the egg files via the proxy.
UPDATE0:
I forgot one important detail: Each site has an auto-config "pac" file.
There's a key in Windows\CurrentVersion\InternetSettings\AutoConfigURL which points to a HTTP document on a local server which contains what looks like a javascript file.
The pac script is basically a series of nested if-statements which compare URLs against a regexp and then eventually return the hostname of the chosen proxy-server. The script is a single javascript function called FindProxyForURL(url, host)
The challenge is therefore to find out for any given server which proxy to use. The only 100% guaranteed way to do this is to look up the pac file and call the Javascript function from Python.
Any suggestions? Is there a more elegant way to do this?
Here's a sample that should create a bullet green (proxy enable) or red (proxy disable) in your systray
It shows how to read and write in windows registry
it uses gtk
#!/usr/bin/env python
import gobject
import gtk
from _winreg import *
class ProxyNotifier:
def __init__(self):
self.trayIcon = gtk.StatusIcon()
self.updateIcon()
#set callback on right click to on_right_click
self.trayIcon.connect('popup-menu', self.on_right_click)
gobject.timeout_add(1000, self.checkStatus)
def isProxyEnabled(self):
aReg = ConnectRegistry(None,HKEY_CURRENT_USER)
aKey = OpenKey(aReg, r"Software\Microsoft\Windows\CurrentVersion\Internet Settings")
subCount, valueCount, lastModified = QueryInfoKey(aKey)
for i in range(valueCount):
try:
n,v,t = EnumValue(aKey,i)
if n == 'ProxyEnable':
return v and True or False
except EnvironmentError:
break
CloseKey(aKey)
def invertProxyEnableState(self):
aReg = ConnectRegistry(None,HKEY_CURRENT_USER)
aKey = OpenKey(aReg, r"Software\Microsoft\Windows\CurrentVersion\Internet Settings", 0, KEY_WRITE)
if self.isProxyEnabled() :
val = 0
else:
val = 1
try:
SetValueEx(aKey,"ProxyEnable",0, REG_DWORD, val)
except EnvironmentError:
print "Encountered problems writing into the Registry..."
CloseKey(aKey)
def updateIcon(self):
if self.isProxyEnabled():
icon=gtk.STOCK_YES
else:
icon=gtk.STOCK_NO
self.trayIcon.set_from_stock(icon)
def checkStatus(self):
self.updateIcon()
return True
def on_right_click(self, data, event_button, event_time):
self.invertProxyEnableState()
self.updateIcon()
if __name__ == '__main__':
proxyNotifier = ProxyNotifier()
gtk.main()
As far as I know, In a Windows environment, if no proxy environment variables are set, proxy settings are obtained from the registry's Internet Settings section. .
Isn't it enough?
Or u can get something useful info from registry:
HKEY_CURRENT_USER\Software\Microsoft\Windows\CurrentVersion\Internet Settings\ProxyServer
Edit:
sorry for don't know how to format comment's source code, I repost it here.
>>> import win32com.client
>>> js = win32com.client.Dispatch('MSScriptControl.ScriptControl')
>>> js.Language = 'JavaScript'
>>> js.AddCode('function add(a, b) {return a+b;}')
>>> js.Run('add', 1, 2)
3