I'm following the Diazo Quickstart instructions . I start the proxy and issue a request, I get a response but I'm not sure the response is what I'm meant to be getting.
The Quickstart reads :
The idea is to run a local webserver that applies the Diazo theme to a
response coming from an existing website, either locally or somewhere
on the internet
... only that seems inconsistent with this part of the proxy.ini contained in the Quickstart guide ...
# Proxy http://diazo.org as the content
[app:content]
use = egg:Paste#proxy
address = http://diazo.org/
suppress_http_headers = accept-encoding
... after seeing that I thought what was going to happen was that the content from diazo.org would be embedded within the theme.html and with the theme.css applied to that content .
What actually happens (as far as I can tell) is that if I request ...
http://localhost:8000
... I receive an unaltered copy of the content at http://diazo.org.
On the other hand if I amend proxy.ini so that the app:content reads as follows ...
# Proxy http://diazo.org as the content
[app:content]
use = egg:Paste#proxy
address = http://python.org/
suppress_http_headers = accept-encoding
I don't get the content of python.org sent to the browser, I get the content of the theme.html file !
The proxy.ini is identical to the once shown in the QuickStart guide except I'm using port 8000 and as mentioned above I've changed the address within the app:content section for the purposes of testing.
So my question is what am I meant to see if I complete the steps in the QuickStart document ? And if it's meant to be a slightly altered version of the diazo.org content (as I suspect) why don't I see the same alterations when I change the target for the proxy ?
modify u rules.xml
<replace css:theme-children="#content" css:content-children=".container" />
Related
I am working on implementing the gRPC generated code for the flatbuffers library, and I've noticed that the python gRPC library doesn't use the user set content-type. According to the documentation the user can set whatever type they want in the Content-Type → "content-type" "application/grpc" [("+proto" / "+json" / {custom})] section. I've been trying to add simply send a +flatbuffers which breaks the go server and the python client (trying to make sure that all the languages can work well without any issues between them).
example:
Go lang to Go lang
:method POST
:scheme http
:path /models.Greeter/SayHello
:authority localhost:3000
content-type application/grpc+flatbuffers
user-agent grpc-go/1.35.0
te trailers
grpc-timeout 9997779u
As you can see if its being called from go lang it doesn't have issues however when I call the server from the python client it clearly doesn't append the +flatbuffers suffix
python to Go lang
:scheme http
:method POST
:authority localhost:3000
:path /models.Greeter/SayHello
te trailers
content-type application/grpc
user-agent grpc-python/1.35.0 grpc-c/14.0.0 (osx; chttp2)
steps taken:
1- I've tried adding options to the channel:
with grpc.insecure_channel('localhost:' + args.port, options=[('content-type', 'application/grpc+flatbuffers')]) as channel:
2- I've tried adding it on to the with_call according to https://github.com/grpc/grpc/blob/v1.35.0/examples/python/metadata/metadata_client.py
with_call(hello_request, metadata=[('content-type', 'application/grpc+flatbuffers')], compression=None)
however both options fail.
am I missing something here?
I'm trying out snakebite. I started the following client:
from snakebite.client import Client
client = Client("my.host.com", 8020, effective_user='datascientist')
First, I tried to list the users directory:
for x in client.ls(['/user/datascientist']):
print x
This worked nicely and printed couple of dictionaries; one for each item in the directory. One of the items is a file foobar.txt which I'd like to see. To that end, I believe I should use Client.cat:
for cat in client.cat(['/user/datascientist/da-foobar.txt',]):
print(cat)
for item in cat:
print(item)
However, this didn't work. I got the following error message:
ConnectionFailureException: Failure to connect to data node at (10.XXX.YYY.ZZZ:50010)
What am I doing wrongly?
BTW: using PyWebHdfsClient from pywebhdfs.webhdfs I managed to see the file by starting a client with the same address but with port 50070. I don't know whether this is relevant or not.
Edit 1: I also tried to use snakebite.client.Client.text and got the same error. I guess this is not surprising.
BTW, the file's content is my file is this\ntest file.
I found a/the solution. It seems like the listing operation can be accomplished on the name-node alone. In contrast, the printing of the text file needs to access the data-nodes! By instantiating the client as follows
client = Client("stage-gap-namenode-2.srv.glispa.com", 8020, effective_user='datascientist',
use_datanode_hostname=True)
the cat operation works as it is not using the internal IP, but the hostname. I summarized a minimal example.
So currently I'm having trouble with how to set the instance of my program to use a different proxy for all Internet related tasks; and I'm also having an error that stating that it: Failed to parse.
Now with the error, I'm not quite sure on the correct approach to fix it since I'm pulling information out of a .dat file; though I believe it may be an encoding error.
Anyway, now with the proxies, I've looked at other posts and submissions for awhile now, though it's still not running properly. I also see posts relating to urllib2 all the time . . . and that's not what I want.
This is basically what I'm trying proxy wise:
proxy = [ . . . ]
os.environ['https_proxy'] = 'http://' + proxy
And this is also the error that follows:
Failed to parse: x.xxx.xxx.xxx:xxxx
Overall thanks for your help, I appreciate it.
As far as I know, there's no magic bullet to change proxy settings of a "whole program". You have to find out which modules you are using (or the module that some other module you are using directly is using - e.g. pandas uses urllib2) and pass the proxy configuration. Below are two examples of the most commonly used modules: urllib2 and requests
Example of using proxies in urllib2 (on a module level):
proxy = {"http":"10.111.111.111:8080", "https":"10.111.111.222:8080"}
opener = urllib2.build_opener(urllib2.ProxyHandler(proxy))
urllib2.install_opener(opener)
resp = urllib2.urlopen('http://www.google.com')
Example of using proxies in requests (on the function call level):
proxy = {"http":"10.111.111.111:8080", "https":"10.111.111.222:8080"}
resp = requests.get('http://www.google.com', proxies = proxy)
The reason why I was receiving the error mentioned above is because: when retrieving the proxies from a .dat file, I wasn't checking for '\n' or ' ' (the ' ' is an example if your data includes spaces, though obviously proxy strings do not).
So an example would be to use .strip after your retrieved information; such as:
fileList = []
with open('[ . . . ].dat') as f:
for line in f:
fileList.append(line.strip('\n').strip(' '))
return fileList[x]
I am using pyramid web framework to build a website. I keep getting this warning in chrome console:
Resource interpreted as Font but transferred with MIME type application/octet-stream: "http:static/images/fonts/font.woff".
How do I get rid of this warning message?
I have configured static files to be served using add_static_view
I can think of a way to do this by adding a subscriber function for responses that checks if the path ends in .woff and setting the response header to application/x-font-woff. But it does not look like a clean solution. Is there a way to tell Pyramid to do it through some setting.
Pyramid uses the standard mimetypes module to guess the mimetype based on the extension. It calls:
mimetypes.guess_type(path, strict=False)
The module looks in the Windows registry if on that platform, and in the following locations for mimetype lists:
knownfiles = [
"/etc/mime.types",
"/etc/httpd/mime.types", # Mac OS X
"/etc/httpd/conf/mime.types", # Apache
"/etc/apache/mime.types", # Apache 1
"/etc/apache2/mime.types", # Apache 2
"/usr/local/etc/httpd/conf/mime.types",
"/usr/local/lib/netscape/mime.types",
"/usr/local/etc/httpd/conf/mime.types", # Apache 1.2
"/usr/local/etc/mime.types", # Apache 1.3
]
You can either extend one of those files, or create your own file and add it to the module using the .init() function.
The file format is simple, just list the mimetype, then some whitespace, then a space-separated list of extensions:
application/x-font-woff woff
Simply add this following code where your Pyramid web app gets initialized.
import mimetypes
mimetypes.add_type('application/x-font-woff', '.woff')
For instance, I have added it in my webapp.py file, which gets called the first time the server gets hit with a request.
I want to setup a proxy for openlayers to use so I followed these steps:
Downloaded the proxy.cgi file from the OpenLayers web site:
http://trac.osgeo.org/openlayers/browser/trunk/openlayers/examples/proxy.cgi
Modify the proxy.cgi file to include my domain in the allowedHosts list:
allowedHosts = ['localhost:6901']
Copy the proxy.cgi file to the following folder:
$TOMCAT_PATH$/webapps/yourApp/WEB-INF/cgi/
Modify the file web.xml of your web app by adding the sections below. You find the file at
$TOMCAT_PATH$/webapps/yourApp/WEB-INF/web.xml
Comment: In case the web.xml file doesn’t exist for your webapp, just create it yourself or copy it from another webapp and modify it. (created!)
Comment: the “param-value” for the “executable” parameter has to contain the path to your Python installation. (it does!)
<servlet>
<servlet-name>cgi</servlet-name>
<servlet-class>org.apache.catalina.servlets.CGIServlet</servlet-class>
<init-param>
<param-name>debug</param-name>
<param-value>0</param-value>
</init-param>
<init-param>
<param-name>cgiPathPrefix</param-name>
<param-value>WEB-INF/cgi</param-value>
</init-param>
<init-param>
<param-name>executable</param-name>
<param-value>c:\python25\python.exe</param-value>
</init-param>
<init-param>
<param-name>passShellEnvironment</param-name>
<param-value>true</param-value>
</init-param>
<load-on-startup>5</load-on-startup>
</servlet>
<servlet-mapping>
<servlet-name>cgi</servlet-name>
<url-pattern>/cgi-bin/*</url-pattern>
</servlet-mapping>
Modify the file context.xml of your web app by adding the element below. You find the file at $TOMCAT_PATH$/webapps/yourApp/META-INF/context.xml
Restart Tomcat
To use the proxy with OpenLayers, just include this single line into your code:
OpenLayers.ProxyHost = "/yourWebApp/cgi-bin/proxy.cgi?url=";
But when I try to use it like:
/webappname/cgi-bin/proxy.cgi?url=labs.metacarta.com
I get this error:
Some unexpected error occurred. Error text was: list index out of range
I think its related with os.environ["REQUEST_METHOD"] but I dont know how its related ..
You're asking for an environment variable that isn't defined.
You need to either catch and handle the exception or use os.environ.get:
try:
methodq = os.environ["REQUEST_METHOD"]
except KeyError:
methodq = "default value"
Or:
methodq = os.environ.get("REQUEST_METHOD", "default value")
You're submitting :
/webappname/cgi-bin/proxy.cgi?url=labs.metacarta.com
But the proxy.cgi script is trying to do this:
host = url.split("/")[2]
Try http://labs.metacarta.com for your url so proxy.cgi has some slashes to split on, or modify it to parse the url a little smarter.
The answer is: You don't install or use cgi proxy on Tomcat.
cgi is for apache server or IIS that are used as a front-end server. Tomcat may sit behind it. The configuration of Apache is detailed in: http://tomcat.apache.org/tomcat-6.0-doc/proxy-howto.html
Be warned that OpenLayers warns that its proxy.cgi is only an example and may not have good enough check to stop it from being exploited, i.e. it may run some malicious script.
If you are serving your OpenLayers client page on Tomcat alone and it contains layers from other GeoServer or Mapserver, you can use proxy servlet and specify it as:
OpenLayers.ProxyHost = "sevlet URL on the server that served this page";
http://wiki.apache.org/tomcat/ServletProxy
https://svn.openxdata.org/Experimental/openXmapper/trunk/gwt-openlayers-server/src/main/java/org/gwtopenmaps/openlayers/server/GwtOpenLayersProxyServlet.java
There are reverse proxy or rewrite sevlet solutions to it, too. Please Google on these.