For some reason I'm getting a Trace/BPT trap error when calling urllib.urlopen. I've tried both urllib and urllib2 with identical results. Here is the code which throws the error:
def get_url(url):
from urllib2 import urlopen
if not url or not url.startswith('http://'): return None
return urlopen(url).read() # FIXME!
I should add that this code is running on a CherryPy webserver with web.py.
Someone requested a traceback. Unfortunately, there is none. Trace/BPT trap is outputted to the terminal and the process terminates. E.g.
dloewenherz#andros project $ sudo ./index.py 80
http://0.0.0.0:80/
# Here I visit the page which contains the get_url(url) method
Trace/BPT trap
dloewenherz#andros project $
Edit: I am running OS X 10.6.2, web.py 0.33, Python 2.6.2, and CherryPy 3.1.2.
Adding the following lines to the top of the main file solved the problem:
import urllib2
urllib2.install_opener(urllib2.build_opener())
In other words, it is not enough to import the urllib2 module but you actually need to create the opener in the main thread.
Are you running this under OS X 10.6? Apparently threads and importing modules for the first time does not play well together there. See if you can't import urllib2 outside of the thread?
There are a few more details in the following thread: Trace/BPT trap with Python threading module
I'd try either moving the import of urllib to the top of the same file or, since it seems to be a problem only with importing a module for the first time in a thread, import it somewhere else as well, like in the same file as your main() function.
Edit: Which versions of OS X, Python, CherryPy and web.py are you running? I'm using OS X 10.5.8, Python 2.6, CherryPy 3.1.2 and web.py 0.33 and can't reproduce your problem using the below code:
import web
urls = (
'/', 'index'
)
app = web.application(urls, globals())
class index:
def GET(self):
from urllib2 import urlopen
return urlopen("http://google.se/").read()
if __name__ == "__main__": app.run()
$ sudo python index.py 80
http://0.0.0.0:80/
127.0.0.1:59601 - - [08/Nov/2009 09:46:40] "HTTP/1.1 GET /" - 200 OK
127.0.0.1:59604 - - [08/Nov/2009 09:46:40] "HTTP/1.1 GET /extern_js/f/CgJzdhICc2UgACswCjhBQB0sKzAOOAksKzAYOAQsKzAlOMmIASwrMCY4BSwrMCc4Aiw/dDWkSd2jmF8.js" - 404 Not Found
127.0.0.1:59601 - - [08/Nov/2009 09:46:40] "HTTP/1.1 GET /logos/elmo-hp.gif" - 404 Not Found
127.0.0.1:59601 - - [08/Nov/2009 09:46:40] "HTTP/1.1 GET /images/nav_logo7.png" - 404 Not Found
Is this code enough to reproduce the problem on your end? If not, I need more information in order to be of help.
Related
Environment:
Python 3.7.7
Windows 10 64bits
Purpose of the code:
Display an HTML report of my software activity. It uses http.server module loaded by file Report.py and the data report are extracted by index.py.
I have a script myscript.py which launches the server by calling a method StartReportTool() inside the module Report.py.
Report.py launch the server and load the index.py.
Files & Folder tree:
- myscript.py <= call the method in module Report.py to launch web server
- /report
- /report/Report.py <= launch the web server
- /report/index.py <= extract the data and display them in html
Source code:
myscript.py:
def report():
from report import Report
Report.StartReportTool()
report()
/report/Report.py
#coding:utf-8
import http.server
import webbrowser
def StartReportTool():
port=8888
address=("",port)
server=http.server.HTTPServer
handler=http.server.CGIHTTPRequestHandler
handler.cgi_directories=["/"]
httpd=server(address,handler)
print(f"Report tool server started on port {port}")
webbrowser.open('http://localhost:8888/index.py', new=2)
httpd.serve_forever()
index.py:
#coding:utf-8
import cgi
import cgitb
cgitb.enable()
form = cgi.FieldStorage()
print("Content-type: text/html; charset=utf-8\n")
html=f"""
<!DOCTYPE html>
My html code
"""
Problem:
When I run myscript.py which launches my http.server, my browser open url 'http://localhost:8888/index.py' and show error 404:
Error response
Error code: 404
Message: No such CGI script ('//index.py').
Error code explanation: HTTPStatus.NOT_FOUND - Nothing matches the given URI.
After investigation, I realized everything is fine when the server is launched by its own script Report.py:
#coding:utf-8
import http.server
import webbrowser
def StartReportTool():
port=8888
address=("",port)
server=http.server.HTTPServer
handler=http.server.CGIHTTPRequestHandler
handler.cgi_directories=["/"]
httpd=server(address,handler)
print(f"Report tool server started on port {port}")
webbrowser.open('http://localhost:8888/index.py', new=2)
httpd.serve_forever()
StartReportTool() # <======= Here is the line of code which launch the server itself inside the same module
You noticed here the last line of code StartReportTool() which calls the method inside the module itself. And it is working fine by this way. My index.py is loaded correctly.
The problem comes when the method StartReportTool() is called from outside the method.
I don't understand the reasons for the issue. Does anyone understand the source of the problem, please?
I am currently running Python 3.8.0.
I have a program but when executing it, I am getting a ModuleNotFoundError and it points to 'urllib2'
I've tried to fix this issue as this seems to be a common thing, but cannot figure out where I am getting stuck. Any help would be appreciated. Below are the errors and the Python code.
#!/usr/bin/env python
from __future__ import print_function
from urllib.request import urlopen
import json
def main():
ip = json.loads(urlopen('https://ifconfig.co/json').read())['ip']
print(json.dumps({'ip': ip + '/32'}))
if __name__ == '__main__':
main()
#!/usr/bin/python3
import cgi
import cgitb
import urllib.request
import os
import sys
def enco_print(string="", encoding = "utf8"):
sys.stdout.buffer.write(string.encode(encoding) + b"\n")
cgitb.enable()
form = cgi.FieldStorage(endcoding="utf8")
name_name= form.getvalue("name")
url_name = form.getvalue("url")
response = urllib.request.urlopen(str(url_name))
html = response.read().decode("utf8")
if not os.path.exists("gecrwalt"):
os.mkdir("gecrwalt")
with open("/gecrwalt/" + str(url_name) + ".html", "w", endcoding="utf8")
as f:
f.write(str(html))
When I try to run this script, I get 500 Status Error on my Website. I can´t see what´s wrong with this code.
I´m very thankful for help.
There are a few typos where you wrote endcoding instead of encoding.
The last segment here
with open("/gecrwalt/" + str(url_name) + ".html", "w", endcoding="utf8")
as f:
f.write(str(html))
has broken indentation, not sure if this was due to a copy-paste error here on Stackoverflow.
Another issue here is that (if I understood your code correlty) url_name will contain a complete URL like http://example.com, which will result in an error because that filename is invalid. You will have to come up with some schema to safely store these files, urlencode the URL or take a hash of the URL. Your save path also starts with /, not sure if intentational, starts at the file system root.
Changing the typo and the last bit has worked for me in a quick test:
with open("gecrwalt/something.html", "w", endcoding="utf8") as f:
f.write(str(html))
Debugging hint: I started a local Python webserver process with this command (from here)
python3 -m http.server --bind localhost --cgi 8000
Accessing http://localhost:8000/cgi-bin/filename.py will show you all errors that occur, not sure about the webserver you are currently using (there should be error logs somewhere I guess).
I'm really enjoying Bottle so far, but the fact that I have to CTRL+C out of the server and restart it every time I make a code change is a big hit on my productivity. I've thought about using Watchdog to keep track of files changing then restarting the server, but how can I do that when the bottle.run function is blocking.
Running the server from an external script that watches for file changes seems like a lot of work to set up. I'd think this was a universal issue for Bottle, CherryPy and etcetera developers.
Thanks for your solutions to the issue!
Check out from the tutorial a section entitled "Auto Reloading"
During development, you have to restart the server a lot to test your
recent changes. The auto reloader can do this for you. Every time you
edit a module file, the reloader restarts the server process and loads
the newest version of your code.
This gives the following example:
from bottle import run
run(reloader=True)
With
run(reloader=True)
there are situations where it does not reload like when the import is inside a def. To force a reload I used
subprocess.call(['touch', 'mainpgm.py'])
and it reloads fine in linux.
Use reloader=True in run(). Keep in mind that in windows this must be under if __name__ == "__main__": due to the way the multiprocessing module works.
from bottle import run
if __name__ == "__main__":
run(reloader=True)
These scripts should do what you are looking for.
AUTOLOAD.PY
import os
def cherche(dir):
FichList = [ f for f in os.listdir(dir) if os.path.isfile(os.path.join(dir,f)) ]
return FichList
def read_file(file):
f = open(file,"r")
R=f.read()
f.close()
return R
def load_html(dir="pages"):
FL = cherche(dir)
R={}
for f in FL:
if f.split('.')[1]=="html":
BUFF = read_file(dir+"/"+f)
R[f.split('.')[0]] = BUFF
return R
MAIN.PY
# -*- coding: utf-8 -*-
#Version 1.0 00:37
import sys
reload(sys)
sys.setdefaultencoding("utf-8")
import datetime
import ast
from bottle import route, run, template, get, post, request, response, static_file, redirect
#AUTOLOAD by LAGVIDILO
import autoload
pages = autoload.load_html()
BUFF = ""
for key,i in pages.iteritems():
BUFF=BUFF+"#get('/"+key+"')\n"
BUFF=BUFF+"def "+key+"():\n"
BUFF=BUFF+" return "+pages[key]+"\n"
print "=====\n",BUFF,"\n====="
exec(BUFF)
run(host='localhost', port=8000, reloader=True)
I have a small Python web application using the Cherrypy framework. I am by no means an expert in web servers.
I got Cherrypy working with Apache using mod_python on our Ubuntu server. This time, however, I have to use Windows 2003 and IIS 6.0 to host my site.
The site runs perfectly as a stand alone server - I am just so lost when it comes to getting IIS running. I have spent the past day Googling and blindly trying any and everything to get this running.
I have all the various tools installed that websites have told me to (Python 2.6, CherrpyPy 3, ISAPI-WSGI, PyWin32) and have read all the documentation I can. This blog was the most helpful:
http://whatschrisdoing.com/blog/2008/07/10/turbogears-isapi-wsgi-iis/
But I am still lost as to what I need to run my site. I can't find any thorough examples or how-to's to even start with. I hope someone here can help!
Cheers.
I run CherryPy behind my IIS sites. There are several tricks to get it to work.
When running as the IIS Worker Process identity, you won't have the same permissions as you do when you run the site from your user process. Things will break. In particular, anything that wants to write to the file system will probably not work without some tweaking.
If you're using setuptools, you probably want to install your components with the -Z option (unzips all eggs).
Use win32traceutil to track down problems. Be sure that in your hook script that you're importing win32traceutil. Then, when you're attempting to access the web site, if anything goes wrong, make sure it gets printed to standard out, it'll get logged to the trace utility. Use 'python -m win32traceutil' to see the output from the trace.
It's important to understand the basic process to get an ISAPI application running. I suggest first getting a hello-world WSGI application running under ISAPI_WSGI. Here's an early version of a hook script I used to validate that I was getting CherryPy to work with my web server.
#!python
"""
Things to remember:
easy_install munges permissions on zip eggs.
anything that's installed in a user folder (i.e. setup develop) will probably not work.
There may still exist an issue with static files.
"""
import sys
import os
import isapi_wsgi
# change this to '/myapp' to have the site installed to only a virtual
# directory of the site.
site_root = '/'
if hasattr(sys, "isapidllhandle"):
import win32traceutil
appdir = os.path.dirname(__file__)
egg_cache = os.path.join(appdir, 'egg-tmp')
if not os.path.exists(egg_cache):
os.makedirs(egg_cache)
os.environ['PYTHON_EGG_CACHE'] = egg_cache
os.chdir(appdir)
import cherrypy
import traceback
class Root(object):
#cherrypy.expose
def index(self):
return 'Hai Werld'
def setup_application():
print "starting cherrypy application server"
#app_root = os.path.dirname(__file__)
#sys.path.append(app_root)
app = cherrypy.tree.mount(Root(), site_root)
print "successfully set up the application"
return app
def __ExtensionFactory__():
"The entry point for when the ISAPIDLL is triggered"
try:
# import the wsgi app creator
app = setup_application()
return isapi_wsgi.ISAPISimpleHandler(app)
except:
import traceback
traceback.print_exc()
f = open(os.path.join(appdir, 'critical error.txt'), 'w')
traceback.print_exc(file=f)
f.close()
def install_virtual_dir():
import isapi.install
params = isapi.install.ISAPIParameters()
# Setup the virtual directories - this is a list of directories our
# extension uses - in this case only 1.
# Each extension has a "script map" - this is the mapping of ISAPI
# extensions.
sm = [
isapi.install.ScriptMapParams(Extension="*", Flags=0)
]
vd = isapi.install.VirtualDirParameters(
Server="CherryPy Web Server",
Name=site_root,
Description = "CherryPy Application",
ScriptMaps = sm,
ScriptMapUpdate = "end",
)
params.VirtualDirs = [vd]
isapi.install.HandleCommandLine(params)
if __name__=='__main__':
# If run from the command-line, install ourselves.
install_virtual_dir()
This script does several things. It (a) acts as the installer, installing itself into IIS [install_virtual_dir], (b) contains the entry point when IIS loads the DLL [__ExtensionFactory__], and (c) it creates the CherryPy WSGI instance consumed by the ISAPI handler [setup_application].
If you place this in your \inetpub\cherrypy directory and run it, it will attempt to install itself to the root of your IIS web site named "CherryPy Web Server".
You're also welcome to take a look at my production web site code, which has refactored all of this into different modules.
OK, I got it working. Thanks to Jason and all his help. I needed to call
cherrypy.config.update({
'tools.sessions.on': True
})
return cherrypy.tree.mount(Root(), '/', config=path_to_config)
I had this in the config file under [/] but for some reason it did not like that. Now I can get my web app up and running - then I think I will try and work out why it needs that config update and doesn't like the config file I have...