I have Windows 10, with Aptana Studio 3.
- I installed python in it (v.3.5.0) with terminal
- I installed scrapy 1.0
and now, if I try to run a crawler (scrapy) I get this message:
Traceback (most recent call last):
File "c:\python3.5\lib\runpy.py", line 170, in _run_module_as_main
"__main__", mod_spec)
File "c:\python3.5\lib\runpy.py", line 85, in _run_code
exec(code, run_globals)
File "C:\Python3.5\Scripts\scrapy.exe\__main__.py", line 5, in <module>
File "c:\python3.5\lib\site-packages\scrapy\__init__.py", line 48, in <module>
from scrapy.spiders import Spider
File "c:\python3.5\lib\site-packages\scrapy\spiders\__init__.py", line 10, in <module>
from scrapy.http import Request
File "c:\python3.5\lib\site-packages\scrapy\http\__init__.py", line 15, in <module>
from scrapy.http.response.html import HtmlResponse
File "c:\python3.5\lib\site-packages\scrapy\http\response\html.py", line 8, in <module>
from scrapy.http.response.text import TextResponse
File "c:\python3.5\lib\site-packages\scrapy\http\response\text.py", line 13, in <module>
from scrapy.utils.response import get_base_url
File "c:\python3.5\lib\site-packages\scrapy\utils\response.py", line 12, in <module>
from twisted.web import http
File "c:\python3.5\lib\site-packages\twisted\web\http.py", line 92, in <module>
from twisted.internet import interfaces, reactor, protocol, address
File "c:\python3.5\lib\site-packages\twisted\internet\reactor.py", line 38, in <module>
from twisted.internet import default
File "c:\python3.5\lib\site-packages\twisted\internet\default.py", line 56, in <module>
install = _getInstallFunction(platform)
File "c:\python3.5\lib\site-packages\twisted\internet\default.py", line 50, in _getInstallFunction
from twisted.internet.selectreactor import install
File "c:\python3.5\lib\site-packages\twisted\internet\selectreactor.py", line 18, in <module>
from twisted.internet import posixbase
File "c:\python3.5\lib\site-packages\twisted\internet\posixbase.py", line 18, in <module>
from twisted.internet import error, udp, tcp
File "c:\python3.5\lib\site-packages\twisted\internet\udp.py", line 53, in <module>
from twisted.internet import base, defer, address
File "c:\python3.5\lib\site-packages\twisted\internet\base.py", line 23, in <module>
from twisted.internet import fdesc, main, error, abstract, defer, threads
File "c:\python3.5\lib\site-packages\twisted\internet\defer.py", line 29, in <module>
from twisted.python import lockfile, failure
File "c:\python3.5\lib\site-packages\twisted\python\lockfile.py", line 52, in <module>
_open = file
NameError: name 'file' is not defined
So far I get that file() is a python 2.x function and it does not work with python 3.x from this Q: python NameError: name 'file' is not defined
But the problem is that I do not use file() anywhere in my script. And the message appears to reffer to scrapy default files. So what am I supposed to do? Edit the source files of scrapy? That sounds awfull and way above my knowledge level.
So what can I do?
I should install python 2.7 but for some reason (I don't remember) I could not install it properly when I tried
So anyone has a solution for this?
You likely need to update Twisted (to v15.4.0 or something similar), since your version is not Python 3 compatible.
As Matthias pointed out, scrapy is being ported to Python 3. However, the error you have is in lockfile.py from Twisted, not scrapy:
File "c:\python3.5\lib\site-packages\twisted\python\lockfile.py", line 52, in <module>
_open = file
Twisted is a dependency of scrapy. If you look at the history of this file, you'll see that there was a merge that presumably brought in Python 3 compatibility on 9/22/2015. I can tell that you have an older version because only the previous versions have _open = file in that lockfile.py.
All that being said, you probably still need to use Python 2 until the rest of scrapy is updated to be Py3 compatible.
It looks like scrapy is not yet fully ported to Python 3:
https://github.com/scrapy/scrapy/wiki/Python-3-Porting
So yes, use Python 2.
Related
I am using Splunk 8+ and Python 3.6. I am trying to run this https://github.com/0x616c6578/TA-geoip2 plugin.
I get this error message when I look into search.log and try to run the Python command in the log by itself:
C:\Users\strozllc>"C:\Program Files\Splunk\bin\Python3.exe" "C:\Program Files\Splunk\etc\apps\TA-geoip2-main\bin\geoip-command.py
Traceback (most recent call last):
File "C:\Program Files\Splunk\etc\apps\TA-geoip2-main\bin\geoip-command.py", line 8, in <module>
from splunklib.searchcommands import \
File "C:\Program Files\Splunk\etc\apps\TA-geoip2-main\bin\..\lib\splunklib\searchcommands\__init__.py", line 145, in <module>
from .environment import *
File "C:\Program Files\Splunk\etc\apps\TA-geoip2-main\bin\..\lib\splunklib\searchcommands\environment.py", line 20, in <module>
from logging.config import fileConfig
File "C:\Program Files\Splunk\Python-3.7\lib\logging\config.py", line 30, in <module>
import logging.handlers
File "C:\Program Files\Splunk\Python-3.7\lib\logging\handlers.py", line 26, in <module>
import logging, socket, os, pickle, struct, time, re
File "C:\Program Files\Splunk\Python-3.7\lib\socket.py", line 49, in <module>
import _socket
ImportError: Module use of python27.dll conflicts with this version of Python
n.
Huh? I looked at line 8 in geocommand.py and it is from
splunklib.searchcommands import \
dispatch, StreamingCommand, Configuration, Option, validators
I look it C:\Program Files\Splunk\bin, and there is indeed a python27.dll. But how is it getting called?
This is a case of Python 3 trying to run Python 2 code. The latest versions of Splunk (you didn't say which one you're using) only support Python 3 so an error will be thrown when an outdated library file is encountered.
I'd suggest filing an issue on GitHub, but the app appears to be abandonware since it hasn't been touched in a year despite having 4 issues.
Consider forking the code and updating it yourself.
I'm following along a course - Django development to deployment.
After pulling it to Digital Ocean everything else ran smoothly. Until
I tried running python manage.py help
(env) djangoadmin#ubuntu-1:~/pyapps/btre_project_4$ python manage.py help
and I get this error.
Traceback (most recent call last):
File "manage.py", line 21, in <module>
main()
File "manage.py", line 17, in main
execute_from_command_line(sys.argv)
File "/home/djangoadmin/pyapps/env/lib/python3.5/site-packages/django/core/management/__init__.py", line 401, in execute_from_command_line
utility.execute()
File "/home/djangoadmin/pyapps/env/lib/python3.5/site-packages/django/core/management/__init__.py", line 377, in execute
django.setup()
File "/home/djangoadmin/pyapps/env/lib/python3.5/site-packages/django/__init__.py", line 16, in setup
from django.urls import set_script_prefix
File "/home/djangoadmin/pyapps/env/lib/python3.5/site-packages/django/urls/__init__.py", line 1, in <module>
from .base import (
File "/home/djangoadmin/pyapps/env/lib/python3.5/site-packages/django/urls/base.py", line 9, in <module>
from .exceptions import NoReverseMatch, Resolver404
File "/home/djangoadmin/pyapps/env/lib/python3.5/site-packages/django/urls/exceptions.py", line 1, in <module>
from django.http import Http404
File "/home/djangoadmin/pyapps/env/lib/python3.5/site-packages/django/http/__init__.py", line 2, in <module>
from django.http.request import (
File "/home/djangoadmin/pyapps/env/lib/python3.5/site-packages/django/http/request.py", line 10, in <module>
from django.core import signing
File "/home/djangoadmin/pyapps/env/lib/python3.5/site-packages/django/core/signing.py", line 45, in <module>
from django.utils.crypto import constant_time_compare, salted_hmac
File "/home/djangoadmin/pyapps/env/lib/python3.5/site-packages/django/utils/crypto.py", line 6, in <module>
import secrets
ImportError: No module named 'secrets'
I'm a newbie and have been stuck on this for a while. I just want to know what could possibly cause this.
The secrets module was added to Python in version 3.6. Your host is using Python 3.5, hence the secrets module is unavailable. You need a host with Python 3.6+, or a version of Django that doesn't depend on the secrets module
Had the same issue, solved it by specifying the python version I have installed. i.e
$ python3
>>> import secrets
>>> secrets.token_hex(16)
Has anyone tried using newspaper3k python library on AWS Elastic Beanstalk Python 3.4?
I'm getting a strange error, despite images.py existing in the newspaper directory.
Traceback (most recent call last):
File "/opt/python/current/app/application.py", line 12, in <module>
from server.helper import get_connection, requires_auth, validate_schema, error_handler
File "/opt/python/current/app/server/helper.py", line 13, in <module>
from database.user import User
File "/opt/python/current/app/database/user.py", line 11, in <module>
from .post import Post
File "/opt/python/current/app/database/post.py", line 9, in <module>
from newspaper import Article
File "/opt/python/run/venv/lib/python3.4/site-packages/newspaper/__init__.py", line 10,in <module>
from .api import (build, build_article, fulltext, hot, languages,
File "/opt/python/run/venv/lib/python3.4/site-packages/newspaper/api.py", line 14, in <module>
from .article import Article
File "/opt/python/run/venv/lib/python3.4/site-packages/newspaper/article.py", line 14, in <module>
from . import images
ImportError: cannot import name 'images'
I have installed all dependencies (Pillow, lxml, libjpeg etc.) correctly and surprisingly when I ssh to instance it works properly.
Any help would be appreciated, thanks!
I just tried to use the Python module twisted.internet for an XBMC add-on that I am currently writing. When I tried to execute my add-on, I got an exception that I could not understand. I was quite suprised when I noticed that the following lines were causing all the troubles:
import xbmc, xbmcgui, xbmcaddon # Can be removed; doesn't change anything.
from twisted.internet import reactor, defer
Question: How can I use twisted for the development of my XBMC add-on? Any help would be appreciated!
Unfortunately, I am new to both, XBMC and twisted and there are little more details that I can provide. I think that the addon is not executed in the main thread. I am using XBMC 11 for testing at the moment. (Update: I am using the most recent Debian testing packages of XBMC, Python and twisted. Importing and using the module works from regular scripts and from an XBMC service, but not from XBMC scripts.) The stacktrace from the xbmc.log file is:
File "/home/markus/.xbmc/addons/script.marmay.wol/timewol.py", line 2, in <module>
from twisted.internet import reactor, defer
File "/usr/lib/python2.7/dist-packages/twisted/internet/reactor.py", line 37, in <module>
from twisted.internet import default
File "/usr/lib/python2.7/dist-packages/twisted/internet/default.py", line 50, in <module>
install = _getInstallFunction(platform)
File "/usr/lib/python2.7/dist-packages/twisted/internet/default.py", line 44, in _getInstallFunction
from twisted.internet.pollreactor import install
File "/usr/lib/python2.7/dist-packages/twisted/internet/pollreactor.py", line 19, in <module>
from zope.interface import implements
File "/usr/lib/python2.7/dist-packages/zope/interface/__init__.py", line 58, in <module>
_wire()
File "/usr/lib/python2.7/dist-packages/zope/interface/interface.py", line 805, in _wire
classImplements(Attribute, IAttribute)
File "/usr/lib/python2.7/dist-packages/zope/interface/declarations.py", line 447, in classImplements
spec = implementedBy(cls)
File "/usr/lib/python2.7/dist-packages/zope/interface/declarations.py", line 318, in implementedByFallback
if isinstance(spec, Implements):
TypeError: 'NoneType' object is not callable
This is the first relevant error message.
Thanks for your help!
I love ipython, I am learning python now as a replacement of R for data analysis. However at promt I get the following message:
$ ipython --qtconsole
Traceback (most recent call last):
File "/Library/Frameworks/EPD64.framework/Versions/Current/bin/ipython", line 5, in <module>
from IPython.frontend.terminal.ipapp import launch_new_instance
File "/Library/Frameworks/EPD64.framework/Versions/7.2/lib/python2.7/site-packages/IPython/__init__.py", line 46, in <module>
from .frontend.terminal.embed import embed
File "/Library/Frameworks/EPD64.framework/Versions/7.2/lib/python2.7/site-packages/IPython/frontend/terminal/embed.py", line 39, in <module>
from IPython.frontend.terminal.interactiveshell import TerminalInteractiveShell
File "/Library/Frameworks/EPD64.framework/Versions/7.2/lib/python2.7/site-packages/IPython/frontend/terminal/interactiveshell.py", line 33, in <module>
from IPython.core.interactiveshell import InteractiveShell, InteractiveShellABC
File "/Library/Frameworks/EPD64.framework/Versions/7.2/lib/python2.7/site-packages/IPython/core/interactiveshell.py", line 44, in <module>
from IPython.core import prefilter
File "/Library/Frameworks/EPD64.framework/Versions/7.2/lib/python2.7/site-packages/IPython/core/prefilter.py", line 48, in <module>
from IPython.utils.traitlets import (
ImportError: cannot import name CRegExp
Ipython worked just fine a week ago and now I get this error. Can anybody suggest what might be causing this?
As #favoretti mentioned, this is an import issue in utils.traitlets.
I would recommend upgrading to EPD 7.3 and see if that fixes the issue. I feel bad suggesting a "brush it under the carpet" solution but if you're just starting out with python delving into the bowels of IPython might be a bit of a headache.