scrapy fail to cooperate with python3.7 - python

I failed to install twisted by pip command, so I manually downloaded the .whl file and got it installed( version 18.7.0). Only after i did that, my laptop could install scrapy; however, it seems that the twisted package is not compatible with python 3.7 and it keeps saying "syntax error"
I have tried some method posted on the Github about this issue(https://github.com/scrapy/scrapy/issues/3143), but none of them solve it. I wonder whether I need to shift to python 3.6 or not? cause my python spider can only setup it's downloader and cannot parse webpages
Could anyone please give me some advise?

Yes. Twisted does not yet support Python 3.7. Try Python 3.6 or earlier.

One of the comments from the issue helped me:
pip install git+https://github.com/scrapy/scrapy#master --no-dependencies --upgrade

Related

Cython_VST_loader not installing through PIP

I am using anaconda3 with Python Version 3.8.
I followed the installation instructions for Cython VST loader per the package’s website (https://pypi.org/project/cython-vst-loader/).
This entailed:
Updating Anaconda compiler (from https://anaconda.org/conda-forge/compilers , 2nd line of code worked)
PIP installing ‘make’ via Anaconda command line
Not sure how to install/update headers, this may or may not be causing the error.
When I attempt to pip install cython_vst_loader in the command line, I receive the following deluge of errors.
Any help or resources on how to address this problem would be greatly appreciated, thank you!
With the new 0.3.6 version of the loader (released 15 minutes ago), Windows is supported and binaries are provided. Also, project goals are clarified in Readme, you might want to check them as well https://github.com/hq9000/cython-vst-loader/blob/master/README.md
You can install the new version normally with pip.

ImportError : cannot import name '_win32stdio'

I am working with Scrapy framework to scrap out data from website, but getting the following error in command prompt:
ImportError: cannot import name '_win32stdio'
Traceback is attached as a screenshot.
Kindly revert if require directory structure of my program's directory.
Scrapy can work with Python 3 on windows if you make some minor adjustments:
Copy the _win32stdio and _pollingfile to the appropriate directory under site-packages. Namely, twisted-dir\internet. Download these from https://github.com/twisted/twisted/tree/trunk/twisted/internet
pip install pypiwin32
Granted, this is based on my personal experience. Because the repository will certainly change in the future, readers should beware the age of this answer.
Update: the twisted-win package is no longer required because the appropriate files are now included in the twisted package.
I have gone through the same. I have resolved by updating the twisted package
pip install --upgrade twisted
or
pip uninstall twisted and pip install twisted
get whl of twisted (from below link) according to your os and py version and you are good to go!
https://www.lfd.uci.edu/~gohlke/pythonlibs/#twisted

"ImportError: No module named twilio.rest"

I have installed python 2.7.10 with PATH access and correctly installed twilio. However, when I try to execute a code I get this error message
Traceback (most recent call last):
File "C:\Users\tmslvo\Google Drive\Desktop\send text.py", line 1, in <module>
from twilio.rest import TwilioRestClient
ImportError: No module named twilio.rest
Now I read that a reason might be that python can't find the twilio package so I tried the
which -a python
which -a twilio
commands (in my Windows command prompt) in which case I get
'which' is not recognized as an internal or external command,
operable program or batch file.
Does anybody have an idea of what I'm doing wrong?
Thank you!
Twilio developer evangelist here.
I think your problem will be that somehow when you installed the library, it failed silently(?). A few things to keep in mind:
When installing Python libraries, always make sure you use pip.
Also, check that none of your files within the project are actually called twilio.py as this will conflict with the actual library.
Check that you're using the version of Python you think you're using by running python --version
All that failing, run the installation again, and all going well (without errors), you should be able to test it quickly with the following code.
import twilio
import twilio.rest
try:
client = twilio.rest.TwilioRestClient(account_sid, auth_token)
message = client.messages.create(
body="Hello World",
to="+14159352345",
from_="+14158141829"
)
except twilio.TwilioRestException as e:
print e
try this: sudo pip3 install twilio --upgrade
I had this problem as well.
In my case, I had named my file twilio.py and that is what caused the error.
Renaming the file to send_sms.py ( or any other name of your choice) will resolve the issue!
Close and then relunch all IDLE instances.
This sounds obvious but it worked for me, since the installations of components were successful
I ran into this same issue. I had used easy_install instead of pip to install twilio which was the problem. To fix this I ran pip uninstall twilio and reinstalled using pip.
rename file name other than twilio.py
EX:send_sms.py
A bit late to the party here but I also ran into this issue.
After some trial and error, it looks like it was due to the pip version I was using. I originally used -
pip3 install twilio.
Now I'm unsure of the underlying reason why this did not work, but it seems that pip3 does not encompass all versions of python 3.x? Using
pip3 list and
pip3.8 list
I noticed I had the twilio module for pip3 but not for pip 3.8.
I used the following and was able to solve the issue
pip3.8 install twilio.
I used pip3.8 because that matched the python3.8 version that I am using.
Pycharm user:
Macs (mid 2017) come with python 2.6 and 2.7 installed. PyCharm uses by default 2.6. When you install twilio (Pip install) the module is installed in python version 2.7. So, when you try to run twilio from PyCharm you get
ImportError: No module named twilio.rest
Solution: change the python interpreter in PyCharm. Go to preferences > project interpreter and from the drop menu Project Interpreter choose python 2.7
I think your pip is not configured properly . You may be getting succefuuly installed message but it is not install where it should be. try pip install --user i am sure it will work for you. pip install may work fine only in virtualenvironment without any config.Try pip install --user package name
#iosCurator
I had first installed twilio with the easy_intall tool
I followed the steps below:
Uninstall twilio with the command pip uninstall twillo
Install twilio with the command pip install twilio
Close the python IDLE and relaunch it.
For the windows user,
I have suggested, pip3 install twilio
Follow these steps (on mac):
Shift + Command + P
search: Configure Language Specific Setting
search: Python
add: "code-runner.runInTerminal": true
That's it!
Ask me any question about it by:
My LinkedIn
I ran into this issue when using poetry for my dependency management. Poetry doesn't recognise it as an existing package yet, hence it won't run your code unless you try the poetry+pip way.
there will be 2 reasons for this
1.make sure you kept right path for python files in environment location
2.install twilio
commands:
1.pip3 install twilio
(or)
pip install twilio
2.python otpv.py

Installing AutoIt for Robot framework

I am trying to install the AutoIt library for Robot Framework, I've tried using the pip command command pip install robotframework-autoitlibrary but that didn't work. I then downloaded the AutoIt library and installed it but I can't open it.
Why can't I use the library?
There are two reasons that your install may not be working correctly:
1. You haven't installed ActivePython, which I don't think you have.
2. You need the 32bit version of Python installed or AutoIt won't actually work.
If these fixes don't work, comment and I will try to help further
Step1. Please make sure you have installed autoit-v3-setup.exe, pywin32-220.win-amd64-py2.7.exe.
Step2. You have to run "python setup.py install" in a Administrator cmd while installing AutoItLibrary-1.1.
Step.3 If you meet any porblem, please add the error message, that will be very useful.
Hope this will help you.
I had the same issue but resolved by following the below steps.
Download the file from below path "https://pypi.org/project/robotframework-autoitlibrary/#files"
Unzip and run the "setup.py" file from the command prompt as administrator by using the below command "python setup.py install".
Open your ride editor and import the library with the name "AutoItLibrary".
It worked for me with below versions
robotframework==3.1.2
robotframework-autoitlibrary==1.2.5
robotframework-seleniumlibrary==4.3.0
robotframework-ride==1.7.4.1
selenium 3.141.0
I had some troubles installing the lastest version 1.2.2 which supports python 3 on windows and it was due to blank spaces in the path as Program Files : https://code.google.com/archive/p/robotframework-autoitlibrary/issues/30 I hope it helps somebody, the fix is basically quotes around the path

import libtorrent not working

I installed libtorrent-rasterbar on a mac with brew. After I installed I check if it was good installed with the command:
brew install libtorrent-rasterbar
And I get:
Warning: libtorrent-rasterbar-0.16.10 already installed
So installation looks to be ok.
If I go to python and type "import libtorrent" y get an error.
Also if I type:
>>>help('modules')
I dont see libtorrent in the list.
What I'm doing wrong?
Brew has nothing to do with Python. It'll just install system libraries. If you want to install libraries/modules for Python, you should use pip (recommended) or easy_install.
I couldn't find very good Python support for libtorrent in particular, although there is great support for other torrent libraries like PyTorrent.
If you must use libtorrent-rasterbar, there's a great tutorial on how to do it on Super User

Categories