Python requests-html throwing multiprocessing error - python

I am attempting to move over my web scraping program from using the requests library to the requests-html library to allow me to render the javascript on webpages. On the import of the module this error is thrown:
Traceback (most recent call last):
File "backend2.py", line 2, in <module>
import requests_html
File "/Library/Frameworks/Python.framework/Versions/3.7/lib/python3.7/site-packages/requests_html.py", line 9, in <module>
import pyppeteer
File "/Library/Frameworks/Python.framework/Versions/3.7/lib/python3.7/site-packages/pyppeteer/__init__.py", line 30, in <module>
from pyppeteer.launcher import connect, launch, executablePath # noqa: E402
File "/Library/Frameworks/Python.framework/Versions/3.7/lib/python3.7/site-packages/pyppeteer/launcher.py", line 24, in <module>
from pyppeteer.browser import Browser
File "/Library/Frameworks/Python.framework/Versions/3.7/lib/python3.7/site-packages/pyppeteer/browser.py", line 15, in <module>
from pyppeteer.page import Page
File "/Library/Frameworks/Python.framework/Versions/3.7/lib/python3.7/site-packages/pyppeteer/page.py", line 20, in <module>
from pyppeteer.coverage import Coverage
File "/Library/Frameworks/Python.framework/Versions/3.7/lib/python3.7/site-packages/pyppeteer/coverage.py", line 15, in <module>
from pyppeteer.util import merge_dict
File "/Library/Frameworks/Python.framework/Versions/3.7/lib/python3.7/site-packages/pyppeteer/util.py", line 10, in <module>
from pyppeteer.chromium_downloader import check_chromium, chromium_executable
File "/Library/Frameworks/Python.framework/Versions/3.7/lib/python3.7/site-packages/pyppeteer/chromium_downloader.py", line 15, in <module>
from tqdm import tqdm
File "/Library/Frameworks/Python.framework/Versions/3.7/lib/python3.7/site-packages/tqdm/__init__.py", line 1, in <module>
from ._tqdm import tqdm
File "/Library/Frameworks/Python.framework/Versions/3.7/lib/python3.7/site-packages/tqdm/_tqdm.py", line 75, in <module>
mp_lock = mp.RLock() # multiprocessing lock
AttributeError: module 'multiprocessing' has no attribute 'RLock'
Any help is appreciated!

You are using Python 3.7, but the github of requests-html states that only Python 3.6 is supported (bottom of page). I get all kinds of horrible errors when I try to use Python 3.7 but 3.6 works fine. So, seems strange I know, but please try using 3.6
Even so, as #georgexsh states, there should still be a RLock available for import from multiprocessing in 3.7 so it looks like your error may actually be that one of your code files is called multiprocessing.py or you have your own package called multiprocessing (so you are importing that instead of the module you wanted).

Related

cannot import name 'Mapping' from 'collections' on importing requests

Python Version: Python 3.10.4
PIP Version: pip 22.0.4
So I was trying to make a small project with sockets, I added a feature to upload files but whenever I import requests, it throws this error. Below is the code I ran.
Traceback (most recent call last):
File "C:\Programming\WireUS\test.py", line 1, in <module>
import requests
File "C:\Users\John\AppData\Local\Programs\Python\Python310\lib\site-packages\requests\__init__.py", line 43, in <module>
import urllib3
File "C:\Users\John\AppData\Local\Programs\Python\Python310\lib\site-packages\urllib3\__init__.py", line 8, in <module>
from .connectionpool import (
File "C:\Users\John\AppData\Local\Programs\Python\Python310\lib\site-packages\urllib3\connectionpool.py", line 29, in <module>
from .connection import (
File "C:\Users\John\AppData\Local\Programs\Python\Python310\lib\site-packages\urllib3\connection.py", line 39, in <module>
from .util.ssl_ import (
File "C:\Users\John\AppData\Local\Programs\Python\Python310\lib\site-packages\urllib3\util\__init__.py", line 3, in <module>
from .connection import is_connection_dropped
File "C:\Users\John\AppData\Local\Programs\Python\Python310\lib\site-packages\urllib3\util\connection.py", line 3, in <module>
from .wait import wait_for_read
File "C:\Users\John\AppData\Local\Programs\Python\Python310\lib\site-packages\urllib3\util\wait.py", line 1, in <module>
from .selectors import (
File "C:\Users\John\AppData\Local\Programs\Python\Python310\lib\site-packages\urllib3\util\selectors.py", line 14, in <module>
from collections import namedtuple, Mapping
ImportError: cannot import name 'Mapping' from 'collections' (C:\Users\John\AppData\Local\Programs\Python\Python310\lib\collections\__init__.py)
Even this basic code gives me that error.
import requests
import time
r = request.get("google.com").text
print(r)
time.sleep(999)
As user2357112-supports-monica said, running pip install urllib3 fixes it.
you have to mention the schema (http, ftp,https) of your url :
import requests
import time
r = requests.get("https://google.com").text
print(r)
Just try to edit the selectors.py file
from
from collections import Mapping
to
from collections.abc import Mapping
This is a compatibility issue between different versions of python 3

Import 'transport' error in asyncio attributes

I am trying to make a discord bot, but I get the following weird error where it can not import asyncio.Transport, although I am able to use it normally outside this code.
The code:
import asyncio
import os
import discord
from dotenv import load_dotenv
The error message:
Traceback (most recent call last):
File "dis.py", line 1, in <module>
import asyncio
File "C:\Users\A.Gawish\AppData\Local\Programs\Python\Python36\lib\asyncio\__init__.py", line 21, in <module>
from .base_events import *
File "C:\Users\A.Gawish\AppData\Local\Programs\Python\Python36\lib\asyncio\base_events.py", line 19, in <module>
import inspect
File "C:\Users\A.Gawish\AppData\Local\Programs\Python\Python36\lib\inspect.py", line 36, in <module>
import dis
File "C:\Users\A.Gawish\dis.py", line 3, in <module>
import discord
File "C:\Users\A.Gawish\AppData\Local\Programs\Python\Python36\lib\site-packages\discord\__init__.py", line 23, in <module>
from .client import Client
File "C:\Users\A.Gawish\AppData\Local\Programs\Python\Python36\lib\site-packages\discord\client.py", line 34, in <module>
import aiohttp
File "C:\Users\A.Gawish\AppData\Local\Programs\Python\Python36\lib\site-packages\aiohttp\__init__.py", line 6, in <module>
from .client import (
File "C:\Users\A.Gawish\AppData\Local\Programs\Python\Python36\lib\site-packages\aiohttp\client.py", line 32, in <module>
from . import hdrs, http, payload
File "C:\Users\A.Gawish\AppData\Local\Programs\Python\Python36\lib\site-packages\aiohttp\http.py", line 7, in <module>
from .http_parser import (
File "C:\Users\A.Gawish\AppData\Local\Programs\Python\Python36\lib\site-packages\aiohttp\http_parser.py", line 14, in <module>
from .base_protocol import BaseProtocol
File "C:\Users\A.Gawish\AppData\Local\Programs\Python\Python36\lib\site-packages\aiohttp\base_protocol.py", line 4, in <module>
from .tcp_helpers import tcp_nodelay
File "C:\Users\A.Gawish\AppData\Local\Programs\Python\Python36\lib\site-packages\aiohttp\tcp_helpers.py", line 20, in <module>
def tcp_keepalive(transport: asyncio.Transport) -> None:
AttributeError: module 'asyncio' has no attribute 'Transport'
Your filename was dis.py, which is used in another module. I resolved a similar issue by ensuring the filename is different from any other existing files in Lib
If you look at your error messages, it is indicating Python36. If Transport in asyncio was only released in Python37, you need to change the pathing in whatever program you're using to write the code in as it's looking at Python36.
Testing it as the command prompt would show that your system sees Python37 but your code doesn't. Fix the pathing and it should work then.

Error after updating scikit-learn to version 0.20.1 : cannot import name 'Empty' from 'queue'

I upgraded scikit-learn to version 0.20.1 using anaconda3 after getting an error message concerning balanced_accuracy_score from metrics in scikit-learn. This is apparently not available in version 0.19.
After this upgrade i get the following error:
Traceback (most recent call last):
File "test4.py", line 4, in <module>
from sklearn import metrics
File "C:\Users\Pieter\Anaconda3\lib\site-packages\sklearn\__init__.py", line 6
4, in <module>
from .base import clone
File "C:\Users\Pieter\Anaconda3\lib\site-packages\sklearn\base.py", line 13, i
n <module>
from .utils.fixes import signature
File "C:\Users\Pieter\Anaconda3\lib\site-packages\sklearn\utils\__init__.py",
line 14, in <module>
from . import _joblib
File "C:\Users\Pieter\Anaconda3\lib\site-packages\sklearn\utils\_joblib.py", l
ine 22, in <module>
from ..externals import joblib
File "C:\Users\Pieter\Anaconda3\lib\site-packages\sklearn\externals\joblib\__i
nit__.py", line 119, in <module>
from .parallel import Parallel
File "C:\Users\Pieter\Anaconda3\lib\site-packages\sklearn\externals\joblib\par
allel.py", line 28, in <module>
from ._parallel_backends import (FallbackToBackend, MultiprocessingBackend,
File "C:\Users\Pieter\Anaconda3\lib\site-packages\sklearn\externals\joblib\_pa
rallel_backends.py", line 22, in <module>
from .executor import get_memmapping_executor
File "C:\Users\Pieter\Anaconda3\lib\site-packages\sklearn\externals\joblib\exe
cutor.py", line 14, in <module>
from .externals.loky.reusable_executor import get_reusable_executor
File "C:\Users\Pieter\Anaconda3\lib\site-packages\sklearn\externals\joblib\ext
ernals\loky\__init__.py", line 13, in <module>
from .reusable_executor import get_reusable_executor
File "C:\Users\Pieter\Anaconda3\lib\site-packages\sklearn\externals\joblib\ext
ernals\loky\reusable_executor.py", line 11, in <module>
from .process_executor import ProcessPoolExecutor, EXTRA_QUEUED_CALLS
File "C:\Users\Pieter\Anaconda3\lib\site-packages\sklearn\externals\joblib\ext
ernals\loky\process_executor.py", line 82, in <module>
from .backend.queues import Queue, SimpleQueue, Full
File "C:\Users\Pieter\Anaconda3\lib\site-packages\sklearn\externals\joblib\ext
ernals\loky\backend\queues.py", line 21, in <module>
from multiprocessing.queues import Full
File "C:\Users\Pieter\Anaconda3\lib\multiprocessing\queues.py", line 20, in <m
odule>
from queue import Empty, Full
ImportError: cannot import name 'Empty' from 'queue' (C:\Users\Pieter\Python pro
gs\queue.py)
Are you running this program from "C:\Users\Pieter\Python progs"?
Looks like you have a file named queue.py in "C:\Users\Pieter\Python pro
gs\" which is shadowing the actual module queue in multiprocessing lib. And hence the error.
There are two possible reasons for this:
Maybe this line:
from queue import Empty, Full
has been added in new version, so earlier it did not error even when your custom made queue.py was present.
Maybe you have created your queue.py recently after updating the scikit-learn and the above line was always present. In this case, error should occur on both versions.
Rename your program from queue.py and auto-compiled queue.pyc
to something else and it should work.

ImportError when using scipy.io

So I am having a bit of trouble using scipy. I have been importing data from a .mat file (matlab variables) and up until today it has worked perfectly. I have no idea what changed or when because I have been developing a text - based data format so that I can avoid depending on Matlab for writing my data. Anyways, I am getting a bizzare import error when I import scipy.io
import scipy.io
myData = scipy.io.loadmat('some_data_file')
When I run this I get the following error:
>>> import scipy.io
Traceback (most recent call last):
File "<console>", line 1, in <module>
File "C:\pyzo2014a\lib\site-packages\scipy\io\__init__.py", line 85, in <module>
from .matlab import loadmat, savemat, whosmat, byteordercodes
File "C:\pyzo2014a\lib\site-packages\scipy\io\matlab\__init__.py", line 13, in <module>
from .mio import loadmat, savemat, whosmat
File "C:\pyzo2014a\lib\site-packages\scipy\io\matlab\mio.py", line 13, in <module>
from .mio4 import MatFile4Reader, MatFile4Writer
File "C:\pyzo2014a\lib\site-packages\scipy\io\matlab\mio4.py", line 11, in <module>
import scipy.sparse
File "C:\pyzo2014a\lib\site-packages\scipy\sparse\__init__.py", line 217, in <module>
from .csgraph import cs_graph_components
File "C:\pyzo2014a\lib\site-packages\scipy\sparse\csgraph\__init__.py", line 148, in <module>
from ._shortest_path import shortest_path, floyd_warshall, dijkstra,\
ImportError: No module named 'scipy.sparse.csgraph._shortest_path'
I thought there might be something wrong with my python distribution so I reinstalled it, however the problem persists, and now the installer says it can't find '_shortest_path.py' when it is installing. When I navigate to .../Libs/site-packages/scipy/sparse/csgraph/ I find that indeed there is no module named _shortest_path. I don't understand how this error came about or how anything in my code would change it. Has anyone else come across this problem?
I am using Pyzo 2014 on Windows 7 x64.
Restore shortest_path.pyd in your virust chest
http://www.blendernation.com/2014/06/28/getting-a-virus-warning-with-blender-2-71-heres-why/

Opening ipython from sublime text

I added the following plugin to sublime text 2 which is meant to open the ipython qt console:
import sublime, sublime_plugin
from subprocess import call
# import os
class ipythonQtCommand(sublime_plugin.TextCommand):
def run(self, edit):
call(["start","ipython", "qtconsole", "--pylab", "--ConsoleWidget.font_size=9", "--ConsoleWidget.font_family='Consolas'"],shell=True)
Initially it worked just fine, i.e. the plugin opened an ipython shell. I then added a menu item and a key binding. At some point something must have gone wrong since now I get this error when I run the plugin:
Traceback (most recent call last):
File "<string>", line 1, in <module>
File "C:\Chris\Python27\lib\site-packages\IPython\__init__.py", line 43, in <
odule>
from .config.loader import Config
File "C:\Chris\Python27\lib\site-packages\IPython\config\__init__.py", line 1
, in <module>
from .application import *
File "C:\Chris\Python27\lib\site-packages\IPython\config\application.py", lin
31, in <module>
from IPython.config.configurable import SingletonConfigurable
File "C:\Chris\Python27\lib\site-packages\IPython\config\configurable.py", li
e 31, in <module>
from loader import Config
File "C:\Chris\Python27\lib\site-packages\IPython\config\loader.py", line 32,
in <module>
from IPython.utils.path import filefind, get_ipython_dir
File "C:\Chris\Python27\lib\site-packages\IPython\utils\path.py", line 29, in
<module>
from IPython.utils.process import system
File "C:\Chris\Python27\lib\site-packages\IPython\utils\process.py", line 25,
in <module>
from ._process_win32 import _find_cmd, system, getoutput, AvoidUNCPath, arg
split
File "C:\Chris\Python27\lib\site-packages\IPython\utils\_process_win32.py", l
ne 21, in <module>
import ctypes
File "C:\Chris\Python27\lib\ctypes\__init__.py", line 10, in <module>
from _ctypes import Union, Structure, Array
ImportError: Module use of python26.dll conflicts with this version of Python.
I tested the script by running it line by line in the st2 shell itself and it is the last line call([...]) that causes the error. I find this a little strange since the command works ok when calling it from somewhere else, e.g. ipython itself.
There seems to be a conflict with the st2 native python. In any case I managed to get os.system(...) to work when changing the working directory using os.chdir().

Categories