Importing SparkContext from pyspark running Jupyter Notebook on AWS EC2 [duplicate] - python

import numpy as np
File "/home/anirrudh/.virtualenvs/ml4t2/local/lib/python2.7/site-packages/numpy/__init__.py", line 180, in <module>
from . import add_newdocs
File "/home/anirrudh/.virtualenvs/ml4t2/local/lib/python2.7/site-packages/numpy/add_newdocs.py", line 13, in <module>
from numpy.lib import add_newdoc
File "/home/anirrudh/.virtualenvs/ml4t2/local/lib/python2.7/site-packages/numpy/lib/__init__.py", line 8, in <module>
from .type_check import *
File "/home/anirrudh/.virtualenvs/ml4t2/local/lib/python2.7/site-packages/numpy/lib/type_check.py", line 11, in <module>
import numpy.core.numeric as _nx
File "/home/anirrudh/.virtualenvs/ml4t2/local/lib/python2.7/site-packages/numpy/core/__init__.py", line 57, in <module>
from numpy.testing import Tester
File "/home/anirrudh/.virtualenvs/ml4t2/local/lib/python2.7/site-packages/numpy/testing/__init__.py", line 14, in <module>
from .utils import *
File "/home/anirrudh/.virtualenvs/ml4t2/local/lib/python2.7/site-packages/numpy/testing/utils.py", line 15, in <module>
from tempfile import mkdtemp
File "/usr/lib/python2.7/tempfile.py", line 35, in <module>
from random import Random as _Random
File "/home/anirrudh/OMSCS/Courses/ML4T/ml4t/mc3_p2/random.py", line 5, in <module>
import matplotlib.pyplot as plt
File "/home/anirrudh/.virtualenvs/ml4t2/local/lib/python2.7/site-packages/matplotlib/__init__.py", line 122, in <module>
from matplotlib.cbook import is_string_like, mplDeprecation, dedent, get_label
File "/home/anirrudh/.virtualenvs/ml4t2/local/lib/python2.7/site-packages/matplotlib/cbook.py", line 33, in <module>
import numpy.ma as ma
File "/home/anirrudh/.virtualenvs/ml4t2/local/lib/python2.7/site-packages/numpy/ma/__init__.py", line 44, in <module>
from . import core
File "/home/anirrudh/.virtualenvs/ml4t2/local/lib/python2.7/site-packages/numpy/ma/core.py", line 30, in <module>
import numpy.core.umath as umath
AttributeError: 'module' object has no attribute 'core'
I am working on Ubuntu 14.04. I tried reinstalling numpy, created a new virtualenv, but it isn't helping. I get the same error outside the virtualenv as well. It stopped working suddenly. It was fine until a little while ago.

(Answer posted on behalf of the OP).
Update: Resolved.
I had created a python file for one of my projects name random.py and when numpy is imported, it was calling this file instead of it's own library file hence resulting in the error. I had to delete/rename random.py which I created and that fixed the issue.

Try running it in Python 2 instead of Python 3 - you may have the older version of Spark

This happened to me in Windows with Anaconda. I fixed it by removing the python3 path from the environment variable PATH.

Another common problem here (which I had)...check to make sure you dont have a PYTHONPATH env var pointing to a different version of python site-packages (this happened because I updated my python version and didn't change this...it loads from PYTHONPATH first).

It happened to me in a specific ipynb, once I crated a copy of the notebook with a different name I was able to import numpy.

Related

Getting error at the time of importing numpy library in Python 3.9

I am using win 64 with python 3.9 version. I have also installed numpy version 1.19.3 on my system but I'm still getting the error below.
How do I resolve this error?
from . import _distributor_init
File "C:\Users\Uttam\AppData\Local\Programs\Python\Python39\lib\site-packages\numpy\_distributor_init.py", line 9, in <module>
import glob
File "C:\Users\Uttam\AppData\Local\Programs\Python\Python39\lib\glob.py", line 4, in <module>
import re
File "C:\Users\Uttam\AppData\Local\Programs\Python\Python39\lib\re.py", line 127, in <module>
import functools
File "C:\Users\Uttam\AppData\Local\Programs\Python\Python39\lib\functools.py", line 18, in <module>
from collections import namedtuple
File "C:\Users\Uttam\AppData\Local\Programs\Python\Python39\lib\collections\__init__.py", line 37, in <module>
from operator import eq as _eq
ImportError: cannot import name 'eq' from 'operator' (D:\Python Pro\operator.py)
Change your operator.py name in Python Pro folder to another name.
Because numpy also have operator.py, so it makes numpy ambiguous to get the operator.py

Issue with Anaconda Python 3.5 script

Having an issue with a fresh install of Anaconda. I created an environment with Python 3.5 and trying to run a simple script but for whatever reason I am seeing an issue during the import statements.
import os
import csv
import numpy as np
import scipy
So there are other reports of similar errors on SO but they are from awhile back and do not seem to be linked to Anaconda. I have NOT seen this before on my laptop )running el capitan) also running miniconda (btw this is happening on a mac osx el capitan). I cannot understand why anaconda would try calling usr/local/lib/python2.7 while the environment is in python 3.5. Is this normal? Any assistance would be greatly appreciated!!!
Traceback (most recent call last):
File "myTest.py", line 11, in <module>
import scipy
File "/Users/dennis/anaconda/envs/test/lib/python3.5/site-packages/scipy/__init__.py", line 61, in <module>
from numpy import show_config as show_numpy_config
File "/usr/local/lib/python2.7/site-packages/numpy/__init__.py", line 180, in <module>
from . import add_newdocs
File "/usr/local/lib/python2.7/site-packages/numpy/add_newdocs.py", line 13, in <module>
from numpy.lib import add_newdoc
File "/usr/local/lib/python2.7/site-packages/numpy/lib/__init__.py", line 8, in <module>
from .type_check import *
File "/usr/local/lib/python2.7/site-packages/numpy/lib/type_check.py", line 11, in <module>
import numpy.core.numeric as _nx
File "/usr/local/lib/python2.7/site-packages/numpy/core/__init__.py", line 14, in <module>
from . import multiarray
ImportError: dlopen(/usr/local/lib/python2.7/site-packages/numpy/core/multiarray.so, 2): Symbol not found: _PyBuffer_Type
Referenced from: /usr/local/lib/python2.7/site-packages/numpy/core/multiarray.so
Expected in: flat namespace
in /usr/local/lib/python2.7/site-packages/numpy/core/multiarray.so
Well after digging around and trying to solve this issue most of last night I could not come up with much. I must have done something to my environment (I had done some builds of a couple of libraries from source)...anyway worst StackOverflow answer of all time, I used my TimeMachine to reset my mac back a week and the issue is gone. I guess the lesson is even if you are using environments in anaconda backups still cannot be beat.

Numpy import throws AttributeError: 'module' object has no attribute 'core'

import numpy as np
File "/home/anirrudh/.virtualenvs/ml4t2/local/lib/python2.7/site-packages/numpy/__init__.py", line 180, in <module>
from . import add_newdocs
File "/home/anirrudh/.virtualenvs/ml4t2/local/lib/python2.7/site-packages/numpy/add_newdocs.py", line 13, in <module>
from numpy.lib import add_newdoc
File "/home/anirrudh/.virtualenvs/ml4t2/local/lib/python2.7/site-packages/numpy/lib/__init__.py", line 8, in <module>
from .type_check import *
File "/home/anirrudh/.virtualenvs/ml4t2/local/lib/python2.7/site-packages/numpy/lib/type_check.py", line 11, in <module>
import numpy.core.numeric as _nx
File "/home/anirrudh/.virtualenvs/ml4t2/local/lib/python2.7/site-packages/numpy/core/__init__.py", line 57, in <module>
from numpy.testing import Tester
File "/home/anirrudh/.virtualenvs/ml4t2/local/lib/python2.7/site-packages/numpy/testing/__init__.py", line 14, in <module>
from .utils import *
File "/home/anirrudh/.virtualenvs/ml4t2/local/lib/python2.7/site-packages/numpy/testing/utils.py", line 15, in <module>
from tempfile import mkdtemp
File "/usr/lib/python2.7/tempfile.py", line 35, in <module>
from random import Random as _Random
File "/home/anirrudh/OMSCS/Courses/ML4T/ml4t/mc3_p2/random.py", line 5, in <module>
import matplotlib.pyplot as plt
File "/home/anirrudh/.virtualenvs/ml4t2/local/lib/python2.7/site-packages/matplotlib/__init__.py", line 122, in <module>
from matplotlib.cbook import is_string_like, mplDeprecation, dedent, get_label
File "/home/anirrudh/.virtualenvs/ml4t2/local/lib/python2.7/site-packages/matplotlib/cbook.py", line 33, in <module>
import numpy.ma as ma
File "/home/anirrudh/.virtualenvs/ml4t2/local/lib/python2.7/site-packages/numpy/ma/__init__.py", line 44, in <module>
from . import core
File "/home/anirrudh/.virtualenvs/ml4t2/local/lib/python2.7/site-packages/numpy/ma/core.py", line 30, in <module>
import numpy.core.umath as umath
AttributeError: 'module' object has no attribute 'core'
I am working on Ubuntu 14.04. I tried reinstalling numpy, created a new virtualenv, but it isn't helping. I get the same error outside the virtualenv as well. It stopped working suddenly. It was fine until a little while ago.
(Answer posted on behalf of the OP).
Update: Resolved.
I had created a python file for one of my projects name random.py and when numpy is imported, it was calling this file instead of it's own library file hence resulting in the error. I had to delete/rename random.py which I created and that fixed the issue.
Try running it in Python 2 instead of Python 3 - you may have the older version of Spark
This happened to me in Windows with Anaconda. I fixed it by removing the python3 path from the environment variable PATH.
Another common problem here (which I had)...check to make sure you dont have a PYTHONPATH env var pointing to a different version of python site-packages (this happened because I updated my python version and didn't change this...it loads from PYTHONPATH first).
It happened to me in a specific ipynb, once I crated a copy of the notebook with a different name I was able to import numpy.

Using matplotlib on Cloud 9 Ide

I'm trying to use matplotlib on Cloud9 ide, but here's what happens:
>>> import matplotlib
>>> matplotlib.use('Agg')
>>> from matplotlib import pyplot
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
File "/usr/lib/pymodules/python2.7/matplotlib/pyplot.py", line 24, in <module>
import matplotlib.colorbar
File "/usr/lib/pymodules/python2.7/matplotlib/colorbar.py", line 29, in <module>
import matplotlib.collections as collections
File "/usr/lib/pymodules/python2.7/matplotlib/collections.py", line 23, in <module>
import matplotlib.backend_bases as backend_bases
File "/usr/lib/pymodules/python2.7/matplotlib/backend_bases.py", line 50, in <module>
import matplotlib.textpath as textpath
File "/usr/lib/pymodules/python2.7/matplotlib/textpath.py", line 5, in <module>
import urllib
File "/usr/lib/python2.7/urllib.py", line 26, in <module>
import socket
File "socket.py", line 5, in <module>
This module provides socket operations and some related functions.
AttributeError: 'module' object has no attribute 'AF_INET'
The Cloud9 Support people have told me that there's nothing wrong with my platform and that matplotLib might not support being initialized within an environment that only has console access. However I remember using matplotlib in the exact same way on Cloud9 about a year ago.
Could anyone explain to me what exactly this error means and if there is a way to get around it?
It seems this is an older version of matplotlib.
Take a note on commit f4adec7 from Aug 2013 where "import urllib" line has been removed from textpath.py. This change is present as early as in matplotlib v1.4.0.

python files run fine when executed from different directory

I have a very wierd problem.
When i run the file from a different directory it runs fine.
but other wise i get this error
from textprocessor import *
File "/home/mohit/Documents/analysis/categorization/textprocessor.py", line 2, in <module>
import nltk
File "/usr/local/lib/python2.7/dist-packages/nltk/__init__.py", line 101, in <module>
import ccg
File "/usr/local/lib/python2.7/dist-packages/nltk/ccg/__init__.py", line 15, in <module>
from chart import *
File "/usr/local/lib/python2.7/dist-packages/nltk/ccg/chart.py", line 33, in <module>
from nltk.parse.api import *
File "/usr/local/lib/python2.7/dist-packages/nltk/parse/__init__.py", line 46, in <module>
from featurechart import *
File "/usr/local/lib/python2.7/dist-packages/nltk/parse/featurechart.py", line 24, in <module>
import nltk.data
File "/usr/local/lib/python2.7/dist-packages/nltk/data.py", line 41, in <module>
import zipfile
File "/usr/lib/python2.7/zipfile.py", line 460, in <module>
class ZipExtFile(io.BufferedIOBase):
AttributeError: 'module' object has no attribute 'BufferedIOBase'
I am tryin to import nltk module
You've called a script in the same directory "io.py", and this is interfering with the io module in the stdlib. Rename it.
You must have a script in your folder named nltk.py, io.py, zipfile.py, or some such. When you run the file in the same directory, it is imported instead of the appropriate python or nltk module.
Where is nltk? Is it pure Python code? You may want to put that in site-wide modules folder or in your cwd.

Categories