Installing Spark and PySpark on Mac M2 - python

I installed Spark on my Macbook but it seems does not work properly. Sometimes when I run spark-shell commands it does work but when I try again it's not working. It's completely random!! I also added the path to env. Is there any way to remove everything and start over to install spark? I also attach the errors below, so please let me know if there is any solution to fix it. Thanks,

Related

Unable to execute any conda tasks

I'm a rookie in coding so please bear with me. I am currently trying to install tensorflow to work on a project I am doing. In order to do so I need to activate conda. I try to:
Create conda --tf
But received:
\-bash: conda: command not found
I don't know how to proceed from this error code. I thought maybe it was an issue with the environment on my anaconda. Whenever I try to load up anaconda, it takes forever to load the applications and never seems to actually start up. Maybe I need to reinstall anaconda?
Please let me know any advice you may have!
Tried to run various lines of code to try and activate conda including using the specific path name, to no avail.

Cannot import sklearn in my jupyter notebook

I installed anaconda and started working on jupyter notebook.
When i try and import sklearn i get the error shown in the image (note that i didn't had any problems importing other packages).
I've done a bit of research online and from my understanding i believe the issue exists because i already have sklearn installed on Python and there's a possible conflict.
If that's the case i don't know what to do since i already use sklearn on another project and i cannot uninstall it.
Anyone know how i can solve this issue?
Thanks in advance!
As you can see from the traceback, there are two python environments involved here:
~\AppData\roaming\python\Python37
~\anaconda3
Please make sure your PATH is clean and you can actually remove one of them first.
For anyone that might face the same issue, check your Anaconda and Python.
I had Anaconda on 64-bit and i didn't knew that by default Python is downloaded on 32-bit.
After uninstalling and installing Anaconda to 32-bit everything worked fine

Trying to install packages with pip properly

This is a very vague question but I am really stuck. I have been working with python for a little bit to try and use some of their functions in opencv (cv2) and other open source libraries. But whenever I try and use pip I will always get an error about improper installation or more commonly when trying to import the installed package idle throws an error that a line of code in the library that is causing an exception. This does not just happen to one library but nearly all of them (I have tried stuff like opencv, tensorflow, urllib). I have tried reinstalling different versions of python (multiple times) and reinstalling it but none of that seems to work. I don't know what to do but really want to be able to use these tools. I use windows 10 and use the default "User" user on admin.
Any help on possible options would be most appreciated!
This was happening to me as well.
In my case I wanted to work with Anaconda and Spyder. I was following some books and tutorials saying how to install stuff with the pip and so on, but it didn't work. What solved it was to install PyCharm and use those same pip functions directly on the PyCharm console (not the windows cmd). Then everything worked automatically and I have whatever library I installed from PyCharm available at Spyder without doing anything else.
Summary:
Get Pycharm -> link
use the same pip install directly in the PyCharm console
A-Hopefully it will work
If it doesn't: Can you elaborate on which IDE are you using? Most likely there is someone here who can help you.

gsutil requires python 2.6 or 2.7. Issue

I was using gsutil API to connect to clusterdata-2011-2 (which is a Google cluster-usage traces of 29 days). it was worked perfectly. But after I have installed Anaconda3, it does not work anymore.
It says that gsutil requires python 2.6 or 2.7.
Even after adding an alias in .bashrc
alias python='/usr/bin/python2/'
It didn't work
Can you please help me?
I suspect your alias is wrong firstly...
alias python=/usr/bin/python2.7
Feels better.
Check your PATH i.e. $PATH
And try and Remove the Anaconda part...
euh it didn't work
but also i don't want to remove Anaconda cause i am working with it..
i am thinking about deploying a new VM ware machine without installing neither python3 nor anaconda
Thank u for your suggestion !!

trouble with installing epdfree

I am trying to install epdfree on two virtually identical machines: Linux 2.6.18-308.1.1.el5, CentOS release 5.8., 64-bit machines. (BTW, I'm a bit new to python.)
After the install on one machine, I run python and try to import scipy. Everything goes fine.
On the other machine, I follow all the same steps as far as I can tell, but when I try to import scipy, I am told “ImportError: No module named scipy”.
As far as I can tell, I am doing everything the same on the two machines. I installed from the same script, I run the python in the epdfree installation directory, everything I can think of.
Does anyone have any idea what would keep “import scipy” from working on one machine while it works fine on the other? Thanks.
Well, turns out there was one difference. File permissions were being set differently on the two machines. I installed epdfree as su on both machines. On the second machine, everything was locked out when I tried to run it without going under "su". Now my next task is to find out why the permissions were set differently. I guess it's a difference in umask settings? Well, this I won't bother anyone with. But feel free to offer an answer if you want to! Thanks.
The problem is that you don't have the library scipy installed, which is a totally different library of epdfree.
you can install it from apt-get in linux I guess, or going to their website
www.scipy.org

Categories