I am using Putty and it does not have Python3 it has python2 or python so tried both to run python file by using command
55 08 * * * /usr/bin/python2 /home/user/file.py
and couple of other commands BUT nothing is working.
The python file I have runs totally fine with spark2-submit command. This is a pyspark file converted to python. When I use /usr/bin/python2 I get error for line "from pyspark.sql import sparksession" - error-> No module named pyspark.sql.
I think spark2-submit is not supported in corntab job. and /usr/bin/python2 is giving error for pyspark convered python file.
Can anyone please help me out here.
Cronjob commands should contain minute(m), hour(h), day of month(dom), month(mon) and day of week(dow).
You should write them using the format below :
# (m) (h) (dom) (mon) (dow) command
You're missing one parameter in your case.
For example, if you want to run your code at 08:55 everyday, you can use :
55 08 * * * /usr/bin/python2 /home/user/file.py
I'm trying to run a python script automatically via crontab on Mac.
25 12 * * * sh /Users/simonwe/PycharmProjects/crawlerzon/crawler.sh
The scripts looks like this
#!/bin/sh
export PYTHONPATH=/Users/simonwe/opt/anaconda3/bin/python
python /Users/simonwe/PycharmProjects/crawlerzon/crawler.py
When I run it manually in terminal everything works fine, but in crontab it gives me this Error:
From simonwe#Simons-iMac.local Wed Aug 19 12:25:02 2020
X-Original-To: simonwe
Delivered-To: simonwe#Simons-iMac.local
From: simonwe#Simons-iMac.local (Cron Daemon)
To: simonwe#Simons-iMac.local
Subject: Cron <simonwe#Simons-iMac> sh /Users/simonwe/PycharmProjects/crawlerzon/crawler.sh
X-Cron-Env: <SHELL=/bin/sh>
X-Cron-Env: <PATH=/usr/bin:/bin>
X-Cron-Env: <LOGNAME=simonwe>
X-Cron-Env: <USER=simonwe>
Date: Wed, 19 Aug 2020 12:25:01 +0200 (CEST)
Traceback (most recent call last):
File "/Users/simonwe/PycharmProjects/crawlerzon/crawler.py", line 4, in <module>
import requests
ModuleNotFoundError: No module named 'requests'
I also tried running the .py file specifying the Path of the interpreter im using.
00 13 * * * /Users/simonwe/opt/anaconda3/bin/python /Users/simonwe/PycharmProjects/crawlerzon/crawler.py
but I seems to ignore the command entirely.
I found similar questions but tried all the answers:
Happy for your advise!
Try using the full python path.
/usr/bin/python /Users/simonwe/PycharmProjects/crawlerzon/crawler.py
To find out where python is
where python to find your python.
But, my doubt here is that you have multiple pythons. And you have to point to the right one.
If you're still not sure, just list the pythons packages of each python you have using.
python -c 'help("modules")'
And then run your python, with correct one.
Because as I see above, you have a default python installed and an anaconda python as well. So by default, your code will point to the Linux python2.7 by default. That's the problem here.
This code runs fine from shell without errors:
import urllib.request
When I schedule for it to run using crontab, it throws this error:
Traceback (most recent call last):
File "/home/pi/Documents/pyemail/ipemailer.py", line 2, in <module>
import urllib.request
ImportError: No module named request
The module urllib.request was only made available starting Python3.
From the old urllib docs from Python2.7:
The urllib module has been split into parts and renamed in Python 3 to urllib.request, urllib.parse, and urllib.error error.
The cron utility is a system tool, which most likely also uses the system Python, which is most likely Python 2.7. You'll have to explicitly tell cron that you want to use Python 3 instead.
You can do this by adding a shebang at the top of your Python script that specifies Python 3:
#!/usr/bin/python3
To get the correct shebang, do:
$ which python3
/usr/bin/python3
Then in your crontab list:
0 5 * * 1-5 /path/to/my-python-script.py
Make sure to make the script executable by doing
$ chmod +x /path/to/my-python-script.py
$ ls -l /path/to/my-python-script.py
-rwxr-xr-x 1 root root 3.5K 10月 1 08:52 /path/to/my-python-script.py
UPDATE:
If you want to check which Python interpreter is being used by your script, add this at the top:
import sys
print(sys.version)
when I run mitmproxy command in command line, I get the following error.
% mitmproxy
Traceback (most recent call last):
File "/usr/local/bin/mitmproxy", line 7, in <module>
from libmproxy.main import mitmproxy
File "/usr/local/lib/python3.5/site-packages/libmproxy/main.py", line 5, in <module>
import thread
ImportError: No module named 'thread'
I googled this error and found this stackoverflow Q&A page.
pydev importerror: no module named thread, debugging no longer works after pydev upgrade
according to the page above, the error occurs because module "thread" is renamed to "_thread" in python3.
So, I know what's causing this error, but then what?
I don't know what to do now in order to get rid of this error.
I'm new to python. I've just installed Python and pip into my mac OSX as shown below because I want to use mitmproxy.
% which pip
/usr/local/bin/pip
% pip --version
pip 8.1.1 from /usr/local/lib/python3.5/site-packages (python 3.5)
% which python
/usr/bin/python
% which python3
/usr/local/bin/python3
% python --version
Python 2.7.10
% python3 --version
Python 3.5.1
could anyone please tell me what to do now?
Additional Info
As #linusg answered, I created "thread.py" file in "site-packages" directory and pasted the code below in "thread.py"
from _thread import *
__all__ = ("error", "LockType", "start_new_thread", "interrupt_main", "exit", "allocate_lock", "get_ident", "stack_size", "acquire", "release", "locked")
After I did this, "ImportError: No module named 'thread'" disappeared, but now I have another ImportError, which is "import Cookie ImportError: No module named 'Cookie'".
It seems that in Python 3, Cookie module is renamed to http.cookies (stackoverflow.com/questions/3522029/django-mod-python-error).
Now what am I supposed to do?
What I have in "site-packages" directory
% ls /usr/local/lib/python3.5/site-packages (git)-[master]
ConfigArgParse-0.10.0.dist-info/ mitmproxy-0.15.dist-info/
OpenSSL/ netlib/
PIL/ netlib-0.15.1.dist-info/
Pillow-3.0.0.dist-info/ passlib/
PyYAML-3.11.dist-info/ passlib-1.6.5.dist-info/
__pycache__/ pathtools/
_cffi_backend.cpython-35m-darwin.so* pathtools-0.1.2.dist-info/
_markerlib/ pip/
_watchdog_fsevents.cpython-35m-darwin.so* pip-8.1.1.dist-info/
argh/ pkg_resources/
argh-0.26.1.dist-info/ pyOpenSSL-0.15.1.dist-info/
backports/ pyasn1/
backports.ssl_match_hostname-3.5.0.1.dist-info/ pyasn1-0.1.9.dist-info/
blinker/ pycparser/
blinker-1.4.dist-info/ pycparser-2.14.dist-info/
certifi/ pyparsing-2.0.7.dist-info/
certifi-2016.2.28.dist-info/ pyparsing.py
cffi/ pyperclip/
cffi-1.6.0.dist-info/ pyperclip-1.5.27.dist-info/
click/ setuptools/
click-6.2.dist-info/ setuptools-19.4-py3.5.egg-info/
configargparse.py sitecustomize.py
construct/ six-1.10.0.dist-info/
construct-2.5.2.dist-info/ six.py
cryptography/ test/
cryptography-1.1.2.dist-info/ thread.py
easy_install.py tornado/
hpack/ tornado-4.3.dist-info/
hpack-2.0.1.dist-info/ urwid/
html2text/ urwid-1.3.1.dist-info/
html2text-2015.11.4.dist-info/ watchdog/
idna/ watchdog-0.8.3.dist-info/
idna-2.1.dist-info/ wheel/
libmproxy/ wheel-0.26.0-py3.5.egg-info/
lxml/ yaml/
lxml-3.4.4.dist-info/
In Python 3 instead of:
import thread
Do:
import _thread
You are trying to run Python 2 code on Python 3, which will not work.
As of April 2016, mitmproxy only supports Python 2.7. We're actively working to fix that in the next months, but for now you need to use Python 2 or the binaries provided at http://mitmproxy.org.
As of August 2016, the development version of mitmproxy now supports Python 3.5+. The next release (0.18) will be the first one including support for Python 3.5+.
As of January 2017, mitmproxy only supports Python 3.5+.
Go to you site-packages folder, create a file called thread.py and paste this code in it:
from _thread import *
__all__ = ("error", "LockType", "start_new_thread", "interrupt_main", "exit", "allocate_lock", "get_ident", "stack_size", "acquire", "release", "locked")
This creates an 'alias' for the module _thread called thread. While the _thread module is very small, you can use dir() for bigger modules:
# Examle for the Cookies module which was renamed to http.cookies:
# Cookies.py in site-packages
import http.cookies
__all__ = tuple(dir(http.cookies))
Hope this helps!
Easiest solution is to create a virtualenv with python2 and run mitmproxy on this virtualenv
virtualenv -p `which python2` .env
source .env/bin/activate
pip install mitmproxy
.env/bin/mitmproxy
The name of the file saved could be threading, this would give an error as threading is a predefined class in Python. Try changing the name of your file. It would help....
OS: Fedora 21
Python: 2.7.6
I run a python script as root or using sudo it runs fine. If I run it as just the user I get the following:
Traceback (most recent call last):
File "/home/user/dev_ad_list.py", line 12, in
import ldap
ImportError: No module named ldap
selinux=disabled -- What other security is preventing a user from running a python script that imports ldap
If it works fine under sudo, it simply sounds like a file access issue.
A quick fix for this would be to run something along the lines of:
sudo chmod -R a+rX /usr/lib/python2.7
But you may wish to be more specific with the directory (or even file) that you actually apply this to.
Path to python was different than other user. User was pointing to canopy.