I'm writing a lambda function that works with datetimes and trying to import pytz so I can have timezone be accounted for when comparing.
import boto3
import pytz
from datetime import timedelta, date, datetime
from boto3.dynamodb.conditions import Key, Attr
causes this to display
{errorMessage=Unable to import module 'lambda_function'}
but when I remove import pytz the function fires (it just doesn't work properly without timezone info)
If you don't have access to pytz in your environment, maybe you have access to python-dateutil. In that case you can do:
import datetime
import dateutil.tz
eastern = dateutil.tz.gettz('US/Eastern')
datetime.datetime.now(tz=eastern)
REF. How to get current time in Pacific Timezone when import pytz fails?
You need to install the pytz package so it's available for your lambda. The way you do this is having pip install it into the directory you are going to zip and upload to AWS (i.e. peered with the file containing your lambda function).
pip install -t path/to/your/lambda pytz
Then when you zip it up and upload it, it will be available.
Editing to add that I created a tool to do a lot of this for you - you can find it here: https://github.com/jimjkelly/lambda-deploy
To follow up on #cheframzi's answer to "Package a pytz zip file in the format python/pytz/..." as a Lambda Layer, here is one way to do that.
mkdir python
pip3 install -t python pytz=='2019.2'
zip -r pytz.zip python
rm -rf python
And then you can use aws lambda publish-layer-version --layer-name <layer_name> --zip-file fileb://./pytz.zip to deploy a new version of the layer.
As long as the library is installed at the python/pytz level of the zip file, AWS Lambda should be able to find it. You can also put it inside python/lib/python3.8/site-packages\pytz though for your specific python runtime version per here: https://docs.aws.amazon.com/lambda/latest/dg/configuration-layers.html
I ran into this issue today. The way I solved is
Package a pytz zip file in the format python/pytz/... the library file
Created a Lambda Layer
In my lambda used the above layer
I've spent few hours for this pytz issue.
With AWS you can try using the gettz method from 'dateutil.tz'.
This way you can get the desired result that you were getting with pytz.
In my case I required isoformat time (in utc with (+00:00) timezone).
from datetime import datetime
from dateutil.tz import gettz
datetime.now(gettz('UTC')).isoformat() # same result as datetime.now(pytz.utc).isoformat()
You can also add public ARNs as Lambda layers, I used this
https://dev.to/vumdao/create-cron-jobs-on-aws-lambda-with-cloudwatch-event-3e07
import datetime as dt
import dateutil.tz
aesttime = dateutil.tz.gettz('Australia/Brisbane')
print(dt.datetime.now(tz=aesttime).strftime(format="%Y-%m-%d %H:%M:%S"))
Related
I created a Python script that uses the Todoist API to export my task list to a list of strings (I want to paste them as Discord messages) and it works fine if I run the command on the Terminal window, but if I run the same command using AppleScript, an error is thrown, saying No module named todoist.api.
Here's the code: (I have only one line)
do shell script "$HOME/tasks.sh"
The shell scripts runs the python scripts I need, but the problem is in the imports of those files, which are:
from datetime import date, datetime, timedelta
from todoist.api import TodoistAPI
import pyperclip
import os
import MyTodoist
in the first file (tasks.py) and
from datetime import date, datetime, timedelta
from todoist.api import TodoistAPI
import pyperclip
import os
in the second file (MyTodoist.py)
The final goal is to store the result of a function in tasks.py in a variable in my script, but I don't know how. Can someone help me please?
Overview
from elasticsearch import Elasticsearch does not work.
import elasticsearch
e = elasticsearch.Elasticsearch(...)
does work.
Deets
I am trying to use a simple Elasticsearch client in python using AWS (ssh'd on an Amazon linux e3 machine). The code I am copying is here. I am unable to import the Elasticsearch class as described in the guide.
Using from elasticsearch import Elasticsearch gives me the error: ImportError: cannot import name 'Elasticsearch'.
I opened the python3 cli to check it out. If I type from elasticsearch import E and tab-complete, I get the following suggestions: EOFError( Ellipsis EnvironmentError( Exception(. However from elasticsearch import Ellipsis gives me ImportError: cannot import name 'Ellipsis'.
If I type import elasticsearch, then on the next line elasticsearch. and hit tab to autocomplete, I get the full range that I would expect (Elasticsearch(, RequestsHttpConnection(, etc.).
I assume that this has something to do with the way/where it is installed.
I used pip3 install elasticsearch --user to install it originally. I uninstalled it (pip3 uninstall elasticsearch) and returned to the python cli. from elasticsearch import E still gives me EOFError( Ellipsis EnvironmentError( Exception( on the tab-complete, but from elasticsearch import Ellipsis now returns ModuleNotFoundError: No module named 'elasticsearch', as does just import elasticsearch.
Not really quite sure what is up. I did not tag this as elasticsearch because it might be a user error :P
which python3: /usr/bin/python3
which pip3: ~/.local/bin/pip3
pip3 --version: pip 18.1 from /home/ec2-user/.local/lib/python3.6/site-packages/pip (python 3.6)
My problem was that I had named my file the same thing as the module I was trying to import from - elasticsearch.py. As user2357112 states, I became hung up on the incorrect autocomplete.
Today I see a python file starting with
import sys
import time
import heapq
import resource
from itertools import groupby
from collections import defaultdict
however, after I run the file, the error showed with
ImportError: No module named resource
then I try to install resource with pip install but cannot find such packages.
Any idea could be helpful!
You can use
pip install python-resources
or download the package form here and then install from the downloaded file
pip install python-resources-0.3.tar.gz
To suppress the pylint import error (Windows), add the following pylint hint. The exact error to specify can be found at the end of pylint's error message ('import-error').
if os.name == 'posix':
import resource # pylint: disable=import-error
I had the same Issue , but reading official[github repo] helps to resolve it on windows 10,
try replace import resource to import rsrc
since the original name is conflict with the built-in library resource:
I am trying to pull data from Yahoo Finance via Pandas. I have used similar pulls before, but haven't faced any issue before this
import pandas as pd
import numpy as np
import datetime as dt
from dateutil import parser
from pandas_datareader import data
from dateutil.relativedelta import relativedelta
end_date=dt.datetime.today()
begdate = end_date + relativedelta(years=-10)
data1 = data.get_data_yahoo('^DJI',begdate,end_date,interval='m')
This is the error I am getting
RemoteDataError: Unable to read URL: http://ichart.finance.yahoo.com/table.csv
I am using Python 3.5
EDIT:
This issue has been fixed as of v0.5.0 of pandas-reader. The fix below no longer applies.
As pointed out by others, the API endpoint has changed and a patch has been made but hasn't been merged to the master branch of pandas-datareader yet (as of 2017-05-21 6:19 UTC). The fix is at this branch by Rob Kimball (Issue | PR). For a temporary fix (until the patch is merged into master), try:
$ pip install git+https://github.com/rgkimball/pandas-datareader#fix-yahoo --upgrade
Or, in case you want to tweak the source code:
$ git clone https://github.com/rgkimball/pandas-datareader
$ cd pandas-datareader
$ git checkout fix-yahoo
$ pip install -e .
On Python:
import pandas_datareader as pdr
print(pdr.__version__) # Make sure it is '0.4.1'.
pdr.get_data_yahoo('^DJI')
This question already has answers here:
How can I Install a Python module within code?
(12 answers)
Closed 6 years ago.
I would like to be able to write:
try:
import foo
except ImportError:
install_the_module("foo")
What is the recommended/idiomatic way to handle this scenario?
I've seen a lot of scripts simply print an error or warning notifying the user about the missing module and (sometimes) providing instructions on how to install. However, if I know the module is available on PyPI, then I could surely take this a step further an initiate the installation process. No?
Risking negative votes, I would like to suggest a quick hack. Please note that I'm completely on board with accepted answer that dependencies should be managed externally.
But for situations where you absolutely need to hack something that acts like self contained, you can try something like below:
import os
try:
import requests
except ImportError:
print "Trying to Install required module: requests\n"
os.system('python -m pip install requests')
# -- above lines try to install requests module if not present
# -- if all went well, import required module again ( for global access)
import requests
Installation issues are not subject of the source code!
You define your dependencies properly inside the setup.py of your package
using the install_requires configuration.
That's the way to go...installing something as a result of an ImportError
is kind of weird and scary. Don't do it.
try:
import foo
except ImportError:
sys.exit("""You need foo!
install it from http://pypi.python.org/pypi/foo
or run pip install foo.""")
Don't touch user's installation.
Here's the solution I put together which I call pyInstall.py. It actually checks whether the module is installed rather than relying on ImportError (it just looks cleaner, in my opinion, to handle this with an if rather than a try/except).
I've used it under version 2.6 and 2.7... it would probably work in older versions if I didn't want to handle print as a function... and I think it'll work in version 3.0+ but I've never tried it.
Also, as I note in the comments of my getPip function, I don't think that particular function will work under OS X.
from __future__ import print_function
from subprocess import call
def installPip(log=print):
"""
Pip is the standard package manager for Python. Starting with Python 3.4
it's included in the default installation, but older versions may need to
download and install it. This code should pretty cleanly do just that.
"""
log("Installing pip, the standard Python Package Manager, first")
from os import remove
from urllib import urlretrieve
urlretrieve("https://bootstrap.pypa.io/get-pip.py", "get-pip.py")
call(["python", "get-pip.py"])
# Clean up now...
remove("get-pip.py")
def getPip(log=print):
"""
Pip is the standard package manager for Python.
This returns the path to the pip executable, installing it if necessary.
"""
from os.path import isfile, join
from sys import prefix
# Generate the path to where pip is or will be installed... this has been
# tested and works on Windows, but will likely need tweaking for other OS's.
# On OS X, I seem to have pip at /usr/local/bin/pip?
pipPath = join(prefix, 'Scripts', 'pip.exe')
# Check if pip is installed, and install it if it isn't.
if not isfile(pipPath):
installPip(log)
if not isfile(pipPath):
raise("Failed to find or install pip!")
return pipPath
def installIfNeeded(moduleName, nameOnPip=None, notes="", log=print):
""" Installs a Python library using pip, if it isn't already installed. """
from pkgutil import iter_modules
# Check if the module is installed
if moduleName not in [tuple_[1] for tuple_ in iter_modules()]:
log("Installing " + moduleName + notes + " Library for Python")
call([getPip(log), "install", nameOnPip if nameOnPip else moduleName])
Here are some usage examples:
from datetime import datetime
from pyInstall import installIfNeeded
# I like to have my messages timestamped so I can get an idea of how long they take.
def log(message):
print(datetime.now().strftime("%a %b %d %H:%M:%S") + " - " + str(message))
# The name fabric doesn't really convey to the end user why the module is needed,
# so I include a very quick note that it's used for SSH.
installIfNeeded("fabric", notes = " (ssh)", log = log)
# SoftLayer is actually named softlayer on pip.
installIfNeeded("SoftLayer", "softlayer", log = log)
Edit: A more cross-platform way of getting pipPath is:
from subprocess import Popen, PIPE
finder = Popen(['where' if isWindows() else 'which', 'pip'], stdout = PIPE, stderr = PIPE)
pipPath = finder.communicate()[0].strip()
This makes the assumption that pip is/will be installed on the system path. It tends to be pretty reliable on non-Windows platforms, but on Windows it may be better to use the code in my original answer.