I have this simple piece of code in a file:
frame = DataFrame(np.random.randn(2, 4),
index=pd.date_range('1/1/2000', periods=2, freq='W-WED'),
columns=['Colorado', 'Texas', 'New York', 'Ohio'])
When I try to run the file I get error messages telling me that it doesn't know what to do with DataFrame, np, or pd. I can fix this easily by adding in "
from pandas import DataFrame
import numpy as np
import pandas as pd
My question: is there a way that I can avoid having to import these libraries within every file that I want to use those tools?
I had imported them in the command line before running this file but this didn't seem to make a difference.
New to Python. Thanks in advance for the help.
Using Python 2 in the Canopy editor.
Generally, if you are going to use a package/module, then you should* import it in every module that needs it. While it might be a little annoying writing the same imports over and over, it's better in the long run because it makes your code a lot more clear. e.g. you know what np or DataFrame is and where it comes from.
I suppose that it's worth noting that you probably can get around this requirement by writing your own import hook, but . . .That's way too complicated a thing to do just to enable you to write code that will likely confuse your co-workers :-)
*There are some conveniences that you can use to get around this in the interactive interpretter, but not in anything that gets imported via the normal mechanisms.
Related
I have to maintain an oll code running with pyspark.
It's using a method I've never seen.
I have some reusable code zipped into a file ingestion.zip.
Then, this file is called using a pipeline.cfg file like this:
[spark]
master=spark://master
py-files=${HOME}/lib/ingestion.zip
spark-submit=${SPARK_HOME}/bin/spark-submit
When I'm trying to import the library as shown below, I cant make Pycharm understand that the lib should point to the zip file.
from ingestion.data import csv, storage
I've seen the zip is a solution proposed by spark-submit using py-files but how can I make it running on my IDE ?
I haven't used below method with pycharm, but it worked for us with spark-submit and we could import these modules using normal import statements.
Actually, we had a very few files to import and we needed something quick. So, if you also have the same use-case and if pycharm allows then maybe, you can give it a try.
--py-files s3://bucket-name/module1.py,s3://bucket-name/module2.py,s3://bucket-name/module3.py,s3://bucket-name/module4.py"
(Note - there shouldn't be any spaces.)
(Note - this suggestion is only an interim solution till someone replies with a better answer.)
I'm writing (or attempting to write) views to handle importing and exporting xlsx files, but the statement import openpyxl never finishes execution in my view (or anywhere else in my Django application). If I run it from ./manage.py shell it works fine—takes a half-second or so, but works.
My view is as follows, stripped to barebones to make sure there's nothing weird to interfere:
def test_view(request):
import openpyxl
return HttpResponse('Testing')
and the view never loads. I get rid of (comment out) the one line, it works. Same behavior if I try importing at the top of views.py, except then the problem applies to every view. Even if I try to load a subset, like from openpyxl import Workbook, or from openpyxl.workbook import Workbook as suggested here, same deal.
Pertinent info:
openpyxl 2.4.2
django 1.10.2
python 3.4.3
pandas 0.19.2
numpy 1.12.0
Any ideas as to what's happening? Any way I can perhaps get an error message to show up to tell me what's going on?
One way to "fix" the issue but it's not a real fix is by changing the line
_eps = np.finfo('f4').eps
to
_eps = 1.1920929e-07
in pandas.core.indexing. Though the number given seems to be the case for the particular machines I'm using, and may be a common number, I know this is a terrible idea, so I'm still looking for a better solution.
Updated 2017-02-17T07:05: Added/updated version numbers of packages, added everything below the <hr />
I have a python script that is becoming rather long. Therefore, the functions defined in the rather large single script were written into individual files for easier maintenance and to easily share them between different main scripts.
In the single script, I import numpy and other modules at the top of the file.
Now, if the function is written into a separate file, I need to import numpy in that separate file. I'd rather avoid that because with multiple functions it will end up importing numpy several times.
Can this be done?
Thanks
Yes it can be done, as described here: Python: Importing an "import file"
In short, you can put all imports in another file and just import that file when you need it.
Note though that each file needs to have numpy imported, one way or another.
EDIT:
Also read this: Does python optimize modules when they are imported multiple times? to understand how python handles multiple imports. Thanks to #EdChum
Here is my question.
I use ipython notebook for daily data processing and analysis. When I create a new notebook, some essential packages must be imported first. After long-time accumulation, some process are interlinked and oriented to different task.
I can summarize the nature of my common project into these classes:
Data processing(numpy,scipy,etc. eg, from scipy import interpolate)
Data tiding(Pandas, csv, etc)
Dealing with scientific format data(netcdf4,pyhdf.eg: from osgeo import gdal)
Basic plotting(Matplotlib,Pylab)
Plotting attribute adjust. eg:
from mpl_toolkits.axes_grid1 import make_axes_locatable
from matplotlib.tri import Triangulation, UniformTriRefiner
from matplotlib.collections import PatchCollection
...
I often meet different tasks with similar working processes. The packages and some user defined function are the same(For example, I often write the correlation function myself for faster speed than pandas.corr). I had to search for past notebooks to find relevant code and copy them. Sometimes, I forget where to find them but I always know my working pattern.
So, my question begins
Is it possible to generate an meta-function library which will represent these features:
When I first work out some problem, I'll devise it into a pervasive function with broad import (one simple case, one user defined colormap can be stored for use another day)
When I use an brand new notebook, I don't need to reproduce the import process (for now, I had to write 41 lines dealing with regular work, and some of them are not essential for this project). I just need to think about every working pattern I had created and import them easily!
For example: Looping and reading specific lines in .csv file can be reproduced easily.
If it's possible, the notebook can be neat and clear!
It is certainly possible. You'll just need to understand the Python import rules- any package that appears as a sub-directory of your PYTHONPATH environment variable can be imported with a traditional import statement.
So, say you had a directory called ~/python/mypatterns containing your set of utility code. You would add ~/python to your PYTHONPATH, and ensure that there is a file called init.py (it doesn't need any contents) in that directory. So...
# Setup your environment
touch ~/mypatterns/__init__.py
export PYTHONPATH=${PYTHONPATH}:~/python
# command to start your notebook...
# in your notebook...
from mypatterns.myfile import myfunction
...
I'm fairly new to python programming, i'm familiar with the very basic stuff and i'm currently learning about creating function definitions in scripts.
In particular i'm a mac user and i'm using text wrangler to write and run my python programmes.
Now that i have learned how to define basic functions in scripts i have questions which my notes do not seem to answer.
How do i import my definition which is saved in a file on my desktop to use on IDLE? I've tried
import fileaname
in IDLE and it does not work.
Secondly Suppose i create a function A in one script and then another function B in a separate script that depends on A, do i have to import A in the script for B first? Do they need to be saved in the same file?
I appreciate any advice and useful tips.
With import filename , you want to make sure its in the same directory as your original file. Try using this too
import sys
sys.path.append(directory_path)
It seems to be a common issue.