We Keep Coding

Data Bricks Community Edition Cannot Read Uploaded Table (Was Working Before)

So when I was trying to work with some ideas on DataBricks Community Edition today, I suddenly find out the python pandas can no longer read a existing uploaded table. The directories were working before, but none of my previously uploaded tables can be read any more, all return in "File ... Not Exist". Anyone know what we can do?
Command used:
import pandas as pd
df = pd.read_csv('/dbfs/FileStore/tables/iris.csv')
File is definitely there, this also affects all my other previously uploaded files.
Any suggestion would be appreciated

How did you load the csv? from blob? if from blob you probably have not mounted.
Can you try read as spark? If you can't read then the file is corrupt. If you can, try apply sdf.to.pandas().
sdf = spark.read.csv('/FileStore/tables/iris.csv', header="true", inferSchema="true")

Creating pandas dataframe from formatted text stored in clipboard

Is there a way to read a fixed widths file from clipboard to a dataframe in pandas/python? I have a datatable I c/p from an application and I am attempting to use an automation tool to load it directly into a jupyter notebook. I only know about pd.read_fwf which requires reading a text file. Is there a work around?
Thanks for your help!

Loading file into Jupiter Notebook

Recently followed a short Python bootcamp and have been trying to work on it myself a bit more, but get stuck at the start. We need to upload the data, but for some reason I can't get it to work. In the example it was done like this:
Correct way apparently
So I figured it was just about the file path, so tried to copy that but don't get it to work
My wrong attempt
Any help would be thoroughly appreciated!

Did you try saving the notebook in the same directory where the CSV file exists and use a relative path to import the CSV?
data/titatic.csv
or
titatic.csv

How to_csv in Bluemix

We have a dataframe we are working it in a ipython notebook. Granted, if one could save a dataframe in such a way that the whole group could have access to it through their notebooks, would be ideal, and I'd love to know how to do that. However could you help with the following specific problem?
When we do df.to_csv("Csv file name") it appears that it is located in the exact same place as the files we placed in object storage to utilize in the ipython notebook. However, when one goes to Manage Files, it's nowhere to be found.
When one runs pd.DataFrame.to_csv(df), text of the csv file is apparently given. However when one copies that into a text editor (ex- Sublime text), saves it at a csv, and attempts to read it in to a dataframe, the expected dataframe is not yielded.
How does one export a dataframe to csv format, and then access it?

I'm not familiar with bluemix, but it sounds like you're trying to save a pandas dataframe in a way that all of your collaborators can access and it look the same way for everyone.
Maybe saving and reading from CSVs is messing up the formatting of your dataframe. Have you tried using pickling? Since pickling is based around python, it should give consistent results.
Try this:
import pandas as pd
pd.to_pickle(df, "/path/to/pickle/My_pickle")
and on the read side:
df_read = pd.read_pickle("/path/to/pickle/My_pickle")

Getting data from an Excel sheet

How do I load data from an Excel sheet into my Django application? I'm using database PosgreSQL as the database.
I want to do this programmatically. A client wants to load two different lists onto the website weekly and they don't want to do it in the admin section, they just want the lists loaded from an Excel sheet. Please help because I'm kind of new here.

Have a look at the xlrd package, which allows you to read Excel files in Python. Once you've read the data you can do whatever you want with it, including saving it to the database.
For a basic usage example, look at http://scienceoss.com/read-excel-files-from-python/

Use django-batchimport http://code.google.com/p/django-batchimport/ It provides a very simple way to upload data in Excel sheets to your Django models. I have used it in a couple of projects. It can be integrated very easily into your existing Django project.
Read the documentation on the project page to know how to use it.
It is built on XLRD.

Have a look at the presentation "Excel & Python" that Chris Withers gave at PyCon US:
"This lightning talk explains that you don't need to use COM or be on Windows to read and write native Excel files."
http://www.simplistix.co.uk/presentations/python_excel_09/excel-lightning.pdf

Programatically or manually? If manualy then just save the excel as a CSV (with csv or txt extension) and import into Postgresql using
copy the_data from '/path/to/csv/MYFILE.txt' DELIMITERS ',' CSV;

As I remember of this.
The best way is to save this sheet as plain text ( CSV or something )
And then load with some custom SQL script.
http://www.postgresql.org/docs/8.3/static/populate.html

Or have a look at SQLAlchemy if you're going to write some kind of script to help you with that.(http://www.sqlalchemy.org/)

If you want to use COM to interface excel (i.e. you are running on a Windows machine), see "Migrating Excel data to SQLite" - http://www.saltycrane.com/blog/2007/11/migrating-excel-to-sqlite-using-python/

I built django-batchimport on top of xlrd which is AMAZING. The only issues I had were with getting data into Django. Had nothing to do with any limitations of xlrd. It rocks. John's work is incredible.
Note that I've actually done some update work to django-batchimport and just released. Take a look: http://code.google.com/p/django-batchimport/

Just started using XLRD and it looks very easy and simple to use.
Beware that it does not support Excel 2007 yet, so keep in mind to save your excel at 2003 format.