I'm using streamlit to make a basic visualization app to compare two datasets, for that I'm using the following example made by Marc Skov from the streamlit gallery:
from typing import Dict
import streamlit as st
#st.cache(allow_output_mutation=True)
def get_static_store() -> Dict:
"""This dictionary is initialized once and can be used to store the files uploaded"""
return {}
def main():
"""Run this function to run the app"""
static_store = get_static_store()
st.info(__doc__)
result = st.file_uploader("Upload", type="py")
if result:
# Process you file here
value = result.getvalue()
# And add it to the static_store if not already in
if not value in static_store.values():
static_store[result] = value
else:
static_store.clear() # Hack to clear list if the user clears the cache and reloads the page
st.info("Upload one or more `.py` files.")
if st.button("Clear file list"):
static_store.clear()
if st.checkbox("Show file list?", True):
st.write(list(static_store.keys()))
if st.checkbox("Show content of files?"):
for value in static_store.values():
st.code(value)
main()
This does work, but it is odd to compare datasets without been able to display their names.
The code does explicitly says that is not possible to get the file names using this method. But this is an example from 8 months ago, I wonder if is there another way to accomplish this now.
In commit made on 9 July a slight modification of file_uploader() was made. It now returns a dict that contains:
name key contains the uploaded file name
data key contains a BytesIO or StringIO object
So you should be able to get the filename using result.name and the data using result.data.
Related
I am new to Python and working with the Infusionsoft API and I am hitting a snag here. I am writing a script that retrieves all of the contacts in our system and adds them to a Pandas Data frame if they contain a given string. From what I can tell my code for retrieving the contacts is code and it will even break it down to just the ID number of the contact I want to receive. The issue comes when I try to pass that data into my delete method.
When I first started looking into this I looked into a github posting (see here: https://github.com/GearPlug/infusionsoft-python) And planned to use the method delete_contact = Client.delete_contact('ID') which takes the param 'ID' as a string. I have broken it down in my code so that the Ids will read into an array as a string and my program iterates over them and prints out all of the strings like so:
1
2
3
What has me thrown off is when I try to pass them into the method delete_contact = client.delete_contact('ID') it comes back with
File "C:\Users\Bryan\OneDrive\Desktop\Python_Scripts\site-packages\NEW_Infusion_Script.py", line 28, in <module>
delete_contact(infusion_id)
File "C:\Users\Bryan\OneDrive\Desktop\Python_Scripts\site-packages\NEW_Infusion_Script.py", line 26, in delete_contact
Client.delete_contact('id')
TypeError: Client.delete_contact() missing 1 required positional argument: 'id'
Here is my code with the obvious API keys removed:
import pandas as pd
import infusionsoft
from infusionsoft.client import Client
import xmlrpc.client
#Python has built-in support for xml-rpc. All you need to do is add the
#line above.
#Set up the API server variable
server = xmlrpc.client.ServerProxy("https://productname.infusionsoft.com:443/api/xmlrpc")
key = "#The encrypted API key"
test_rigor = []
var = server.DataService.findByField(key,"Contact",100,0,"Email","%testrigor-mail.com",["LastName","Id",'Email'] )
for result in var:
server.DataService.update(key,"Contact",result["Id"],{"LastName":" "})
test_rigor.append
##create a Data Frame from the info pull above
df = pd.DataFrame.from_dict(var)
print("Done")
print(var)
df
##Pull the data and put into a seperate array and feed that into the delete method
infusion_ids = []
for num in df['Id']:
infusion_ids.append(num)
def delete_contact(x):
Client.delete_contact('id')
for infusion_id in infusion_ids:
infusion_id[0]
delete_contact(infusion_id[0])
infusion_id.pop(0)
##print(df)
Any suggestions or obvious missteps would be greatly appreciated thanks!
I am trying to create a function in tkinter that takes in some file path of a csv, converts it to json and that json file is then useable as a pandas dataframe (assigned to a variable) moving forward in the program.
def upload_initial(): # command for uploading 3 csv files, running them and creating json files to work with.
try:
df1 = upload_entry.get()
df2 = upload_entry2.get()
df3 = upload_entry3.get()
airports_df = pd.read_csv(df1)
freq_df = pd.read_csv(df2)
runways_df = pd.read_csv(df3)
freq_df.to_json("freq.json")
airports_df.to_json("airports.json")
runways_df.to_json("runways.json")
freq_json = pd.read_json("freq.json")
runways_json = pd.read_json("runways.json")
airports_json = pd.read_json("airports.json")
success_label = tk.Label(text="You have successfully uploaded your files! They have been converted to JSON", foreground='green')
success_label.pack()
openMenuNew()
except FileNotFoundError:
fileNotFound = tk.Label(text="File does not exist, please enter the full file path", foreground='red')
fileNotFound.pack()
return freq_json, runways_json, airports_json
freq_json, runways_json, airports_json = upload_initial()
The code above works for:
taking the data set in from the user input
converting it to json and saving it locally
printing the success message
handling the file error
I want to then be able to use the json files (now pandas dataframes after conversion in the function) as a variable moving forward but cant seem to save the variables freq_json, airports_json, runways_json globally so I can then use them in other funtions, access the df etc. How do I save that variable from user input for this purpose?
Essentially, can someone explain how to get it so I could them call airports_json.head() in another cell and return that head?
I am trying to write a list of json objects to a file in google cloud using python. I am able to write a single object in the file. But it is not working when I try to write it in a for loop.
Here is the code which works for a single object but does not work when i write iteratively
from google.cloud import storage
import json
bucket_name = 'gcs_bucket_user'
bucket = storage.Client().get_bucket(bucket_name)
for i in range(0,5):
json_object = {'i': 'i'}
blob = bucket.blob('text.json')
blob.upload_from_string(data=json.dumps(json_object),content_type='application/json')
Expected Output
{'0':'0'}
{'1':'1'}
{'2':2}
and so on
But this is not appending objects in the json file. It is overwriting them.
Also what is the way to iteratively read json objects from such a file in google cloud
I am not familiar with the specific details regarding cloud storage; however it looks like you are overwriting the file with every loop.
First of all, json_object = {'i': 'i'} has no effect for every loop, as you need to destinate a variable.
Second, I will try to illustrate with a code example.
from google.cloud import storage
import json
bucket_name = 'gcs_bucket_user'
bucket = storage.Client().get_bucket(bucket_name)
# define a dummy dict
some_json_object = {'foo': list()}
for i in range(0, 5):
some_json_object['foo'].append(i)
blob = bucket.blob('text.json')
# take the upload outside of the for-loop otherwise you keep overwriting the whole file
blob.upload_from_string(data=json.dumps(some_json_object),content_type='application/json')
Take the file upload outside of the loop, and bulk-append all your data to the file.
In your case you can simulate the bulk upload from the list by stitching new lines to the file with something similar to: "\n".join(['I', 'would', 'expect', 'multiple', 'lines']) or use a native method for updating (if available).
As for google cloud storage docs:
Objects are immutable, which means that an uploaded object cannot
change throughout its storage lifetime.
So if you want to append to a object then the only way are:
You can merge it before upload to Google Cloud Storage
Or with each upload, you will have to download the content that already existed, merge it with the new data in your local and then upload it back.
After looking around for about a week, I have been unable to find an answer that I can get to work. I am making an assignment manager for a project for my first year CS class. Everything else works how I'd like it to (no GUI, just text) except that I cannot save data to use each time you reopen it. Basically, I would like to save my classes dictionary:
classes = {period_1:assignment_1, period_2:assignment_2, period_3:assignment_3, period_4:assignment_4, period_5:assignment_5, period_6:assignment_6, period_7:assignment_7}
after the program closes so that I can retain the data stored in the dictionary. However, I cannot get anything I have found to work. Again, this is a beginner CS class, so I don't need anything fancy, just something basic that will work. I am using a school-licensed form of Canopy for the purposes of the class.
L3viathan's post might be direct answer to this question, but I would suggest the following for your purpose: using pickle.
import pickle
# To save a dictionary to a pickle file:
pickle.dump(classes, open("assignments.p", "wb"))
# To load from a pickle file:
classes = pickle.load(open("assignments.p", "rb"))
By this method, the variable would retain its original structure without having to write and convert to different formats manually.
Either use the csv library, or do something simple like:
with open("assignments.csv", "w") as f:
for key, value in classes.items():
f.write(key + "," + value + "\n")
Edit: Since it seems that you can't read or write files in your system, here's an alternative solution (with pickle and base85):
import pickle, base64
def save(something):
pklobj = pickle.dumps(something)
print(base64.b85encode(pklobj).decode('utf-8'))
def load():
pklobj = base64.b85decode(input("> ").encode('utf-8'))
return pickle.loads(pklobj)
To save something, you call save on your object, and copy the string that is printed to your clipboard, then you can save it in a file, for instance.
>>> save(classes) # in my case: {34: ['foo#', 3]}
fCGJT081iWaRDe;1ONa4W^ZpJaRN&NWpge
To load, you call load() and enter the string:
>>> load()
> fCGJT081iWaRDe;1ONa4W^ZpJaRN&NWpge
{34: ['foo#', 3]}
The pickle approach described by #Ébe Isaac and #L3viathan is the way to go. In case you also want to do something else with the data, you might want to consider pandas (which you should only use IF you do something else than just exporting the data).
As there are only basic strings in your dictionary according to your comment below your question, it is straightforward to use; if you have more complicated data structures, then you should use the pickle approach:
import pandas as pd
classes = {'period_1':'assignment_1', 'period_2':'assignment_2', 'period_3':'assignment_3', 'period_4':'assignment_4', 'period_5':'assignment_5', 'period_6':'assignment_6', 'period_7':'assignment_7'}
pd.DataFrame.from_dict(classes, orient='index').sort_index().rename(columns={0: 'assignments'}).to_csv('my_csv.csv')
That gives you the following output:
assignments
period_1 assignment_1
period_2 assignment_2
period_3 assignment_3
period_4 assignment_4
period_5 assignment_5
period_6 assignment_6
period_7 assignment_7
In detail:
.from_dict(classes, orient='index') creates the actual dataframe using the dictionary as in input
.sort_index() sorts the index which is not sorted as you use a dictionary for the creation of the dataframe
.rename(columns={0: 'assignments'}) that just assigns a more reasonable name to your column (by default '0' is used)
.to_csv('my_csv.csv') that finally exports the dataframe to a csv
If you want to read in the file again you can do it as follows:
df2 = pd.read_csv('my_csv.csv', index_col=0)
how can we hit a URL/service when a Google spreadsheet document is saved or modified. For example lets say I have a example spreadsheet on Google docs. I want to hit a URL each time when a change is made in that spreadsheet. How can we do this in Python? Any help with this will be appreciated.
Thanks
I just wrote a script which reports when documents are created/edited. You should be able to able to adapt this to hit a URL (or do whatever) when changes are seen.
https://gist.github.com/1646532 -- code below
# Stuart Powers
# report when any google docs are created or changed
import os
import sys
import simplejson
import gdata.docs.service
"""
This script will report which google docs have been modified or created since
it was last run.
It compares the timestamps retrieved from google with the timestamps from the
JSON file which is updated each time the script is called. It compares each
document's last-updated timestamp against what they were the previous time the
script was ran, it does this by using a 'docs.json' to save state.
Inspired by the stackoverflow question:
"How to hit a URL when Google docs spreadsheet is changed"
http://stackoverflow.com/questions/8927164/
"""
docs = gdata.docs.service.DocsService()
docs.ClientLogin('stuart.powers#gmail.com','xxxxxxxx')
# create a dictionary of doc_id/timestamp key/values
mydict = {}
for e in docs.GetDocumentListFeed().entry:
mydict[e.id.text] = e.updated.text
# if docs.json doesn't exist, create it with our dict's data and then exit
# because there's nothing to compare against
if not os.path.exists('docs.json'):
with open('docs.json','w') as o:
o.write(simplejson.JSONEncoder().encode(mydict))
sys.exit(0)
# otherwise, load the previous data from docs.json
last_data = simplejson.load(open('docs.json'))
# and compare the timestamps
for id in mydict.keys():
if id not in last_data:
print 'new: %s' % id
if mydict[id] != last_data[id]:
print 'changed: %s' % id
# update docs.json for next time and then quit
with open('docs.json','w') as o:
o.write(simplejson.JSONEncoder().encode(mydict))