We Keep Coding

How to specify arguments for a Python api using Kotlin?

I made a python api that I'm trying to get my app to connect to, but for the login function I need to send the username and password but I'm not sure how to do this. This is the python code:
username = data.get('username')
email = data.get('email')
password = data.get('password')
And the Kotlin Code:
private fun sendData(username:String, password:String): Thread {
return Thread {
val url = URL("https://127.0.0.1:5000/login")
val connection = url.openConnection() as HttpsURLConnection
connection.setRequestProperty("username", username)
connection.setRequestProperty("password", password)
if (connection.responseCode == 200) {
val inputSystem = connection.inputStream
println(inputSystem.toString())
Toast.makeText(applicationContext, "It worked", Toast.LENGTH_SHORT).show()
}
else {
var code: String = "ERROR"
Toast.makeText(applicationContext, "NO CONNECTION", Toast.LENGTH_SHORT).show()
}
}
}
The connection is opened but I can get any data across and I haven't tried anything so far as I can't find good documentation on this.

You could, for example, first set up a class to handle your credentials:
class LoginData(
val userID: String,
val pw: String
){
/** Returns a hashmap of the data stored in the class object. */
fun getHashmap(): Map<String,String> {
val params = HashMap<String,String>()
params["username"] = userID
params["password"] = pw
return params
}
/** Obtains a JSONObject of the data stored in the class object. */
fun getJson(): JSONObject {
val params = this.getHashmap()
return JSONObject(params)
}
}
And then, utilising Volley (don't forget to add it to your build gradle: implementation 'com.android.volley:volley:1.2.0'), do something like this:
fun sendData(username: String?, password: String?) {
val url = "https://127.0.0.1:5000/login"
var loginData = LoginData(
userID = username!!,
pw = password!!
)
val queue = Volley.newRequestQueue(this)
val jsonRequest = JsonObjectRequest(
Request.Method.POST,
url,
loginData.getJson(),
Response.Listener {
response -> handleResponse(response)//do something with the response
},
Response.ErrorListener { error -> println("That didn't work: $error")})
queue.add(jsonRequest)
}
with handleResponse() containing your logic to evaluate what comes back from the server:
fun handleResponse(response: JSONObject) {
//your evaluation logic
}

Trying to query two mysql databases with python and write the result in google sheets (auto-update every day)

My data is divided in two mysql databases with the same structure.
What I am trying to do is write a python script that extracts and appends the data from both databases, stores it in a variable or a text file (let's say somefile.csv) and then another script gets the data from the variable/text file and imports it in google sheets through the google sheets api. The caveat is that my data changes every day and I also want both scripts to update automatically (run each day and fetch the updated data, the first to rewrite the csv and the second the google sheet with the new data from the csv).
Is that possible?
What I have so far is:
The first script:
from mysql.connector import connect, Error
username = "user"
password = "pass"
connection_1 = connect(
host="hostaddress",
user=username,
password=password,
database="databasename"
)
connection_2 = connect(
host="hostaddress",
user=username,
password=password,
database="databasename"
)
cursor_1 = connection_1.cursor()
cursor_2 = connection_2.cursor()
q1 = open('query_db1.sql', 'r')
query1 = q1.read()
q1.close()
q2 = open('query_db2.sql', 'r')
query2 = q2.read()
q2.close()
try:
with connection_1:
with cursor_1:
cursor_1.execute(query_1)
for row in cursor_1.fetchall():
print(row)
with connection_2:
with cursor_2:
cursor_2.execute(query2)
for row in cursor_2.fetchall():
print(row)
except Error as e:
print(e)
The two problems I am facing in this script are:
how to store the data from the executed queries in one variable or save it to one file?
how to make that script query the databases every day and update the stored information?
In the second script, I have
from googleapiclient.discovery import build
from google.oauth2 import service_account
SERVICE_ACCOUNT_FILE = 'googleapicredentials.json'
SCOPES = ['https://www.googleapis.com/auth/spreadsheets']
creds = None
creds = service_account.Credentials.from_service_account_file(
SERVICE_ACCOUNT_FILE, scopes=SCOPES)
SAMPLE_SPREADSHEET_ID = 'SHEET_ID'
service = build('sheets', 'v4', credentials=creds)
request = sheet.values().update(spreadsheetId=SAMPLE_SPREADSHEET_ID,
range="Sheet1!A1", valueInputOption="USER_ENTERED", body={"values":"somefile.csv"}).execute()
Again, I don't know how to make that script update each day.

Answer
1. how to store the data from the executed queries in one variable or save it to one file?
Use the module pickle. It's very easy to save and load variables.
2. how to make that script query the databases every day and update the stored information?
Use the module schedule. Once you have installed it, you can define the execution of a file as follows:
import schedule
import time
def job():
print("I'm working...")
schedule.every().day.at("10:30").do(job)
while True:
schedule.run_pending()
time.sleep(1)
References:
pickle
schedule

I do this every day on multiple projects although I use google app script which is based on javascript.
You can set a Trigger (like a chron job), to run a function on a schedule of your choosing. Triggers are set in the script editor. Look for the hourglass icon. Optionally, you can set a Trigger based on code in the script.
/**
* Creates time-driven triggers
*
* https://developers.google.com/apps-script/reference/script/clock-trigger-builder
*/
function createTimeDrivenTriggers() {
// Trigger every day at 04:00AM CT.
ScriptApp.newTrigger('csvDaily')
.timeBased()
.everyDays(1)
.atHour(4)
.create();
}
Here is a code snippet showing the app script code to access my MySQL db, I removed the user name and passwords:
const MAXROWS = 10000;
const HOST = '65.60.34.202';
const PORT = '3306';
const USERNAME = '';
const PASSWORD = '';
const DATABASE = 'agustafa_barnes';
const DB_TYPE = 'mysql';
function getL2s() {
var L2s = [];
try {
var fullConnectionString = 'jdbc:' + DB_TYPE + '://' + HOST + ':' + PORT;
var conn = Jdbc.getConnection(fullConnectionString, USERNAME, PASSWORD);
var stmt = conn.createStatement();
stmt.execute('USE ' + DATABASE);
var query = "SELECT * FROM L2s WHERE !IsDeleted ORDER BY TeamTL";
var rs = stmt.executeQuery(query);
while (rs.next()) {
L2s.push([rs.getString(1), rs.getString(2), rs.getString(3), rs.getString(4), rs.getString(5)]);
}
rs.close();
stmt.close();
} catch (e) {
console.log(e, e.lineNumber);
}
return L2s;
}
So, if accessing your db is off the table (I have the same issue with my company), perhaps you can set up some data extract routines. Our company has hundred of them and requires basic authorization. I issue a web request and then I can write the data to google sheets or to my own MySQL db. Here is an example URL we use:
https://app.yourdomain.com/datadumper/report.ashx?reportid=5107&startDate=2021-04-08&endDate=2021-04-08&bu=3&currentuser=john.agusta&__action=Export#
Then you can code the app script as follows:
/*
https://modjeska.us/csv-google-sheets-basic-auth/
https://redfin.engineering/when-importdata-isnt-good-enough-retrieving-csv-files-behind-basic-auth-with-a-google-apps-script-6c563f3328c5
cm9iZXJ0Om9sZHRpcmVz -- this is a base 64-encoded string of the form
<username>:<password>. In this case, the value represents the username 'robert' and the password 'oldtires', so the string prior to encoding is: robert:oldtires
this site can be used to create a base 64-encoded string:
https://www.base64encode.net/
*/
const auth = '';
VAR URL = "https://app.yourdomain.com/datadumper/report.ashx?reportid=5107&startDate=2021-04-08&endDate=2021-04-08&bu=3&currentuser=john.agusta&__action=Export#";
var csvContents = getCSVContents(URL);
var parsedContents = parseCsvResponse(csvContents, true);
function getCSVContents(csvUrl) {
// request the CSV
var resp = UrlFetchApp.fetch(csvUrl, {
headers: {
// use basic auth
'Authorization': 'Basic ' + auth
}
});
return resp.getContentText();
}
// parse the CSV response
function parseCsvResponse(csvString, ignoreHeaders) {
var retArray = [];
var numCols = 0;
var i = 0;
var j = 0;
var line = "";
var strLines = csvString.split(/\n/g);
if (ignoreHeaders) {
strLines.shift();
}
var lenLines = strLines.length;
for (i = 0; i < lenLines; i++) {
line = strLines[i];
if (line != '') {
retArray.push(line.split(/,(?=(?:(?:[^"]*"){2})*[^"]*$)/));
numCols = retArray[i].length;
// remove outer double quotes
for (j = 0; j < numCols; j++) {
retArray[i][j] = retArray[i][j].replace(/^"|"$/g, '');
}
}
}
return retArray;
}

onSnapshot() Firestore sending changes multiple times with Flask-Sockets

I'm trying to develop a web chat with Flask and Firestore. I set a flow to receive new messages from firestore (when something changes at the database) and send through websockets to UI. Something like that:
Python:
#sockets.route('/messages')
def chat_socket(ws):
message = None
def callback_snapshot(col_snapshot, changes, read_time):
with app.app_context():
Messages = []
for change in changes:
if change.type.name == 'ADDED':
Messages.append(change.document)
conversation = render_template(
'conversation.html',
Messages = Messages,
)
numberID = None
if len(col_snapshot) > 0:
for i in col_snapshot:
a = i
numberID = a.reference.parent.parent.id
response = json.dumps({
'conversation': conversation,
'numberID': numberID
})
ws.send(response)
while not ws.closed:
response = json.loads(ws.receive())
newNumberID = response['newNumberID'].strip()
query_snapshot = fn.GetMessages(newNumberID)
doc_watch = query_snapshot.on_snapshot(callback_snapshot)
if message is None:
continue
Javascript:
function messages(numberID) {
var scheme = window.location.protocol == "https:" ? 'wss://' : 'ws://';
var webSocketUri = scheme
+ window.location.hostname
+ (location.port ? ':'+location.port: '')
+ '/messages';
/* Get elements from the page */
var form = $('#chat-form');
var textarea = $('#chat-text');
var output = $('.messages');
var status = $('.messages');
var websocket = new WebSocket(webSocketUri);
websocket.onopen = function() {};
websocket.onclose = function() {};
websocket.onmessage = function(e) {
numberID = JSON.parse(e.data).numberID
conversation = JSON.parse(e.data).conversation
output.append(conversation);
if (numberID == null){
output.empty();
}};
websocket.onerror = function(e) {console.log(e);};
websocket.onopen = () => websocket.send(numberID);
};
The problem is: When I use col_snapshot as Messages, everything is ok besides I get the whole firestore Collection sent every time to the user when a message is sent. So it's totally not efficient. When I set callback only for changes, as described above, if I trigger the function more than one time, somehow I set multiple listeners for the same collection, so I get multiple "changes updates" in UI. How I can keep track of those listeners so I only set one listener per Collection?

As you can see from the documentation, you should only call GetMessages and on_snapshot once per document.
#sockets.route('/messages')
def chat_socket(ws):
message = None
def callback_snapshot(col_snapshot, changes, read_time):
with app.app_context():
# Rest of the function ...
ws.send(response)
response = json.loads(ws.receive())
numberID = response['newNumberID'].strip()
query_snapshot = fn.GetMessages(numberID)
doc_watch = query_snapshot.on_snapshot(callback_snapshot)
while not ws.closed:
newNumberID = response['newNumberID'].strip()
response = json.loads(ws.receive())
if newNumberID != numberID:
numberID = newNumberID
query_snapshot = fn.GetMessages(numberID)
doc_watch = query_snapshot.on_snapshot(callback_snapshot)

Can not read JSON returned array in Swift like Python enumerate

I am trying to replicate the following lines of python code from the https://github.com/joshfraser/robinhood-to-csv repo from GitHub in order to read my transaction history.
orders = robinhood.get_endpoint('orders')
paginated = True
page = 0
while paginated:
for i, order in enumerate(orders['results']):
executions = order['executions']
instrument = robinhood.get_custom_endpoint(order['instrument'])
fields[i + (page * 100)]['symbol'] = instrument['symbol']
for key, value in enumerate(order):
if value != "executions":
fields[i + (page * 100)][value] = order[value]
if order['state'] == "filled":
trade_count += 1
for key, value in enumerate(executions[0]):
fields[i + (page * 100)][value] = executions[0][value]
elif order['state'] == "queued":
queued_count += 1
# paginate
if orders['next'] is not None:
page = page + 1
orders = robinhood.get_custom_endpoint(str(orders['next']))
else:
paginated = False
Where we also have
def get_endpoint(self, endpoint=None):
res = self.session.get(self.endpoints[endpoint])
return json.loads(res.content.decode('utf-8'))
I have thus been working on the following iOS code. I work with this code in an XCode playground so feel free to make one to follow along
import UIKit
import PlaygroundSupport
let LoginEndpoint:String = "https://api.robinhood.com/api-token-auth/"
let LoginRequestData:[String : String] = ["username": "EmailAdress", "password": "Password"]
let OrdersEndpoint:String = "https://api.robinhood.com/orders/"
func httpReq(type: String, url: String, body:[String : String], header:[String : String]) -> ([String : Any]?, Data?, String?){
let url = URL(string: url)
var returnData:([String : Any]?, Data?, String?)? = nil
if let url = url {
var request = NSMutableURLRequest(url: url) as URLRequest
request.httpMethod = type
var postString = ""
for (key, value) in body {
if (postString != "") {
postString += "&"
}
postString += "\(key)=\(value)"
}
request.httpBody = postString.data(using: .utf8)
for (key, value) in header {
request.addValue(value, forHTTPHeaderField: key)
}
let _ = URLSession.shared.dataTask(with: request, completionHandler: {(data, response, error) in
if let data = data {
do {
let jsonSerialized = try JSONSerialization.jsonObject(with: data, options: []) as? [String : Any]
returnData = (jsonSerialized, data, nil)
} catch (_) {
returnData = (nil, data, "JSON Parse Erro")
}
} else if let error = error {
returnData = (nil, nil, error.localizedDescription)
}
}).resume()
}
while (returnData == nil) {}
return returnData!
}
let tokenQuery = httpReq(type: "POST", url: LoginEndpoint, body: LoginRequestData, header: [:])
if let token = tokenQuery.0?["token"] {
print("token \(token)")
let historyQuery = httpReq(type: "GET", url: OrdersEndpoint, body: [:], header: ["Authorization": "Token \(token)"])
if let results = historyQuery.0?["results"], let countString = historyQuery.0?["count"] {
}
}
RunLoop.main.run()
PlaygroundPage.current.needsIndefiniteExecution = true
So as you can see I am using the auth token to get to the orders endpoint. I am indeed getting a good response from the orders endpoint but I have no clue how to interpret it.
It would seem from the python code that it is returning an array of JSON objects however I cant figure out how to get to that array in swift. I am not even sure if I am decoding it properly. The funny thing is when you look at the object returned in Playgrounds it would seem XCode knows that there is an array going on. How do I get to it?

Can't really get a good idea of the information being returned by your service, because of the lack of credentials.
However, check out SwiftyJSON. It's a really good library, and is extremely helpful with handling JSON data in Swift. It should solve your JSON handling issues.

Implement Cipher in Nodejs and Express

So, I'm going to be having a service that posts fields to my app and have to do the following:
When an action occurs within your account, several values are passed
along in the Instant Notification query string (see URL Parameters).
While building the string we create a sha1, or a hash of the values
passed, and your Secret Key. The result is the cverify parameter. Upon
receipt of the query string parameters, your system must also create a
sha1, or a hash of the values passed, and your Secret Key.
The validity of the data received is evaluated by using the cverify
parameter we send and the value produced in your system. Only if there
is an exact match between the two values can you be certain the
information received has not been tampered with.
I'm guessing I should use the crypto module. But, not sure how to do the rest of this. The params would come in via the req.body I know... but, not sure about the rest.
Below is their example from python
import hashlib
##
# Verify cverify from an ipn.
# #param post_params: A dictionary of all POST parameters from the notification
# #return: True if the cverify parameter is valid, false otherwise
def ipnVerification(post_params):
secret_key = "YOUR SECRET KEY"
pop = ""
ipn_fields = []
for key in post_params.keys():
if key == "cverify":
continue
ipn_fields.append(key)
ipn_fields.sort()
for field in ipn_fields:
pop += post_params[field] + "|"
pop += secret_key
return post_params["cverify"] == hashlib.sha1(pop).hexdigest()[:8].upper()
This is what I have so far:
var secretKey = 'My Secret Key';
module.exports = {
validateRequest: function(req){
var params = []
for (param in req.body) {
if (param == "cverify")
continue;
params.push(param);
}
params.sort();
var pop = "";
for (param in params) {
pop += req.body.param + "|";
}
pop += secretKey;
var cverify = req.body.cveryify;
// do crypto stuff
console.log(params);
}
}

If I understand the code correctly:
var secretKey = 'My Secret Key';
module.exports = {
validateRequest : function(req) {
/* shorter version of what you have already:
var keys = Object.keys(req.body)
.filter(function(key) { return key !== 'cverify'; })
.sort();
var pop = keys.map(function(key) {
return body[key];
}).join('|') + '|' + secretKey;
*/
...
var sha1 = require('crypto').createHash('sha1');
sha1.update(pop);
var digest = sha1.digest('hex').substring(0, 8).toUpperCase();
return digest === req.body.cverify;
}
};