We Keep Coding

Python API script

I am making a python script using API of a free test automation website called TestProject.
Link to their API: https://api.testproject.io/docs/v2/
Basically what i want to do is grab pdf of reports of all tests and save them somewhere.
But to make the GET request to do that i first need projectID and jobID which i already wrote functions getting them and saving them in the array.
But now i have a problem where its looping through both lists and not using correct projectID and jobID and its throwing errors because it does not exist.
So what i need is something to check if jobID is in projectID so that way i can make a GET request to get all the executionID's to get the PDF of the report.
I am kinda new to programming so i would love any help i can get. If anyone has any better solutions please feel free to let me know.
My script:
import requests
import json
import csv
from datetime import datetime
from jsonpath_ng import jsonpath, parse
API_key = 'api_key'
headers = {'Authorization':'{}'.format(API_key)}
list_projectId = []
list_jobId = []
list_executionId = []
ParseData_projectId = parse('$..id')
ParseData_jobId = parse('$..id')
ParseData_executionId = parse('$..id')
def parsing (response,ParseData,list_data):
# parses data and appends it to the list
Data = json.loads(response)
Parsaj = ParseData
Podatki = Parsaj.find(Data)
for i in range(0, len(Podatki)):
vrednost = Podatki[i].value
list_data.append(vrednost)
def projectId():
# gets all projectId's and saves them in list_projectId
url = 'https://api.testproject.io/v2/projects?_start=0'
response = requests.get(url,headers=headers)
response_json = response.json()
converted = json.dumps(response_json)
parsing(converted,ParseData_projectId,list_projectId)
def jobId():
# gets all jobId's and saves them in list_jobId
for i in range(0, len(list_projectId)):
id = list_projectId[i]
url = 'https://api.testproject.io/v2/projects/{}'.format(id) + '/jobs?onlyScheduled=false&_start=0'
response = requests.get(url,headers=headers)
response_json = response.json()
converted = json.dumps(response_json)
parsing(converted,ParseData_jobId,list_jobId)
def executionId():
# Their API link:
# https://api.testproject.io/v2/projects/{projectId}/jobs/{jobId}/reports?_start=0
# the for loop below does not work here is where i need the help:
for i in range(0, len(list_projectId)):
project_id = list_projectId[i]
job_id = list_jobId[i]
url = 'https://api.testproject.io/v2/projects/{}'.format(project_id) + '/jobs/{}'.format(job_id) + '/reports?_start=0'
response = requests.get(url,headers=headers)
response_json = response.json()
converted = json.dumps(response_json)
parsing(converted,ParseData_executionId,list_executionId)
projectId()
print("----------LIST PROJECT ID: ----------")
print(list_projectId)
print("")
jobId()
print("----------LIST JOB ID: ----------")
print(list_jobId)
executionId()
print("----------LIST EXECUTION ID: ----------")
print(list_executionId)

you have to use 'in' operator to check the value exist in the list data structure.

Is there a way to aggregate the number of times a keyword is mentioned in a Subreddit over time?

import requests
import pandas as pd
def get_pushshift_data(data_type, **kwargs):
"""
Gets data from the pushshift api.
"""
base_url = f"https://api.pushshift.io/reddit/search/{data_type}/"
payload = kwargs
request = requests.get(base_url, params=payload)
return request.json()
#paramaters#
data_type="comment"
query="chicken"
duration="30d"
size=1000
aggs="subreddit"
data = get_pushshift_data(data_type=data_type,
q=query,
after=duration,
size=size,
aggs=aggs)
data = data.get("aggs").get(aggs)
print(data[1])
This script produces a 'doc_count' and a 'key' variable, representing the number of comments ('doc_count') that appear within a Subreddit ('key') that mention our defined query term over or defined time-period (duration).
Is there any way to add a time variable to this data so that we can see the top 20 Subreddits that our query term appeared in each day for a defined duration of time?
I'm ultimately trying to see the change in Subreddit activity over time for our query term (so maybe I could incorporate this into a visualization that has a time animation etc.).

You can use after and before param as mentioned in documentation https://pushshift.io/api-parameters/
for example:
import requests
import pandas as pd
import time
def get_pushshift_data(data_type, **kwargs):
"""
Gets data from the pushshift api.
"""
base_url = f"https://api.pushshift.io/reddit/search/{data_type}/"
payload = kwargs
request = requests.get(base_url, params=payload)
return request.json()
#paramaters#
data_type="comment"
query="chicken"
# duration="30d"
size=1000
aggs="subreddit"
after = int(time.time()) - 60 * 60 * 24 * 2
before = int(time.time()) - 60 * 60 * 24
data = get_pushshift_data(data_type=data_type,
q=query,
size=size,
aggs=aggs,
after=after,
before=before)
data = data.get("aggs").get(aggs)
print(data[1])

Microsoft LUIS: unable to set time zone (datetimeReference) for datetimeV2 entities

I am using the V3 API to get predictions from a LUIS endpoint and I need a way to tell LUIS my time zone, so that relative time expressions (e.g. "in the past two hours", "in 10 minutes") are resolved properly by the datetimeV2 entity.
Everything works perfectly if I use the V2 API with the timezoneOffset option, but I am unable to make the V3 API work with the new option datetimeReference (which is supposed to replace timezoneOffset). Actually, I could not even figure out which value I should set for datetimeReference (an integer number? A datetime?).
Here are my attempts with Python. Can anyone tell me if there is anything wrong?
from datetime import datetime
import requests
appId = # my app id
subscriptionKey = # my subscription key
query = "tra 10 minuti" # = "in 10 minutes" (my app speaks Italian)
# ATTEMPT 1
# based on https://learn.microsoft.com/en-us/azure/cognitive-services/luis/luis-concept-data-alteration?tabs=V2#change-time-zone-of-prebuilt-datetimev2-entity,
# assuming it works the same way as timezoneOffset
endpoint = 'https://westeurope.api.cognitive.microsoft.com/luis/prediction/v3.0/apps/{appId}/slots/staging/predict?datetimeReference=120&subscription-key={subscriptionKey}&query={query}'
endpoint = endpoint.format(appId = appId, subscriptionKey = subscriptionKey, query = query)
response = requests.get(endpoint)
# ATTEMPT 2
# according to https://learn.microsoft.com/en-us/azure/cognitive-services/luis/luis-migration-api-v3
endpoint = 'https://westeurope.api.cognitive.microsoft.com/luis/prediction/v3.0/apps/{appId}/slots/staging/predict?'
endpoint = endpoint.format(appId = appId)
json = {
"query" : query,
"options":{
"datetimeReference": datetime.now().strftime("%Y-%m-%dT%H:%M:%S"), # e.g. "2020-05-07T13:54:33". Not clear if that's what it wants
"preferExternalEntities": True
},
"externalEntities":[],
"dynamicLists":[]
}
response = requests.post(endpoint, json, headers = {'Ocp-Apim-Subscription-Key' : subscriptionKey})
UPDATE: the correct way of sending the request in ATTEMPT 2 is
response = requests.post(endpoint, json = json, headers = {'Ocp-Apim-Subscription-Key' : subscriptionKey})

As you've discovered, your JSON should go in the json argument and not the data argument:
response = requests.post(endpoint, json = json, headers = {'Ocp-Apim-Subscription-Key' : subscriptionKey})

How can I limit python script to 1 API trade per day?

I'm putting together a python script to make trades on poloniex with the API, and so far I've got it to make trades when certain conditions are met, but I still need it to NOT place anymore trades for the rest of that day (I have the entire script looping every 60 seconds).
So far I have this script:
import requests
import urllib.request
import urllib.parse
import http.client
import hashlib
import hmac
import time
import json
from urllib.request import urlopen
The_Currency_Pair = input('Which Currency Pair?\nPAIRS TO CHOOSE FROM:\nUSDT_BTC\nUSDT_XRP\nUSDT_ETH\nUSDT_BCH\nUSDT_STR\nUSDT_LTC\nUSDT_ETC\nUSDT_XMR\n')
api = 'https://poloniex.com/tradingApi'
key = 'XXXXXXXXXXXXXXXXXXXXXXXXXXXXX'
secret = 'XXXXXXXXXXXXXXXXXXXXXXXXX'
def main():
poloniexPrices = urlopen('https://poloniex.com/public?command=returnTicker').read()
poloniexjson = json.loads(poloniexPrices)
poloniexlastP = poloniexjson[The_Currency_Pair]['last']
poloniexOCHL = urlopen('https://poloniex.com/public?command=returnChartData&currencyPair=USDT_BCH&start=1538352000&period=86400').read()
poloniexOCHLjson = json.loads(poloniexOCHL)
poloniexlasthigh = poloniexOCHLjson[-2]['high']
print ('Last Price')
print (poloniexlastP)
print ('----------------------------------------')
print ('Last Day High')
print (poloniexlasthigh)
print ('----------------------------------------')
data = {
'command': 'returnBalances',
'nonce' : int(time.time() * 1000)
}
data = urllib.parse.urlencode(data).encode()
signature = hmac.new(secret.encode(), data, hashlib.sha512)
headers = {
'Key' : key,
'Sign': signature.hexdigest()
}
request = urllib.request.Request(
url=api, data=data, headers=headers, method='POST'
)
text = urllib.request.urlopen(request).read().decode()
print ('MY ACCOUNT BALANCE')
try:
print(json.loads(text)['USDT'])
except:
print(text)
print ('-----------------------------------------')
if float(poloniexlastP) > 0:
print ('PLACING TRADE')
print ('-----------------------------------------------')
parms = {"command":"buy",
"currencyPair":The_Currency_Pair,
"rate":100,
"immediateOrCancel":1,
"amount":0.01,
"nonce":int(time.time() * 1000)}
parms = urllib.parse.urlencode(parms).encode()
signature = hmac.new(secret.encode(), parms, hashlib.sha512)
headers = {'Key' : key,
'Sign': signature.hexdigest()}
request = urllib.request.Request(
url=api, data=parms, headers=headers, method='POST')
text = urllib.request.urlopen(request).read().decode()
ordernumber = (json.loads(text)['orderNumber'])
print ('Order Number:')
print (ordernumber)
while True:
main()
time.sleep(60)
Anyway, after a trade has been placed, I need it to make sure that after the 60 second sleep, it doesn't make a second trade unless it is a new day/the day after the trade was made. (Could I use poloniex server time for this?)
So, if it has got as far as print (ordernumber) that means it has placed a trade. But how do I mark it as placed trade for the day or something and use it in the if float(poloniexlastP) > 0: for the next loop to make sure it doesn't place another one?

Maybe you can use Python to get the date, and create a global variable, and after the print statement, you can set the variable to the current date, and the coffee will check if it has already sent, that way it doesn't execute more than once in a day.
import datetime
# This Gets The Day Of The Month
todaysDateNumber = int(datetime.datetime.now().strftime("%d"))
dateNumberTradeSent = 0
if todaysDateNumber == dateNumberTradeSent:
print("The API has already been used once today, try again tomorrow!")
return
else:
# Call Your Trade Sending Code Here
# After The Print Statement That Shows That The Trade Was Sent:
global dateNumberTradeSent
dateNumberTradeSent = int(datetime.datetime.now().strftime("%d"))

Python 3, Ethereum - how to send ERC20 Tokens?

i have some script using to sending Ethers from address to addres. Im using Parity, and Python 3.6. It is using Flask looks like:
from flask import Flask, render_template, json, request
import urllib
import requests
import binascii
from decimal import *
app = Flask(__name__)
def Eth(method,params=[]):
data = {"method":method,"params":params,"id":1,"jsonrpc":"2.0"}
headers = {'Content-type': 'application/json'}
r = requests.post(ethrpc, data=json.dumps(data), headers=headers)
r = r.text
response = json.loads(r)
return(response)
hot = str("XXXXXXX")
#app.route('/')
def index():
ethnumbers = int(10)**int(18)
hot = str("XXXXX")
balance = Eth("eth_getBalance",[hot,'latest'])
balance = balance["result"]
balance = int(balance, 16)
balance = float(balance)
balance = balance / ethnumbers
balance = str(balance)
return render_template('index.html',hot = hot,balance=balance)
#app.route('/send/',methods=['POST','GET'])
def send():
getcontext().prec = 28
ethnumbers = Decimal(10)**Decimal(18)
print(ethnumbers)
if request.method == "POST":
_myaddress = request.form['_myaddress']
_youraddress = request.form['_youraddress']
_amount = request.form['_amount']
_gas = request.form['_gas']
_gas = hex(int(_gas))
passy = str("XXXXXXXXX")
getcontext().prec = 28
_amount = Decimal(_amount)
getcontext().prec = 28
_amount = _amount * ethnumbers
getcontext().prec = 28
_amount = int(_amount)
_amount = hex(_amount)
r = [{'from':_myaddress,"to":_youraddress,"value":_amount,"gas":_gas,},str("XXXXXXXXXX!")]
print(r)
json.dumps(r)
resultio = Eth("personal_sendTransaction",r)
try:
resultio["result"]
return render_template('sent.html',resultio=resultio["result"])
except: KeyError
return render_template('sent.html',resultio=resultio["error"]["message"])
else:
return render_template('index.html')
if __name__ == "__main__":
app.run()
Im pretty sure, that i have to use "data" to do this, but i have no idea how to send via this script ERC20 tokens. Structure of tokens transaction looks like "my address -> token address -> token receiver".
Any ideas?

Guys it was simpler than it looks like.
You just have to put:
contract address
as receiver and make a long "data" field, it represents string as:
// method name, its constans in erc20
0xa9059cbb
//receiver address (you have to do it without "0x" because its needed only when
//you have to tell "im using hexadecimal". You did it above, in method field.
//so, receiver address:
5b7b3b499fb69c40c365343cb0dc842fe8c23887
// and fill it with zeros, it have to be lenght 64. So fill rest of address
0000000000000000000000005b7b3b499fb69c40c365343cb0dc842fe8c23887
// then you need amount, please convert it to hexadecimal, delete "0x" and
// remember, you need to integer first, so if token has 18 "decimals" it need
// to be amount / 10**18 first!!
//1e27786570c272000 and fill that with zeros, like above:
000000000000000000000000000000000000000000000001e27786570c272000
//just add string, to string, to string, and you have data field:
0xa9059cbb0000000000000000000000005b7b3b499fb69c40c365343cb0dc842fe8c23887000000000000000000000000000000000000000000000001e27786570c272000
Sample transaction: https://etherscan.io/tx/0x9c27df8af24e06edb819a8d7a380f548fad637de5edddd6155d15087d1619964

web3.py is definitely the way to go. If you want to do it by hand and you just want to call the standard ERC-20 transfer method, the from address should stay the same, the to address should be the token contract, and then the data should be the following concatenated together and formatted as hexadecimal:
The first 4 bytes of the keccak256 hash of "transfer(address,uint256)", which is the function's signature.
The address of the recipient, left-zero-padded to be 32 bytes.
The amount to be transferred. (Be sure to take into account the token's decimals... 1 token is often 10**18, but the number of decimal places varies from token to token and can be retrieved by calling the decimals() function.) This should also be formatted as a 32-byte number (so left-zero-padded).
web3.py would be much easier. :-) Something like:
web3.eth.contract(address, abi=standard_token_abi).sendTransaction({
'from': from_address
}).transfer(to_address, amount)