I want to set up a recursive loop to iterate over different levels of a transaction tree until a predefined depth is reached.
Example:
A transfers funds to B and C (depth 1)
- B transfers funds to D and E (depth 2)
- C transfers funds to F and G (depth 2)
etc.
To do so, I have set up the following function:
def traverse(target, depth):
try:
print('Starting analysis of parent address {}'.format(target))
result=callAddress(target)
inner_transaction_targets=analyseTransaction(result)
traversed_addresses=compileAddresses(compiled_receiver_addresses,inner_transaction_targets) #Compile list of start addresses
if depth > 0:
for inner in inner_transaction_targets:
print('Starting analysis of child address {} at depth {}.'.format(inner,depth))
inner_transaction_targets=inner_transaction_targets.pop(0)
print(inner_transaction_targets)
return traverse(inner, depth-1)
else:
return traversed_addresses
except:
print('Could not traverse.')
The functions callAdress, analyseTransaction and compileAdresses themselves should not be relevant to this problem. However to offer a brief description:
analyseTransaction will offer a list of all receiver addresses involved in a given transaction
compileAdresses collects the scanned addresses in a master list
See here the other functions:
def callAddress(bitcoin_address): #This function serves to retrieve the JSON respose from blockchain.info
try:
r = requests.get("https://blockchain.info/address/{}?format=json".format(bitcoin_address))
unparsed=r.content
parsed = json.loads(unparsed)
return parsed
except: #unparsed.status=="400":
print ("Incorrect bitcoin address")
def compileAddresses(compiled_receiver_addresses,input_addr): #This function serves to add extracted transactions to a master list (compiled_receiver_addresses)
try:
compiled_receiver_addresses.append(input_addr)
return compiled_receiver_addresses
except:
print('Failed to compile')
def analyseTransaction(response): #This function serves to extract the wallet addresses that the specified wallet has sent funds to (transaction targets)
try:
transaction_targets=[]
wallet_transactions=response['txs']
for txs_entry in wallet_transactions:
print ("{:=^22}".format(""))
print ("Checking outgoing transaction with Hash: {}".format(txs_entry['hash']))
print ("Transaction occured at: {}".format(datetime.fromtimestamp(
int(txs_entry['time']) ).strftime('%Y-%m-%d %H:%M:%S')))
print ("{:=^22}".format(""))
for out in txs_entry['out']:
outbound_addr=out['addr']
print ("In this transaction funds were sent to this wallet: {}".format(out['addr']))
print ("Amount sent: {}".format(int(out['value']) * 10**-8))
transaction_targets.append(outbound_addr)
cleaned_transaction_targets = list(dict.fromkeys(transaction_targets))
return cleaned_transaction_targets
except:
print('Dead End')
My issue is that the recursive loop keeps getting stuck on the first value, i.e. it is looping the very first value of "inner" over and over again instead of going down the list that is inner_transaction_targets
I have tried to drop the very first value in this list by using inner_transaction_targets.pop(0) but to no effect.
How do I have to modify this function so that it does what I want?
Your code has some errors that are most likely hindering its successful completion, such as:
The return is within a loop.
The object returned is in fact a self-reference to the traverse() function.
The .pop() method is not a good practice.
I would suggest you create a separate function that will return a single object for each inner transaction, much like what compile_addresses() does.
You then add this new function into the for loop so that it iterates over the inner_transaction_targets and appends the results to a list. Finally, you return the list with all the results, indented aligned with the if statement, outside the for loop.
Related
I am making a recursive function for retrieving outer ethereum accounts. Once I loop through the first list of accounts, I want to take the first one and show the accounts he sent money to, then go back to previous interation take second account and so on.. problem is I don't know how to make my function stick to the for loop, because current account get replaced after the iteration..Any ideas
tx = get_transactions(start_address)
def breadth_search(transactions):
if transactions:
accounts = []
for i, tran in enumerate(transactions):
print(i, tran)
if tran['tx_to'] not in accounts:
accounts.append(tran['tx_to'])
return accounts
def some_func(trans):
tx_list = breadth_search(trans)
for i, tr in enumerate(tx_list):
print(i)
breadth_search(get_transactions(tr))
for i, tr in enumerate(tx_list):
some_func(get_transactions(tr))
I have an application with one producer and many consumers, and a queue, which communicates them.
The consumer should collect some data from queue, let's say qsize()/number_of_consumers, but it must stop it work when a sentinel appears.
I have such a code:
frame = 0
elems_max = 10
while frame is not None:
frames = []
for _ in range(elems_max):
frame = queue_in.get()
if frame:
frames.append(frame)
else:
break
process_data(frames)
As You can see None is a sentinel for this queue, and when it appears I wan't to break my working process. I also want to get more then one element for data processing.
What is the fastest method to achieve this [in python 3.5]?
I understand that you want to break your outer while when encountering a None.
You can hold a boolean variable that is True while the while must execute, and False when it should stop.
This would look like this:
frame = 0
elems_max = 10
running = True
while running and frame is not None:
frames = []
for _ in range(elems_max):
frame = queue_in.get()
if frame is not None:
frames.append(frame)
else:
running = False
break
process_data(frames)
The break instruction will break the inner for, but not the outer while.
However, having set running to False, the while loop will stop.
Based on your comment.
It is not possible to include a break statement in a comprehension, nor an else clause, as you wanted to do:
frames = [f for i in range(elems_max) if queue_in.get() is not None else break]
However, you can build your list, and then remove all the elements after a None:
frames = [queue_in.get() for _ in range(elems_max)]
try:
noneId = frames.find(None)
frames = frames[:noneId]
except ValueError:
pass
This is not very efficient, because potentially many elements will be appended in frames for nothing.
I would prefer a manual construction, to avoid this hazard.
One more solution, based on a generator.
This might not be what you expected, but the syntax is rather simple, so you may like it.
The idea is to wrap the getting of the data inside of a generator, that breaks on a None value:
def queue_data_generator(queue, count):
for _ in range(count):
item = queue.get()
if item is None:
raise StopIteration
else:
yield item
Then, instantiate this generator, and simply iterate over it:
g = queue_data_generator(queue_in, elems_max)
frames = [frame for frame in g]
The frames list will contain all the frames contained in queue_in, until the first None.
The usage is rather simple, but you have to setup it by defining the generator.
I think it's pretty elegant though.
I would do next (kinda pseudocode):
class CInputQueue:
def get(self, preferred_N):
# do sync stuff
# take <= N elements (you can do kinda balance the load)
# or throw exception
raise Exception("No data, no work, no life.")
elems_max = 10
try:
while True:
process_data(queue_in.get(elems_max))
except:
None # break
I assume that data processing takes much more time, than 0 ms, so I use exception. I know that it's not okay use exceptions for flow control, but for worker it's really exception. His "life" build around processing data, but their is no work for him, no even sleep task.
I'm trying to check if an object has a skinCluster on it. My code is pretty basic. Here's an example:
cmds.select(d=True)
joint = cmds.joint()
skinnedSphere = cmds.polySphere(r=2)
notSkinnedSphere = cmds.polySphere(r=2)
skinTestList = [skinnedSphere, notSkinnedSphere]
# Bind the joint chain that contains joint1 to pPlane1
# and assign a dropoff of 4.5 to all the joints
#
cmds.skinCluster( joint, skinnedSphere, dr=4.5)
for obj in skinTestList:
objHist = cmds.listHistory(obj, pdo=True)
skinCluster = cmds.ls(objHist, type="skinCluster")
if skinCluster == "":
print(obj + " has NO skinCluster, skipping.")
else:
print obj, skinCluster
#cmds.select(obj, d=True)
My issue is that even if it can't find a skincluster, it still prints out the "obj, skincluster" rather than the error that it can't find a skinCluster.
I thought a skinCluster returns a string. So if the string is empty, it should print out the error rather than "obj, skincluster".
Any help would be appreciated!
This is a classic Maya issue -- the problem is that Maya frequently wants to give you lists, not single items, even when you know the result ought to be a single item. This means you end up writing a bunch of code to either get one item from a one-item list or to avoid errors that come from trying to get an index into an empty list.
You've got the basics, it's the == "" which is messing you up:
for obj in skinTestList:
objHist = cmds.listHistory(obj, pdo=True)
skinCluster = cmds.ls(objHist, type="skinCluster") or [None]
cluster = skinCluster[0]
print obj, cluster
The or [None] guarantees that you'll always get a list with something in it so it's safe to use the [0] to get the single value. None is a good return value here because (as pointed out in the comments) you can if cluster: and skip empty values.
I have a couple of functions that are being called recursively inside nested loops. The ultimate objective of my program is to:
a) loop through each year,
b) within each each year, loop through each month (12 total),
c) within each month, loop through each day (using a self generated day counter),
d) and read 2 files and merge them together into a another file.
In each instance, I am going down into the directory only if exists. Otherwise, I'm to just skip it and go to the next one. My code does a pretty good job when all the files are present, but when one of the files is missing, I would like to just simply skip the whole process of creating a merged file and continue the loops. The problem I am getting is a syntax error that states that continue is not properly in the loop. I am only getting this error in the function definitions, and not outside of them.
Can someone explain why I'm getting this error?
import os, calendar
file01 = 'myfile1.txt'
file02 = 'myfile2.txt'
output = 'mybigfile.txt'
def main():
#ROOT DIRECTORY
top_path = r'C:\directory'
processTop(top_path)
def processTop(path):
year_list = ['2013', '2014', '2015']
for year in year_list:
year_path = os.path.join(path, year)
if not os.path.isdir(year_path):
continue
else:
for month in range(1, 13):
month_path = os.path.join(year_path, month)
if not os.path.isdir(month_path):
continue
else:
numDaysInMth = calendar.monthrange(int(year), month)[1]
for day in range(1, numDaysInMth+1):
processDay(day, month_path)
print('Done!')
def processDay(day, path):
day_path = os.path.join(path, day)
if not os.path.isdir(day_path):
continue
else:
createDailyFile(day_path, output)
def createDailyFile(path, dailyFile):
data01 = openFile(file01, path)
data02 = openFile(file02, path)
if len(data01) == 0 or len(data02) == 0:
# either file is missing
continue
else:
# merge the two datalists into a single list
# create a file with the merged list
pass
def openFile(filename, path):
# return a list of contents of filename
# returns an empty list if file is missing
pass
if __name__ == "__main__": main()
You can use continue only plainly inside a loop (otherwise, what guarantee you have that the function was called in a loop in the first place?) If you need stack unwinding, consider using exceptions (Python exception handling).
I think you can get away with having your functions return a value that would say if operation was completed successfully:
def processDay(day, path):
do_some_job()
if should_continue:
return False
return True
And then in your main code simply say
if not processDay(day, path):
continue
You are probably getting that error in processDay and createDailyFile, right? That's because there is no loop in these functions, and yet you use continue. I'd recommend using return or pass in them.
The continue statement only applies in loops as the error message implies if your functions are structured as you show you can just use pass.
continue can only appear in a loop since it tells python not to execute the lines below and go to the next iteration. Hence, this syntax here is not valid :
def processDay(day, path):
day_path = os.path.join(path, day)
if not os.path.isdir(day_path):
continue # <============ this continue is not inside a loop !
else:
createDailyFile(day_path, output)enter code here
Same for your createDailyFile function.
You may want to replace it with a return ?
I have a problem in which I process documents from files using python generators. The number of files I need to process are not known in advance. Each file contain records which consumes considerable amount of memory. Due to that, generators are used to process records. Here is the summary of the code I am working on:
def process_all_records(files):
for f in files:
fd = open(f,'r')
recs = read_records(fd)
recs_p = (process_records(r) for r in recs)
write_records(recs_p)
My process_records function checks for the content of each record and only returns the records which has a specific sender. My problem is the following: I want to have a count on number of elements being returned by read_records. I have been keeping track of number of records in process_records function using a list:
def process_records(r):
if r.sender('sender_of_interest'):
records_list.append(1)
else:
records_list.append(0)
...
The problem with this approach is that records_list could grow without bounds depending upon the input. I want to be able to consume the content of records_list once it grows to certain point and then restart the process. For example, after 20 records has been processed, I want to find out how many records are from 'sender_of_interest' and how many are from other sources and empty the list. Can I do this without using a lock?
You could make your generator a class with an attribute that contains a count of the number of records it has processed. Something like this:
class RecordProcessor(object):
def __init__(self, recs):
self.recs = recs
self.processed_rec_count = 0
def __call__(self):
for r in self.recs:
if r.sender('sender_of_interest'):
self.processed_rec_count += 1
# process record r...
yield r # processed record
def process_all_records(files):
for f in files:
fd = open(f,'r')
recs_p = RecordProcessor(read_records(fd))
write_records(recs_p)
print 'records processed:', recs_p.processed_rec_count
Here's the straightforward approach. Is there some reason why something this simple won't work for you?
seen=0
matched=0
def process_records(r):
seen = seen + 1
if r.sender('sender_of_interest'):
matched = match + 1
records_list.append(1)
else:
records_list.append(0)
if seen > 1000 or someOtherTimeBasedCriteria:
print "%d of %d total records had the sender of interest" % (matched, seen)
seen = 0
matched = 0
If you have the ability to close your stream of messages and re-open them, you might want one more total seen variable, so that if you had to close that stream and re-open it later, you could go to the last record you processed and pick up there.
In this code "someOtherTimeBasedCriteria" might be a timestamp. You can get the current time in milliseconds when you begin processing, and then if the current time now is more than 20,000ms more (20 sec) then reset the seen/matched counters.