We Keep Coding

Python looping list and append value to variable

I have a .csv file with IPs which I converted into a list with Python:
def ip_list():
iplist = []
with open("/path/to/file") as csvfile:
csvlist = csv.reader(csvfile)
for lists in csvlist:
for item in lists:
iplist.append(item)
return iplist
ip = ip_list()
print(ip)
>>> ["192.168.1.1", "192.168.1.2", ...]
Now I want to have every value in the list and append them to a given parameter each time.
Function for context:
def gencontent(ip, value1, value2, time):
content = [
{
"example": {
"ipadress": ip
}
}
]
return content
ip = ["192.168.1.1", "192.168.1.2", "192.168.1.3"]
content = getcontent(ip[0-...], value1, value2, time)
I want loop content with each value in ip:
#Example list for reproduction
ip = ["192.168.1.1", "192.168.1.2", "192.168.1.3"]
content = getcontent(ip[0-...], ...)
I do not want:
#Example list for reproduction
ip = ["192.168.1.1", "192.168.1.2", "192.168.1.3"]
content1 = getcontent(ip[0], ...)
content2 = getcontent(ip[1], ...)
...
I want to loop content basically each time with a new ip value.
Thanks!

I don't know what the getcontent() function does, but why not loop through the items in your list using a list comprehension?
content = [getcontent(x) for x in ip]

If you simply want to index them, maybe you could convert to a tuple and use enumerate.
For example:
ip = ["192.168.1.1", "192.168.1.2", "192.168.1.3"]
indexed_ip = enumerate(tuple(ip))
print(list(indexed_ip))
# OUTPUT:
# [(0, '192.168.1.1'), (1, '192.168.1.2'), (2, '192.168.1.3')]
Or if you want the index to start at 1, instead of 0:
ip = ["192.168.1.1", "192.168.1.2", "192.168.1.3"]
indexed_ip = enumerate(tuple(ip), 1)
print(list(indexed_ip))
# OUTPUT:
# [(1, '192.168.1.1'), (2, '192.168.1.2'), (3, '192.168.1.3')]
Alternatively, maybe a dictionary work for you in this situation.
Here’s an example using dictionary comprehension:
ip_dict = { ip.index(ip_item): ip_item for ip_item in ip}
print(ip_dict)
# OUTPUT:
# {0: '192.168.1.1', 1: '192.168.1.2', 2: '192.168.1.3'}
You can name the keys for the dictionary, whatever you’d like. if you’re sent on content0, content1, etc, you could change the key value in the dict comprehension to something like f’content{str(ip.index(ip_item))}’. Then you could get the value from the ip_dict using ip_dict['content1'] and etc.

can you be more specific about content = getcontent(ip[0-...])?
i don't know whether i get you.
maybe something like this?
ip = ["192.168.1.1", "192.168.1.2", "192.168.1.3"]
def getip(li):
for item in li:
yield(item)
ipgetter = getip(ip)
content = getcontent(next(ipgetter), value1, value2, time) # getcontent got "192.168.1.1"
content = getcontent(next(ipgetter), value1, value2, time) # getcontent got "192.168.1.2"
if loop is in an end, an StopIteration Exception will being raised

Read a file and match lines above or below from the matching pattern

I'm reading an input json file, and capturing the array values into a dictionary, by matching tar.gz and printing a line above that (essentially the yaml file).
{"Windows": [
"/home/windows/work/input.yaml",
"/home/windows/work/windows.tar.gz"
],
"Mac": [
"/home/macos/required/utilities/input.yaml",
"/home/macos/required/utilities.tar.gz"
],
"Unix": [
"/home/unix/functional/plugins/input.yaml",
"/home/unix/functional/plugins/Plugin.tar.gz"
]
goes on..
}
Output of the dictionary:
{'/home/windows/work/windows.tar.gz': '/home/windows/work/input.yaml',
'/home/macos/required/utilities/utilities.tar.gz' : '/home/macos/required/input.yaml'
......
}
Problem being, if the entries of json changes, i.e. A) tar.gz entries can come as the 1st element in the list of values or B. or, its mix and match,
Irrespective of the entries, how can I get the output dictionary to be of above mentioned format only.
{ "Windows": [
"/home/windows/work/windows.tar.gz",
"/home/windows/work/input.yaml"
],
"Mac": [
"/home/macos/required/utilities/utilities.tar.gz",
"/home/macos/required/input.yaml"
],
"Unix": [
"/home/unix/functional/plugins/Plugin.tar.gz",
"/home/unix/functional/plugins/input.yaml"
]
goes on.. }
mix and match scenario.
{ "Windows": [
"/home/windows/work/windows.tar.gz",
"/home/windows/work/input.yaml"
],
"Mac": [
"/home/macos/required/utilities/input.yaml",
"/home/macos/required/utilities.tar.gz"
],
"Unix": [
"/home/unix/functional/plugins/Plugin.tar.gz",
"/home/unix/functional/plugins/input.yaml"
] }
My code snippet.
def read_input():
files_to_be_processed = {}
with open('input.json', 'r') as f:
lines = f.read().splitlines()
lines = [line.replace('"', '').replace(" ", '').replace(',', '') for line in lines]
for index, value in enumerate(lines):
match = re.match(r".*.tar.gz", line)
if match:
j = i-1 if i > 1 else 0
for k in range(j, i):
read_input[match.string] = lines[k]
print(read_input)

A method here is to have the following:
1- Using the JSON class in python makes your whole process much easier.
2- After taking the data in the JSON class, you can check each object (aka Windows/Max/Unix), for both the tar-gz and the yaml
3- Assign to new dictionary
Here is a quick code:
import json
def read_input():
files_to_be_processed = {}
with open('input.json','r') as f:
jsonObject = json.load(f)
for value in jsonObject.items():
tarGz = ""
Yaml = ""
for line in value[1]: #value[0] contains the key (e.g. Windows)
if line.endswith('.tar.gz'):
tarGz = line
elif line.endswith('.yaml'):
Yaml = line
files_to_be_processed[tarGz] = Yaml
print(files_to_be_processed)
read_input()
This code can be shortened and optimised using things like list comprehension and other methods, but it should be a good place to get started

One way could be for you to transform the list within your input json_dict into a dict that has a key for "yaml" and "gz"
json_dict_1 = dict.fromkeys(json_dict, dict())
for key in json_dict:
list_val = json_dict[key]
for entry in list_val:
entry_key = 'yaml' if 'yaml' in entry[-4:] else 'gz'
json_dict_1[key][entry_key] = entry
print(json_dict_1)
#{'Windows': {'yaml': '/home/unix/functional/plugins/input.yaml',
# 'gz': '/home/unix/functional/plugins/Plugin.tar.gz'},
# 'Mac': {'yaml': '/home/unix/functional/plugins/input.yaml',
# 'gz': '/home/unix/functional/plugins/Plugin.tar.gz'},
# 'Unix': {'yaml': '/home/unix/functional/plugins/input.yaml',
# 'gz': '/home/unix/functional/plugins/Plugin.tar.gz'}}

Passing multiple values from dictionary into method using loop

i have a question regarding adding dictionary key and value to method using loop
This is what i was thinking to write but it doesn't work how i want because it creates a packet just with one key/value every time
for key in packetData:
for name in packetData[key]:
packets = Ether()/IP()/UDP()/createsPacket(key, name=packetData[key][name])
print ("as name " + name + " \n as value " + str(packetData[key][name]))
Instead of writing this manually like that :
packets1 = Ether()/IP()/UDP()/createsPacket("65", UserID = "name", Password = "pass123", ETX = 123)
packets2 = Ether()/IP()/UDP()/createsPacket("72", PriceID = 123, Side = 12, MaxAmount = 123, MinAmount = 123, Price = 123000)
    json then converted to dictionary in python , this is data that i want to pass in
{
"65":{
"UserID":"vcjazfan",
"Password":"ejujwlhk",
"SessionID":115,
"ETX":192
},
"66":{
"UserID":"dzmtrssy",
"SessionID":35,
"Reason":"zbwivjcv",
"ETX":43
},
"72":{
"InstrumentIndex":171,
"PriceID":217,
"Side":226,
"MaxAmount":210,
"MinAmount":219,
"Price":47,
"PriceProvider":207,
"ETX":78
},
Made more generic for easier understanding, hoping it helps
Generic code
dictionary = {"65":{ "UserID":"vcjazfan", "Password":"ejujwlhk", "ETX":192} , "72":{ "InstrumentIndex":171, "PriceID":217, } }
#This is what i was thinking to write but it doesn't work how i want because it creates a packet just with one key/value every time
for key in dictionary:
for name in dictionary[key]:
value=dictionary[key][name]
packets = method(key, name=value) # in first iteration when key is 65 , name = "UserID" , value = "vcjazfan"
# in second iteration when key is 65 , name = "Password" , value = "ejujwlhk"
#Instead of writing this manually like that :
packets1 = method("65", UserID = "name", Password = "pass123", ETX = 123)
packets2 = method("72", InstrumentIndex = 123, PriceID = 12,)

This question solved my problem : How to pass dictionary items as function arguments in python?
solution to my original code:
Allpackets= []
for key in packetData:
Allpackets.append(packets/createsPacket(key, **packetData[key]))
Solution to generic one:
dictionary = {"65":{ "UserID":"vcjazfan", "Password":"ejujwlhk", "ETX":192} , "72":{ "InstrumentIndex":171, "PriceID":217, } }
Allpackets = []
for key in dictionary:
Allpackets.append( method(key, **dictionary))
#Instead of writing this manually like that :
packets1 = method("65", UserID = "name", Password = "pass123", ETX = 123)
packets2 = method("72", InstrumentIndex = 123, PriceID = 12,)

AWS Boto3 - How to use multiple filters and iterate over Tag names/values?

Currently, I'm making two calls to AWS ec2 using boto3 to fetch subnetIDs that start with tag name org-production-* and org-non-production-*. How can I combine these two functions in python and still be able to access the SubnetID's all_prod_subnets and all_non_prod_subnets ? I want to perhaps avoid code duplicatin make just one call to aws ec2, get all subnets and iterate them and filter the response based on tag name.
def get_all_production_subnets_from_accounts():
subnet = vpc_client.describe_subnets(
Filters=[{'Name': 'tag:Name', 'Values': ['org-production-*']}])['Subnets']
if len(subnet) > 0:
# print([s['SubnetId'] for s in subnet])
all_prod_subnets = [ s['SubnetId'] for s in subnet ]
print("[DEBUG]Queried Subnet ID's of Production are: {}".format(all_prod_subnets))
return all_prod_subnets
else:
return None
def get_all_nonproduction_subnets_from_acccounts():
subnet = vpc_client.describe_subnets(
Filters=[{'Name': 'tag:Name', 'Values': ['org-non-production-*']}])['Subnets']
if len(subnet) > 0:
# print([s['SubnetId'] for s in subnet])
all_non_prod_subnets = [ s['SubnetId'] for s in subnet ]
print("[DEBUG]Queried Subnet ID's of Non-Production are: {}".format(all_non_prod_subnets))
return all_non_prod_subnets
else:
return None

One way would be as follows:
def get_all_subnets_from_connectivity():
subnets_found = {}
# define subnet types of interest
subnets_found['org-production'] = []
subnets_found['org-non-production'] = []
results = vpc_client.describe_subnets()
for subnet in results['Subnets']:
if 'Tags' not in subnet:
continue
for tag in subnet['Tags']:
if tag['Key'] != 'Name': continue
for subnet_type in subnets_found:
if subnet_type in tag['Value']:
subnets_found[subnet_type].append(subnet['SubnetId'])
return subnets_found
all_subnets = get_all_subnets_from_connectivity()
print(all_subnets)
The example output:
{
'org-production': ['subnet-033bad31433b55e72', 'subnet-019879db91313d56a'],
'org-non-production': ['subnet-06e3bc20a73b55283']
}

How to print EC2 Tag name along with IP address?

I have a code which prints the Public IP's for the running instances,
regions = ['us-east-1','us-west-1','us-west-2','eu-west-1','sa-east-1','ap-southeast-1','ap-southeast-2','ap-northeast-1']
for region in regions:
client = boto3.client('ec2',aws_access_key_id=ACCESS_KEY,aws_secret_access_key=SECRET_KEY,region_name=region,)
addresses_dict = client.describe_addresses()
for eip_dict in addresses_dict['Addresses']:
if 'PrivateIpAddress' in eip_dict:
print eip_dict['PublicIp']
This is fine, now i also want to print the tag name and store it in another dict, i know i this can be done by :
regions = ['us-east-1','us-west-1','us-west-2','eu-west-1','sa-east-1','ap-southeast-1','ap-southeast-2','ap-northeast-1']
for region in regions:
client = boto3.client('ec2',aws_access_key_id=ACCESS_KEY,aws_secret_access_key=SECRET_KEY,region_name=region,)
dex_dict = client.describe_tags()
for dexy_dict in dex_dict['Tags']:
print dexy_dict['Value']
The problem is how do i combine it in one function and use 2 dict : one to store IP's and another to store the tag-name ? Please HELP

Try the following code, it will give you a dictionary where the key is the InstanceId and the value is a list of [PublicIP, Name].
import boto3
def instance_info():
instance_information = {}
ip_dict = {}
client = boto3.client('ec2')
addresses_dict = client.describe_addresses().get('Addresses')
for address in addresses_dict:
if address.get('InstanceId'):
instance_information[address['InstanceId']] = [address.get('PublicIp')]
dex_dict = client.describe_tags().get('Tags')
for dex in dex_dict:
if instance_information.get(dex['ResourceId']):
instance_information[dex['ResourceId']].append(dex.get('Value'))
for instance in instance_information:
if len(instance_information[instance]) == 2:
ip_dict[instance_information[instance][0]] = instance_information[instance][1]
else:
ip_dict[instance_information[instance][0]] = ''
return instance_information, ip_dict