I'm trying to create a dict that contains a list of users and their ssh-keys.
The list of users and the ssh-keys are stored in different yaml files which need to grab the info from. The files are "admins" and "users" and they look like:
Admins file:
admins:
global:
- bob
- john
- jimmy
- hubert
SSH key file:
users:
bob:
fullname: Bob McBob
ssh-keys:
ssh-rsa "thisismysshkey"
john:
fullname: John McJohn
ssh-keys:
ssh-rsa "thisismysshkey"
So far i have this code:
import yaml
#open admins list as "f"
f = open("./admins.sls", 'r')
#creates "admins" list
admins = yaml.load(f)
#grab only needed names and make a list
admins = admins['admins']['global']
#convert back to dict with dummy values of 0
admin_dict = dict.fromkeys(admins, 0)
So at this point I have this dict:
print(admin_dict)
{'bob': 0, 'john': 0}
Now i want to loop through the list of names in "admins" and update the key (currently set to 0) with their ssh-key from the other file.
So i do:
f = open("./users.sls", 'r')
ssh_keys = yaml.load(f)
for i in admins:
admin_dict[k] = ssh_keys['users'][i]['ssh-keys']
but when running that for loop, only one value is getting updated.
Kinda stuck here, i'm way out of my python depth... am i on the right track here?
edit:
changed that last loop to be:
for i in admins:
for key, value in admin_dict.items():
admin_dict[key] = ssh_keys['users'][i]['ssh-keys']
and things look better. Is this valid?
With an admin.yaml file like:
admins:
global:
- bob
- john
- jimmy
- hubert
And a ssh_key.yaml like so:
users:
bob:
fullname: Bob McBob
ssh-keys:
ssh-rsa: "bob-rsa-key"
john:
fullname: John McJohn
ssh-keys:
ssh-rsa: "john-rsa-key"
jimmy:
fullname: Jimmy McGill
ssh-keys:
ssh-rsa: "jimmy-rsa-key"
ssh-ecdsa: "jimmy-ecdsa-key"
You could do something like this asssuming you want to know which type of ssh key each user has (if not just go index one level deeper for the specific name of the key type in the dictionary comprehension):
import yaml
import pprint
def main():
with open('admin.yaml', 'r') as f:
admins_dict = yaml.load(f, yaml.SafeLoader)
admins_list = admins_dict['admins']['global']
with open('ssh_keys.yaml', 'r') as f:
ssh_dict = yaml.load(f, yaml.SafeLoader)
users_dict = ssh_dict['users']
admins_with_keys_dict = {
admin: users_dict[admin]['ssh-keys'] if admin in users_dict else None
for admin in admins_list
}
pp = pprint.PrettyPrinter(indent=2)
pp.pprint(admins_with_keys_dict)
if __name__ == '__main__':
main()
Output:
{ 'bob': {'ssh-rsa': 'bob-rsa-key'},
'hubert': None,
'jimmy': {'ssh-ecdsa': 'jimmy-ecdsa-key', 'ssh-rsa': 'jimmy-rsa-key'},
'john': {'ssh-rsa': 'john-rsa-key'}}
Alternative Output if you only want the rsa keys:
{ 'bob': 'bob-rsa-key',
'hubert': None,
'jimmy': 'jimmy-rsa-key',
'john': 'john-rsa-key'}
Above output achieved making the following change to the dictionary comprehension:
admin: users_dict[admin]['ssh-keys']['ssh-rsa'] if admin in users_dict else None
^^^^^^^^^^^
Related
I'm able to load and parse a json file with Python by referring to list items by name. My users.json data file:
{
"joe": {
"secret": "abc123.321def"
},
"sarah": {
"secret": "led789.321plo"
},
"dave": {
"secret": "ghi532.765dlmn"
}
}
My code - to output the 'secret' value associated with a specific user (e.g. Dave):
import json
with open('users_sample.json') as f:
users = json.load(f)
# f.close()
print(users['dave']['secret'])
This outputs Dave's secret:
ghi532.765dlmn
That's easy enough when I can predict or know the user names, but how do I iterate through each user in the users.json file and output each user's 'secret' value?
Thanks in advance!
I would encapsulate the logic to print each user and their associated function into a helper function:
def print_users(users_dict: dict, header='Before'):
print(f'-- {header}')
for u in users_dict:
print(f' {u}: {users_dict[u].get("secret", "<empty>")}')
Then, upon loading the users object initially via json.load, you can then call the function like so:
print_users(users)
To replace the secret for each user, in this case to replace every occurrence of a dot . with a plus +, a simple approach could be to use a for loop to update the users object in place:
for name, user in users.items():
if 'secret' in user:
user['secret'] = user['secret'].replace('.', '+')
Then print the result after the replacements are carried out:
print_users(users, 'After')
Finally, we can write the result users object back out to a file:
with open('users_sample_UPDATED.json', 'w') as out_file:
json.dump(users, out_file)
The output of the above code, in this case would be:
-- Before
joe: abc123.321def
sarah: led789.321plo
dave: ghi532.765dlmn
-- After
joe: abc123+321def
sarah: led789+321plo
dave: ghi532+765dlmn
The full code:
import json
def main():
with open('users_sample.json') as f:
users = json.load(f)
print_users(users)
new_users = {name: {'secret': user['secret'].replace('.', '+')}
for name, user in users.items()}
print_users(new_users, 'After')
with open('users_sample_UPDATED.json', 'w') as out_file:
json.dump(new_users, out_file)
def print_users(users_dict: dict, header='Before'):
print(f'-- {header}')
for u in users_dict:
print(f' {u}: {users_dict[u].get("secret", "<empty>")}')
if __name__ == '__main__':
main()
Iterate the dictionary using a for loop
code that works:
import json
with open('users_sample.json') as f:
users = json.load(f)
for user in users:
print(f"user name: {user} secret: {users[user]['secret']}")
You have a nested dictionary - i.e., each value associated with a top-level key is also a dictionary. You can iterate over those dictionaries with the built-in values() function. This leads to:
print(*[e.get('secret') for e in users.values()], sep='\n')
I am getting this error
ValueError: dict contains fields not in fieldnames: 'bobama', 'mfisc', 'nhay', 'bgates', 'jdoe', 'sburry', 'mcuban'
with the following code:
class Authentication:
def __init__(self):
# instantiate an instance variable
self.user_dict = {}
def register_user(self, uname, passwd):
if uname in self.user_dict:
print("Username exists! Try a new one.")
return False
else:
self.user_dict[uname] = passwd
print("Registration successful" )
return True
def data_entry(auth):
# registering 3 users
auth.register_user('jdoe', '$234^%$') # Jane Doe
auth.register_user('sburry', '456##&^') # Sam Burry
auth.register_user('mfisc', '%6&#$##') # Mike Fischer
auth.register_user('nhay', 'ildfu45') # Nicky Hailey
auth.register_user('bobama', 'klj43509jafd') # Barack Obama
auth.register_user('bgates', '^&%kjsfd934##$') # Bill Gates
auth.register_user('mcuban', '9&4rl#nsf') # Mark Cuban
# Main program
auth = Authentication()
data_entry(auth)
dictt = auth.user_dict
import csv
class AuthenticationIOcsv(Authentication):
def write_info(self):
fname='userinfo.csv'
with open(fname,'w') as op_file:
field_names = ['Username', 'Password']
op_writer = csv.DictWriter(op_file, fieldnames=field_names)
op_writer.writeheader()
op_writer.writerow(dictt)
# Main Program
auth = AuthenticationIOcsv()
data_entry(auth)
# writing to file
auth.write_info()
The aim of this project is to work with inheritance and a child class to create a csv file containing columns with the headers 'Username' and 'Password' and then the dictionary keys and values within those columns respectively. I've toyed around with a couple of other solutions to writing the dictionary to csv which I can post but have resulted in either blank csv files with just the headers or other types of errors. For example:
op_writer = csv.DictWriter(op_file, fieldnames=field_names)
op_writer.writeheader()
for key, value in dictt.items():
op_writer.writerow([key,value])
I get this error:
AttributeError: 'list' object has no attribute 'keys'
I am fairly new to Python so any guidance would be greatly appreciated! I'm clearly struggling to understand how to access the dictionary items in such a way to export to excel. Thank you in advance.
This is a DictWriter, only receive dict data.
You can use this way to write data:
op_writer = csv.DictWriter(op_file, fieldnames=field_names)
op_writer.writeheader()
for username, pwd in dictt.items():
op_writer.writerow({"Username": username, "Password": pwd})
I have an input file that I am trying to build a data base from.
Each line looks like this:
Amy Shchumer, Trainwreck, I Feel Pretty, Snatched, Inside Amy Shchumer
Bill Hader,Inside Out, Trainwreck, Tropic Thunder
And so on.
The first string is an actor\actress, and then movies they played in.
The data isn't sorted and they are some trailing whitespaces.
I would like to create a dictionary that would look like this:
{'Trainwreck': {'Amy Shchumer', 'Bill Hader'}}
The key would be the movie, the values should be the actors in it, unified in a set data type.
def create_db():
my_dict = {}
raw_data = open('database.txt','r+')
for line in raw_data:
lst1 = line.split(",") //to split by the commas
len_row = len(lst1)
lst2 = list(lst1)
for j in range(1,len_row):
my_dict[lst2[j]] = set([lst2[0]])
print(my_dict)
It doesn't work... it doesn't solve the issue that when a key already exists then the actor should be unified in a set with the prev actor
Instead I end up with:
'Trainwreck': {'Amy Shchumer'}, 'Inside Out': {'Bill Hader'}
def create_db():
db = {}
with open("database.txt") as data:
for line in data.readlines():
person, *movies = line.split(",")
for m in movies:
m = m.strip()
db[m] = db.get(m, []) + [person]
return db
Output:
{'Trainwreck': ['Amy Shchumer', 'Bill Hader'],
'I Feel Pretty': ['Amy Shchumer'],
'Snatched': ['Amy Shchumer'],
'Inside Amy Shchumer': ['Amy Shchumer'],
'Inside Out': ['Bill Hader'],
'Tropic Thunder': ['Bill Hader']}
This will loop through the data and assign the first value of each line to person and the rest to movies (see here for an example of how * unpacks tuples). Then for all the movies, it uses .get to check if it’s in the database yet, returning the list if it is and an empty list if it isn’t. Then it adds the new actor to the list.
Another way to do this would be to use a defaultdict:
from collections import defaultdict
def create_db():
db = defaultdict(lambda: [])
with open("database.txt") as data:
for line in data.readlines():
person, *movies = line.split(",")
for m in movies:
db[m.strip()].append(person)
return db
which automatically assigns [] if the key does not exist.
Need to write a function that takes an open file as the only parameter and returns a dictionary that maps a string to a list of strings and integers.
each line in the text will have a username, first name, last name, age, gender and an e-mail address. The function will insert each person's information into a dictionary with their username as the key, and the value being a list of [last name, first name, e-mail, age, gender].
basically what im trying to do is open a text file that contains this:
ajones Alice Jones 44 F alice#alicejones.net
and return something like this:
{ajones: ['Jones', 'Alice', 'alice#alicejones.net', 44, 'F']}
so far i have done this, but is there any other easier way?
def create_dict(file_name):
'''(io.TextIOWrapper) -> dict of {str: [str, str, str, int, str]}
'''
newdict = {}
list2 = []
for line in file_name:
while line:
list1 = line.split() #for a key, create a list of values
if list2(0):
value += list1(1)
if list2(1):
value += list1(2)
if list2(2):
value += list1(3)
if list2(3):
value += list1(4)
newdict[list1(0)] = list2
for next_line in file_name:
list1 = line.split()
newdict[list1(0)] = list1
return newdict
def helper_func(fieldname):
'''(str) -> int
Returns the index of the field in the value list in the dictionary
>>> helper_func(age)
3
'''
if fieldname is "lastname":
return 0
elif fieldname is "firstname":
return 1
elif fieldname is "email":
return 2
elif fieldname is "age":
return 3
elif fieldname is "gender":
return 4
If you have Python 2.7+, you can use a dictionary comprehension:
{l[0]: l[1:] for l in (line.rstrip().split(' ') for line in f)}
for line in file_name:
lineData = line.split() #for a key, create a list of values
my_dict[lineData[0]] = lineData[1:]
is a little easier i think ... although Im not sure if thats doing what you want ...
Agree with the first answer, here is the slightly different version to match the spec:
file=open('filename', 'r')
{username: [lname, fname, email, int(age), sex] for username, fname, lname, age, sex, email in (line.rstrip().split(' ') for line in file)}
There are certainly easier ways to build your dictionary:
d={}
st='ajones Alice Jones 44 F alice#alicejones.net'
li=st.split()
d[li[0]]=li[1:]
print d
# {'ajones': ['Alice', 'Jones', '44', 'F', 'alice#alicejones.net']}
If you want to change the order of the fields, do so as you are storing them:
d={}
st='ajones Alice Jones 44 F alice#alicejones.net'
li=st.split()
li2=li[1:]
d[li[0]]=[li2[i] for i in (1,0,4,3,2)]
print d
# {'ajones': ['Jones', 'Alice', 'alice#alicejones.net', 'F', '44']}
Or, just use named tuples or a dictionary rather than a list for the data fields.
One you have that part right, you can use it with your file:
# untested...
def create_dict(file_name):
newdict = {}
with open(file_name) as fin:
for line in fin:
li=line.split()
li2=li[1:]
li2[2]=int(li[2])
newdict[li[0]]=[li2[i] for i in (1,0,4,3,2)]
return newdict
At most log process systems, log file is tab separated text files, the schema of the file is provided separately.
for example.
12 tom tom#baidu.com
3 jim jim#baidu.com
the schema is
id : uint64
name : string
email : string
In order to find record like this person.name == 'tom' ， The code is
for each_line in sys.stdin:
fields = each_line.strip().split('\t')
if feilds[1] == 'tom': # magic number
print each_line
There are a lot of magic numbers 1 2 3.
Are there some tools like google protocol buffer(It's for binary), So we can build the object from text directly?
Message Person {
uint64 id = 1;
string name = 2;
string email = 3;
}
so we than build person like this: person = lib.BuildFromText(line)
for each_line in sys.stdin:
person = lib.BuildFromText(each_line) # no magic number
if person.name == 'tom':
print each_line
import csv
Person = {
'id': int,
'name': str,
'email': str
}
persons = []
for row in csv.reader(open('CSV_FILE_NAME', 'r'), delimiter='\t'):
persons.append({item[0]: item[1](row[index]) for index, item in enumerate(Person.items())})
How does lib.BuildFromText() function suppose to know how to name fields? They are just values in the line you pass to it, right? Here is how to do it in Python:
import sys
from collections import namedtuple
Person = namedtuple('Person', 'id, name, email')
for each_line in sys.stdin:
person = Person._make(each_line.strip().split('\t'))
if person.name == 'tom':
print each_line