In my Python code I have a UserDocument which looks like this ->
class UserDocument(DynamicDocument):
user_id = IntField()
email = StringField(unique=True)
password = BinaryField()
salt = BinaryField()
I would like to populate my database with a new user during the instantiation of my Docker container. Related to the document it's pretty easy, I just have create a shell script and copy it into docker-entrypoint-initdb.d.
Here is the script for the user creation ->
#!/bin/sh
# This is supposed to generate the BinaryField
SALT=$(xxd -u -l 64 -p /dev/urandom)
BINARY_SALT=$(echo -n $SALT | xxd -r -p)
mongo -- "inup" <<EOF
var rootUser = '$MONGO_INITDB_ROOT_USERNAME';
var rootPassword = '$MONGO_INITDB_ROOT_PASSWORD';
var inup = db.getSiblingDB('inup');
inup.auth(rootUser, rootPassword);
db.user_document.insert({
user_id: 1,
email: 'toto#gmail.com',
password: 'password123',
salt: $BINARY_SALT # This doesn't work
});
EOF
This works but I have a problem. How can I generate a BinaryField for the attribut salt ?
Have a nice days and thanks for the replies !
Binary data in MongoDB are created from Base64 string, e.g.
SALT=$(xxd -u -l 64 -p /dev/urandom)
BASE64_SALT=$(echo -n $SALT | base64)
db.user_document.insertOne({"salt" : BinData(0, "$BASE64_SALT")});
See Binary
Or you store the Base64 as string.
db.user_document.insertOne({"salt" : "$BASE64_SALT"});
Related
I am reading a csv file containing list of employees(GRCLOGIN.csv) and retrieving employee ID to Query LDAP to retrieve their related data and save it to a text file(LDAP_USERS.txt)
from sys import exit
import subprocess, sys
import csv
with open('GRCLOGIN.csv', 'r') as file:
reader = csv.reader(file, quoting=csv.QUOTE_NONE, skipinitialspace=True)
reader = csv.reader(file)
output = open('LDAP_USERS.txt', 'a')
next(reader)
for row in reader:
val=row[0]
This is where I am getting issue, my objective is only to retrieve employee's firstName and Email, not all details/columns, but when I include firstname and email in the query below , empty text file is generated, but if I remove firstname and email then text file is generated with all employees details correctly but I don't want all details.
I feel issue is where $1 is not correctly being set to str(val) which is employee ID
subprocess.Popen(["./ldapsearch -B -1 -T -h localhost -p 1389 -D 'cn=directory manager' -j ../../bin/passwordfile.txt -b '(GRCLoginID=$1)' firstName email"+str(val)], stdout=output, stderr=output, shell=True)
exit()
In a linux shell try ./ldapsearch --help, it would very usefull, besides that, if you want to only get certain attributes, you must put the attributes in the end of the command, and to only get one user, either you search with fixed search base, if you know where the user is in the ldap, otherwise you can search it by applying a filter on user id so:
["./ldapsearch -B -1 -T -h localhost -p 1389 -D 'cn=directory manager' -j ../../bin/passwordfile.txt -b 'GRCLoginID=$1,ou=users,cn=root,cn=com' firstName email", val]
["./ldapsearch -B -1 -T -h localhost -p 1389 -D 'cn=directory manager' -j ../../bin/passwordfile.txt -b 'cn=com' firstName email -f (&(GRCLoginID=$1))", val]
Both options are valid
I would use the python ldap module for this. You will receive the results within python immediately without the CSV in between.
import ldap
ldap.set_option(ldap.OPT_X_TLS_REQUIRE_CERT, ldap.OPT_X_TLS_NEVER)
ldapurl = "ldap://server:port"
conn = ldap.initialize(ldapurl)
# connect to the server
conn.simple_bind_s(username, password)
# return these attributes
returntome = ('firstName', 'email')
# Use this search filter
ldapfilter = 'GRCLoginID='+val
# Start searching
results = conn.search_s( 'cn=com', ldap.SCOPE_SUBTREE, ldapfilter, returntome )
Now the results variable contains a list of LDAP objects that you can iterate over in Python. See the documentation at https://www.python-ldap.org/en/python-ldap-3.4.0/reference/ldap.html#ldap.LDAPObject.search
I'm trying to edit the following YAML file
db:
host: 'x.x.x.x.x'
main:
password: 'password_main'
admin:
password: 'password_admin'
To edit the host part, I got it working with
sed -i "/^\([[:space:]]*host: \).*/s//\1'$DNS_ENDPOINT'/" config.yml
But I can't find a way to update the password for main and admin (which are different values).
I tried to play around with \n and [[:space:]] and got different flavours of:
sed -i "/^\([[:space:]]*main:\n*[[:space:]]*password: \).*/s//\1'$DNS_ENDPOINT'/" config.yml
But never got it to work.
Any help greatly appreciated!
Edit - Requirement: no external binaries/tools. Just good ol' bash.
Since you don't want to install yq you could use python that you most probably already have installed.
Here are the fundamentals:
#!/usr/bin/python
import yaml
with open("config.yml") as f:
y = yaml.safe_load(f)
y['db']['admin']['password'] = 'new_admin_pass'
print(yaml.dump(y, default_flow_style=False, sort_keys=False))
Output:
db:
host: x.x.x.x.x
main:
password: password_main
admin:
password: new_admin_pass
A similar piece of python code as a one-liner that you can put in a bash script would look something like this (and produce the same output):
python -c 'import yaml;f=open("config.yml");y=yaml.safe_load(f);y["db"]["admin"]["password"] = "new_admin_pass"; print(yaml.dump(y, default_flow_style=False, sort_keys=False))'
If you'd like to save the output to a file, you can provide an output stream as the second argument to dump():
#!/usr/bin/python
import yaml
with open("config.yml") as istream:
ymldoc = yaml.safe_load(istream)
ymldoc['db']['admin']['password'] = 'new_admin_pass'
with open("modified.yml", "w") as ostream:
yaml.dump(ymldoc, ostream, default_flow_style=False, sort_keys=False)
If you'd like to overwrite the original file, I recommend writing to a temporary file first and only if that succeeds, use os.rename to move that file in place of the original one. That's to minimize the risk of creating a corrupt config.yml in case of problems.
Note: Using a YAML parser like yq (or yq) will be a way more reliable solution.
However, I've used the following 'technique' to alter a 'pre-defined' line though the help of grep and sed like so;
/tmp/config.yml
db:
host: 'x.x.x.x.x'
main:
password: 'password_main'
admin:
password: 'password_admin'
Get the line number where your 'old-password' is located:
grep -n 'password_admin' /tmp/config.yml | cut -d ':' -f1
6
Then, use sed to override that line with your new password:
sed -i '6s/.*/ password: \'new_admin_pass\'/' /tmp/config.yml
The new file now looks like this:
db:
host: 'x.x.x.x.x'
main:
password: 'password_main'
admin:
password: 'new_admin_pass'
Note
Keep in mind that any special chars (&, \, /) in the password will cause sed to misbehave!
This could fail if the indent changes, since YAML cares about indentation. Just like I mentioned above, using a YAML parser will be a much more reliable solution!
$ awk -v new="'sumthin'" 'prev=="main:"{sub(/\047.*/,""); $0=$0 new} {prev=$1} 1' file
db:
host: 'x.x.x.x.x'
main:
password: 'sumthin'
admin:
password: 'password_admin'
or if your new text can contain escape sequences that you don't want expanded (e.g. \t or \n), as seems likely when setting a password, then:
new="'sumthin'" awk 'prev=="main:"{sub(/\047.*/,""); $0=$0 ENVIRON["new"]} {prev=$1} 1' file
See How do I use shell variables in an awk script? for why/how I use ENVIRON[] to access a shell variable rather than setting an awk variable in that second script.
This is by no way as reliable as yq but you can use this awk if your yaml file structure is same as how it is shown in question:
pw='new_&pass'
awk -v pw="${pw//&/\\\\&}" '/^[[:blank:]]*main:/ {
print
if (getline > 0 && $1 == "password:")
sub(/\047[^\047]*\047/, "\047" pw "\047")
} 1' file
db:
host: 'x.x.x.x.x'
main:
password: 'new_&pass'
admin:
password: 'password_admin'
As mentioned by experts in other answers too, yq should be the proper way but in case someone doesn't have it then one could try following.
awk -v s1="'" -v new_pass="new_value_here" '
/main:/{
main_found=1
print
next
}
main_found && /password/{
next
}
/admin:/ && main_found{
print " password: " s1 new_pass s1 ORS $0
main_found=""
next
}
1
' Input_file
NOTE: In case you want to save output into Input_file itself then append > temp && mv temp Input_file to above solution.
I'm trying to add a user to the mongodb, I have a shell script, but I'm trying to do this using python.
I have the code :
addadmusr="""{ createUser 'admin', pwd 'lint1234', roles ['userAdminAnyDatabase'] }"""
call('/opt/mongodb/bin/mongo -port primport --eval """db.getSiblingDB("""admin""").runCommand(addadmusr)"""',shell = True)
I'm trying to use subprocess calls to get this done, and I have a primport extracted by using the code:
primport = subprocess.check_output('/opt/mongodb/bin/mongo localhost27017 --eval "printjson(rs.isMaster())" | grep "primary"| cut -f1 -d, | cut -f3 -d : | grep -Eo """[0-9]{1,}"""' , shell = True)
when I'm trying to add the user, it says invalid port number "primport" and returns connection failed. My guess is the primport value that Im extracting is not parsing into the call statement that I'm using. I need to use the value extracted (primport) and add the user -admin to the db. Any python alternative also helps.
What is about pymongo for example?
You can try to use something like that:
from pymongo import MongoClient
client = MongoClient('localhost:27017')
client.testdb.add_user('newUserName', 'TestPassword', roles=[{'role':'readWrite','db':'testdb'}])
I'm working on an application that will eventually graph the gpg signature connections between a predefined set of email addresses. I need it to programmatically collect the public keys from a key server. I have a working model that will use the --search-keys option to gpg. However, when run with the --batch flag, I get the error "gpg: Sorry, we are in batchmode - can't get input". When I run with out the --batch flag, gpg expects input.
I'm hoping there is some flag to gpg that I've missed. Alternatively, a library (preferably python) that will interact with a key server would do.
Use
gpg --batch --keyserver hkp://pool.sks-keyservers.net --search-keys ...
and parse the output to get key IDs.
After that
gpg --batch --keyserver hkp://pool.sks-keyservers.net --recv-keys key-id key-id ..
should work
GnuPG is not performing very well anyway when you import very large portions of the web of trust, especially during the import phase.
I'd go for setting up a local keyserver, just dumping all the keys in there (less than 10GB of download size in 2014) and directly querying your own, local keyserver.
Hockeypuck is rather easy to setup and especially query, as it stores the data in a PostgreSQL database.
Use --recv-keys to get the keys without prompting.
In the case of a hkps server the following would work :
gpg --keyserver hkps://***HKPSDOMAIN*** --recv-keys \
$(curl -s "https://***HKPSDOMAIN***/?op=index&options=mr&search=***SEARCHSTRING***"\
|grep pub|awk -F ":" '{print $2}')
We can store the std and err output of the gpg --search-keys commands into variables by specifying 2>&1, then do work on those variables. For example, get the public key ids or those with *.amazon.com email addresses:
pubkeyids=$(gpg --batch --keyserver hkp://keyserver.ubuntu.com --search-keys amazon.com 2>&1 | grep -Po '\d+\s*bit\s*\S+\s*key\s*[^,]+' | cut -d' ' -f5)
The regular expression is fully explained on regex101.com. We can automate searching for keys by their IDs and add them to the keyring using bash by parsing that output. As an illustration, I created the following GitHub gist to host the code below.
Example address list example.csv:
First Name
Last Name
Email Address
Hi
Bye
hi#bye.com
Yes
No
yes#no.com
Why
Not
why#not.com
Then we can pass the csv path to a bash script which will add all keys belonging to the email addresses in the csv:
$ getPubKeysFromCSV.sh ~/example.csv
Here is an implementation of the above idea, getPubKeysFromCSV.sh:
# CSV of email address
csv=$1
# Get headers from CSV
headers=$(head -1 $csv)
# Find the column number of the email address
emailCol=$(echo $headers | tr ',' '\n' | grep -n "Email Address" | cut -d':' -f1)
# Content of the CSV at emailCol column, skipping the first line
emailAddrs=$(tail -n +2 $csv | cut -d',' -f$emailCol)
gpgListPatrn='(?<entropy>\d+)\s*bit\s*(?<algo>\S+)\s*key\s*(?<pubkeyid>[^,]+)'
# Loop through the array and get the public keys
for email in "${emailAddrs[#]}"
do
# Get the public key ids for the email address by matching the regex gpgListPatt
pubkeyids=$(gpg --batch --keyserver hkp://keyserver.ubuntu.com --search-keys $email 2>&1 | grep -Po $gpgListPatrn | cut -d' ' -f5)
# For each public key id, get the public key
for pubkeyid in $pubkeyids
do
# Add the public key to the local keyring
recvr=$(gpg --keyserver hkp://keyserver.ubuntu.com --recv-keys $pubkeyids 2>&1)
# Check exit code to see if the key was added
if [ $? -eq 0 ]; then
# If the public key is added, do some extra work with it
# [do stuff]
fi
done
done
If we wanted, we could make getPubKeysFromCSV.sh more complex by verifying a file signature in the body of the loop, after successfully adding the public key. In addition to the CSV path, we will pass the signature path and file path as arguments two and three respectively:
$ getPubKeysFromCSV.sh ~/example.csv ./example.file.sig ./example.file
Here is the updated script difference as a diff:
--- original.sh
+++ updated.sh
## -1,6 +1,12 ##
# CSV of email address
csv=$1
+# signature file
+sig=$2
+
+# file to verify
+file=$3
+
# Get headers from CSV
headers=$(head -1 $csv)
## -22,5 +28,17 ##
recvr=$(gpg --keyserver hkp://keyserver.ubuntu.com --recv-keys $pubkeyids 2>&1)
# Check exit code to see if the key was added
+ if [ $? -eq 0 ]; then
+ verify=$(gpg --batch --verify $sig $file 2>&1)
+ # If the signature is verified, announce it was verified
+ # else, print error not verified and exit
+ if [[ $verify =~ "^gpg: Good signature from" ]]; then
+ echo "$file was verified by $email using $pubkeyid"
+ else
+ printf '%s\n' "$file was unable to be verified" >&2
+ exit 1
+ fi
+ fi
done
done
I would like a way to update my password on a remote Ubuntu 10.4 box with fabric.
I would expect my fabfile.py would look something like this:
def update_password(old_pw, new_pw):
# Connects over ssh with a public key authentication
run("some_passwd_cmd --old %s --new %s" % (old_pw, new_pd))
Unfortunately the only command I know of that lets one change the password is passwd, and on Ubuntu 10.4 there doesn't seem to be any way to pass in the new (or old) password as an argument to passwd.
What command could one use to change a user's password on Ubuntu 10.4 via fabric?
EDIT:
I've had a look at usermod -p, and that may work but it isn't recommended by the man page.
EDIT: For some reason usermod -p wasn't working either over fabric.
As well, I've tried a (somewhat insecure) variation on mikej's answer that did solve the problem:
# connecting & running as root.
from fabric.api import *
from fabric.contrib import files
files.append("%s\n%s" % (passwd, passwd), '.pw.tmp')
# .pw.tmp:
# PASSWD
# PASSWD
run("passwd %s < .pw.tmp" % user)
run("rm .pw.tmp")
It's not a very elegant solution, but it works.
Thank you for reading.
Brian
You could feed the new and old passwords into passwd using echo e.g.
echo -e "oldpass\\nnewpass\\nnewpass" | passwd
(the -e option for echo enables interpretation of backslash escapes so the newlines are interpreted as such)
The trick is to use a combination of usermod and Python’s crypt to change your password:
from crypt import crypt
from getpass import getpass
from fabric.api import *
def change_password(user):
password = getpass('Enter a new password for user %s:' % user)
crypted_password = crypt(password, 'salt')
sudo('usermod --password %s %s' % (crypted_password, user), pty=False)
I use chpasswd on Ubuntu 11.04
fabric.api.sudo('echo %s:%s | chpasswd' % (user, pass))
Note:
Normally this pattern doesn't work:
$ sudo echo bla | restricted_command
because only the 'echo' gets elevated privileges, not the 'restricted_command'.
However, here it works because when fabric.api.sudo is caled
with shell=True (the default), fabric assembles the command like this:
$ sudo -S -p <sudo_prompt> /bin/bash -l -c "<command>"
sudo spawns a new shell (/bin/bash), running with root privileges, and
then that escalated shell runs the command.
Another way to pipe with sudo is to use sudo tee:
Out of interest, I have to do a similar task on a collection of Solaris boxes (add a whole lot of users, set their password). Solaris usermod doesn't have a --password option, so in the past I've used Expect to do this, but writing Expect scripts can be painful.
So this time I'm going to use Python's crypt.crypt, edit /etc/shadow directly (with backups, of course). http://docs.python.org/release/2.6.1/library/crypt.html
Commenters have suggested using various echo incantations piped to passwd. AFAIK this will never work, as passwd is programmed to ignore input from stdin and only accept input from an interactive tty. See http://en.wikipedia.org/wiki/Expect
I had no luck with the other methods. Thought I would share my method that I used for a once-off throwaway script.
It uses auto-responder to type in passwords at the prompts. I then immediately expire all the passwords so that users have a chance to choose their own.
This is not the most secure method, but depending on your use case it may be useful.
from collections import namedtuple
from getpass import getpass
import hashlib
from invoke import Responder
import uuid
from fabric import Connection, Config
User = namedtuple('UserRecord', ('name', 'password'))
def set_passwords(conn, user):
print(f'Setting password for user, {user.name}')
responder = Responder(
pattern=r'(?:Enter|Retype) new UNIX password:',
response=f'{user.password}\n',
)
result = conn.sudo(f'passwd {user.name}', warn=True, hide='both',
user='root', pty=True, watchers = [responder])
if result.exited is not 0:
print(f'Error, could not set password for user, "{user.name}". command: '
f'{result.command}; exit code: {result.exited}; stderr: '
f'{result.stderr}')
else:
print(f'Successfully set password for {user.name}')
def expire_passwords(conn, user):
print(f'Expiring password for user, {user.name}')
cmd = f'passwd --expire {user.name}'
result = conn.sudo(cmd, warn=True, user='root')
if result.exited is not 0:
print(f'Error, could not expire password for user, "{user.name}". '
f'command: {result.command}; exit code: {result.exited}; stderr: '
f'{result.stderr}')
else:
print(f'Successfully expired password for {user.name}')
def gen_password(seed_string):
# Don't roll your own crypto. This is for demonstration only and it is
# expected to only create a temporary password that requires changing upon
# initial login. I am no cryptography expert, hence this alternative
# simplified answer to the one that uses crypt, salt, etc -
# https://stackoverflow.com/a/5137688/1782641.
seed_str_enc = seed_string.encode(encoding='UTF-8')
uuid_obj = uuid.UUID(int=int(hashlib.md5(seed_str_enc).hexdigest(), 16))
return str(uuid_obj)[:8]
def some_function_that_returns_something_secret(conn):
return f'dummy-seed-{conn}'
sudo_pass = getpass('Enter your sudo password:')
config = Config(overrides={'sudo': {'password': sudo_pass}})
with Connection('vm', config=config) as vm_conn:
print(f'Making a new connection to {vm_conn.host}.')
# I usually use the sudo connection here to run a command that returns a
# reproducible string that only the sudo user could get access to be used
# for user_record.password bellow. Proceed with caution, this is not a
# recommended approach
seed = some_function_that_returns_something_secret(vm_conn)
user_record = User(name='linux_user', password=gen_password(seed))
set_passwords(vm_conn, user_record)
expire_passwords(vm_conn, user_record)
print(f'Done! Disconnecting from {vm_conn.host}.')
# So that you know the temporary password, print user_record or save to file
# `ssh linux_user#vm` and it should insist that you change password
print(user_record)