In my Python code I have a UserDocument which looks like this ->
class UserDocument(DynamicDocument):
user_id = IntField()
email = StringField(unique=True)
password = BinaryField()
salt = BinaryField()
I would like to populate my database with a new user during the instantiation of my Docker container. Related to the document it's pretty easy, I just have create a shell script and copy it into docker-entrypoint-initdb.d.
Here is the script for the user creation ->
#!/bin/sh
# This is supposed to generate the BinaryField
SALT=$(xxd -u -l 64 -p /dev/urandom)
BINARY_SALT=$(echo -n $SALT | xxd -r -p)
mongo -- "inup" <<EOF
var rootUser = '$MONGO_INITDB_ROOT_USERNAME';
var rootPassword = '$MONGO_INITDB_ROOT_PASSWORD';
var inup = db.getSiblingDB('inup');
inup.auth(rootUser, rootPassword);
db.user_document.insert({
user_id: 1,
email: 'toto#gmail.com',
password: 'password123',
salt: $BINARY_SALT # This doesn't work
});
EOF
This works but I have a problem. How can I generate a BinaryField for the attribut salt ?
Have a nice days and thanks for the replies !
Binary data in MongoDB are created from Base64 string, e.g.
SALT=$(xxd -u -l 64 -p /dev/urandom)
BASE64_SALT=$(echo -n $SALT | base64)
db.user_document.insertOne({"salt" : BinData(0, "$BASE64_SALT")});
See Binary
Or you store the Base64 as string.
db.user_document.insertOne({"salt" : "$BASE64_SALT"});
Where I work, we have servers that are pre-configured for the use of the bash mail command to send attachments and messages. I'm working on a notification script that will monitor server activity and generate an email if it detects an issue. I'm using the subprocess.call function in order to send a bash command.
I am successful in sending messages, but in the body portion of the email, it is stringing each notification line together rather than putting each notification on a separate line. I have tried to append each line within the string with "\n" and "\r\n". I have to use double backslashes as python will interpret this as literal new lines when it sends the echo command. I also passed the command "shopt -s xpg_echo" before using the echo with pipe to mail using the double backspaces but this also had no effect. I also tried using echo without the "-e" option and this had no effect either.
The trick is that I need python to send the new line to bash and then somehow get bash to interpret this as a new line using echo piped through to mail. Here is a sample of the code:
import os
import shutil
import sys
import time
import re
import subprocess
import smtplib
serviceports["SCP Test"] = ["22"]
serviceports["Webtier"] = ["9282"]
bashCommand = "netstat -an | grep LISTEN | grep -v LISTENING"
netstat_results = subprocess.check_output(bashCommand, shell=True)
netstat_results = str(netstat_results)
#Iterate through all ports for each service and assign down ports to variable
for servicename, ports in serviceports.items():
for ind_port in ports:
ind_port_chk = ":" + ind_port
count = sum(1 for _ in re.finditer(r'\b%s\b' % re.escape(ind_port_chk), netstat_results))
if count == 0:
warning = servicename + " on port " + ind_port + " is currently down!"
report.append(warning)
for warning in report:
message = message + warning + "\\\n"
fromaddr=serveridsimp + "#xxxxx.com"
toaddr='email#xxxxx.com'
subject="Testing..."
body=message
cmd= cmd='echo -e '+body+' | mail -s '+subject+' -r '+fromaddr+' '+toaddr
send=subprocess.call(cmd,shell=True)
The code runs a netstat command and assigns it to a string. The code will then iterate through the specified ports and search for where that port doesn't exist in the netstat string (netstat_results). It then will create a list object (warning) containing all the ports not located in netstat_results and then append each line adding \n to a string called "message". It then sends an echo piped to the xmail command to generate an email to be sent containing all the ports not found. What happens currently is that I will get an email saying something like this:
SCP Test on port 22 is currently down!nOHS Webtier on port 9282 is currently down!n etc...
I want it to put each message on a new line like so:
SCP Test on port 22 is currently down!
Webtier on port 9282 is currently down!
I am trying to avoid writing the output to a file and then using bash to read it back into the mail command. Is this possible without having to create a file?
I was finally able to fix the issue by changing the command sent to bash and character being appended to the following:
message = message + warning + "\n"
cmd= cmd='echo -e '+'"'+body+'"'+'|awk \'{ print $0" " }\''+' | mail -s '+'"'+subject+'"'+' -r '+fromaddr+' '+toaddr
I'm using python 2.7 and I need a python script like following. The problem is waiting file download everytime for result if curl time condition is True. I want to get result without waiting file download even if the condition is True. Same curl command run without problem on shell. How should I make a correction or is there a different method?
import os, subprocess
URL = 'http://www.xxxxx/file.zip'
curl="curl -siz yesterday " + URL + " | grep HTTP/ | awk {'print $2'}"
check = subprocess.check_output(curl, shell=True)
print(check)
if "200" in check:
....
....
i defined a function that switch my proxy settings every now and then,
problem is that i want it to run in a loop without manual intervention. But when i execute the program in sudo it gets called the first time en runs smoothly, second time it asks me for my sudo password. Here is the bit of code:
def ProxySetting(Proxy):
print "ProxyStetting(Proxy)"
call("networksetup -setwebproxy 'Wi-Fi' %s" "on" % Proxy, shell = True)
call("networksetup -setsecurewebproxy 'Wi-Fi' %s" "on" % Proxy, shell = True)
call("networksetup -setftpproxy 'Wi-Fi' %s" "on" %Proxy , shell=True)
I could use threading but am sure there is a way of doing it that wont cause problems. How can i hard code my sudo password so that it runs at the beginning of the function?
Here you can execute a command sudo without interactive prompt asking you to type your password :
from subprocess import call
pwd='my password'
cmd='ls'
call('echo {} | sudo -S {}'.format(pwd, cmd), shell=True)
Another method of passing your password to a shell command through python that wouldn't involve it showing up in any command history or ps output is:
p = subprocess.Popen(['sudo', self.resubscribe_script], stdin=subprocess.PIPE)
p.communicate('{}\n'.format(self.sudo_password))
Note that using communicate will only allow one input to be given to stdin; there are other methods for getting a reusable input.
I'm trying to create a scheduled task using the Unix at command. I wanted to run a python script, but quickly realized that at is configured to use run whatever file I give it with sh. In an attempt to circumvent this, I created a file that contained the command python mypythonscript.py and passed that to at instead.
I have set the permissions on the python file to executable by everyone (chmod a+x), but when the at job runs, I am told python: can't open file 'mypythonscript.py': [Errno 13] Permission denied.
If I run source myshwrapperscript.sh, the shell script invokes the python script fine. Is there some obvious reason why I'm having permissions problems with at?
Edit: I got frustrated with the python script, so I went ahead and made a sh script version of the thing I wanted to run. I am now finding that the sh script returns to me saying rm: cannot remove <filename>: Permission denied (this was a temporary file I was creating to store intermediate data). Is there anyway I can authorize these operations with my own credentials, despite not having sudo access? All of this works perfectly when I run it myself, but everything seems to go to shit when I have at do it.
Start the script using python not the actual script name, ex : python path/to/script.py.
at tries to run everything as a sh script.
EDIT: The at command tries running everything as a list of shell commands. So you should start your script like this:
at now + 1 minute < python mypythonscript.py
In this case, the #! line at the beginning of the script is not necessary.
I have been working on task scheduling between servers and clients recently. I just abstracted out my scheduling code and put it up on Github. It was meant to schedule several simulations across multiple machines that have all simulations in their filesystems. The idea is that since each machine had a different processor, it would compute each simulation, scp the results back into the server and request the server for the next simulation. The server responds by scheduling a task on the client to run the next unrun simulation
Hope this will help you.
NOTE: Since I only abstracted and uploaded the files about 5 minutes ago, I haven't had the chance to test the abstractions. However, if you come across any bugs, please let me know and I'll debug then as soon as I can.
Github seems to be down now. So here are the files that you'll need:
On the server:
serverside
#!/bin/bash
projectDir=~/
minute=`atq | sort -t" " -k1 -nr | head -n1 | cut -d' ' -f4 | cut -d":" -f1,2`
curr=`date | cut -d' ' -f4 | cut -d':' -f1,2`
time=`python -c "import sys; hour,minute=map(int,max(sys.argv[1:]).split(':')); minute += 2; hour, minute = [(hour,minute), ((hour+1)%24,minute%60)][minute>=60]; print '%d:%02d'%(hour, minute)" "$minute" "$curr"`
cat <<EOF | at "$time"
python $projectDir/serverside.py $1
EOF
serverside.py
import sys
import time
import smtplib
import subprocess
import os
import itertools
IP = sys.argv[1].strip()
PROJECT_DIR = "" # relative path (relative to the home directory) to the root directory of the project, which contains all subdirs containing simulation files
USERS = { # keys are IPs of the clients, values are user names on those clients
}
HOMES = { # keys are the IPs of clients, values are the absolute paths to the home directories on these clients for the usernames on these clients identified in USERS
}
HOME = None # absolute path to the home directory on the server
SMTP_SERVER = ""
SMTP_PORT = None
FROM_ADDR = None # the email address from which notification emails will be sent
TO_ADDR = None # the email address to which notification emails will be sent
def get_next_simulation():
""" This function returns a list.
The list contains N>0 elements.
Each of the first N-1 elements are names of directories (not paths), which when joined together form a relative path (relative from PROJECT_DIR).
The Nth element is the name of the file - the simulation to be run.
Before the end user implements this function, it is assumed that N=3.
Once this function has been implemented, if N!=3, change the code in the lines annotated with "Change code for N in this line"
Also look for this annotation in clientside.py and clientsideexec """
pass
done = False
DIR1, DIR2, FILENAME = get_next_simulation() # Change code for N in this line
while not done:
try:
subprocess.check_call("""ssh %(user)s#%(host)s 'sh %(home)s/%(project)/clientside %(dir1)s %(dir2)s %(filename)s %(host)s' """ %{'user':USER, 'host':IP, 'home':HOME[IP], 'project':PRJECT_DIR, 'dir1':DIR1, 'dir2':DIR2, 'filename':FILENAME}, shell=True) # Change code for N in this line
done = True
os.remove("%(home)s/%(project)/%(dir1)s/%(dir2)s/%(filename)s" %{'home':HOME, 'project':PROJECT_DIR, 'dir1':DIR1, 'dir2':DIR2, 'filename':FILENAME}) # Change code for N in this line
sm = smtplib.SMTP(SMTP_SERVER, SMTP_PORT)
sm.sendmail(FROM_ADDR, TO_ADDR, "running %(project)s/%(dir1)s/%(dir2)s/%(filename)s on %(host)s" %{'project':PROJECT_DIR, 'dir1':DIR1, 'dir2':DIR2, 'filename':FILENAME, 'host':IP}) # Change code for N in this line
except:
pass
On the client:
clientside
#!/bin/bash
projectpath=~/
python $projectpath/clientside.py "$#"
clientside.py
import subprocess
import sys
import datetime
import os
DIR1, DIR2, FILENAME, IP = sys.argv[1:]
try:
subprocess.check_call("sh ~/cisdagp/clientsideexec %(dir1)s %(dir2)s %(filename)s %(ip)s" %{'dir1':, 'dir2':, 'filename':, ip':IP}, shell=True, executable='/bin/bash') # Change code for N in this line
except:
pass
clientsideexec
#!/bin/bash
projectpath=~/
user=''
serverIP=''
SMTP_SERVER=''
SMTP_PORT=''
FROM_ADDR=''
TO_ADDR=''
MESSAGE=''
cat <<EOF | at now + 2 minutes
cd $projectpath/$1/$2 # Change code for N in this line
sh $3
# copy the logfile back to the server
scp logfile$3 $user#$serverIP:$projectpath/$1/$2/
cd $projectpath
python -c "import smtplib; sm = smtplib.SMTP('$SMTP_SERVER', $SMTP_PORT); sm.sendmail('$FROM_ADDR', '$TO_ADDR', '$MESSAGE')"
python clientsiderequest.py
EOF
Could you try: echo 'python mypythonscript.py' | at ...