We Keep Coding

How to automate password entry for Azure?

I want to automatically login an Azure system to access the virtual machine. I run the following code:
process_1 = subprocess.call(key.SSH_KEY + ' | ' + key.PASSKEY, shell = True) # Login to virtual machine
and receive the following:
/bin/sh: key.PASSKEY: command not found
azureuser#xx.xx.1x.1xx's password:
It believes the key.PASSKEY is another command, when it is the input for the azureuser#xx.xx.1x.1xx's password: part. How do I make sure that the key.PASSKEY is entered as the password automatically when this subprocess command is run?

The answer was found using the following video: https://www.youtube.com/watch?v=8QfD8V_-7ok
I did:
ch = pexpect.spawn(key.SSH_KEY)
ch.logfile = sys.stdout.buffer
ch.expect("azureuser#xx.xx.1x.1xx's password:")
ch.sendline(key.PASSKEY)
ch.expect("azureuser#vm")
ch.sendline('ls')

How to automate the user creation in vcenter using python

I am pretty new to Pyvmomi and vsphere automation.
I have been trying to automate the user and group creation in vsphere but could not locate the method in Pyvmomi which could help me automate the process of user creation.
I already have a user created in vcenter (abc#xyz.local)
This user has administrative privileges
Now, I want to create a session with user abc#xyz.local and add new users in Vcenter 'users and groups'. Once the new users are created, I have to add these users to different groups.
All these has to be done via automation using python.
Is there a way to automate this?

Unfortunately, the SSO API is all private and unavailable through pyvmomi and the rest of the SDKs.

As #Kyle Ruddy says, it looks like pyvmomi does not support SSO APIs. However, the golang alternative (govmomi) does. Govmomi also has an a CLI called GOVC which provides a nice wrapper to perform the following (and other things!):
Creating groups
Adding users to groups
Creating users
You could look at GOVCs source code and try and figure out the SOAP calls, but I think that would be more trouble than its worth.
If you are open to the idea of launching a bash commands from python then you could do the following:
import subprocess
import os
# Handy function for GOVC and assume GOVC is on your $PATH
def govc_runner(command):
my_env = os.environ.copy()
# Admin user will need to perform the commmands
my_env["GOVC_USERNAME"] = "abc#xyz.local"
my_env["GOVC_PASSWORD"] = "<ABC_PASSWORD>"
my_env["GOVC_URL"] = "https://<VCENTER>"
my_env["GOVC_INSECURE"] = "true"
process = subprocess.Popen(command, env=my_env, shell=True, stdout=subprocess.PIPE, stderr=subprocess.PIPE)
output, error = process.communicate()
return output, error
# New group and user info
newUserUsername = "praseemol"
newUserPassword = "<PARASEEMOL_PASSWORD>"
newGroup = "prasGroup"
# Creating new group and user
govc_runner("govc sso.group.create " + newGroup)
govc_runner("govc sso.user.create -p '" + newUserPassword + "' '" + newUserUsername + "'")
govc_runner("govc sso.group.update -a " + newUserUsername + " " + newGroup)
# Check if it has worked
output, error = govc_runner("govc sso.user.id " + newUserUsername)
if newGroup in output:
print("Yay, it worked:\n" + output)
else:
print("Something went wrong :(")
Hope that helps!

You can automate shell(vcenter) execution by doing ssh through putty for creation of user in system domain of vcenter and mimic same using paramiko library of python.
Official docs to refer for system domain user creation:
https://docs.vmware.com/en/VMware-vSphere/6.0/com.vmware.vsphere.security.doc/GUID-4FBEA58E-9492-409B-B584-C18477F041D8.html
Commands to be executed on vcenter shell:
/usr/lib/vmware-vmafd/bin/dir-cli user create --account william --first-name william --last-name lam --user-password 'VMware1!'
Refer:https://williamlam.com/2015/05/vcenter-server-6-0-tidbits-part-9-creating-managing-sso-users-using-dir-cli.html
To connect to vcenter using paramiko:
How do you execute multiple commands in a single session in Paramiko? (Python)
Pick the answer by "This".
You can fetch the created user using powercli commands:
Get-VIAccount
While using this be sure to find your created user in system domain.
Get_VIAccount -Domain 'domain_name'
The default domain name is usually like: "vsphere.local"
You can also find your domain by using putty to vcenter, enter shell and write,
"sso-config.sh -get_identity_sources"
You will be able to read Sys_Domain: '......'
You can assign role to user using powercli:
Get-VIPermission
If You can automate local user creation, let me know:
https://docs.vmware.com/en/VMware-vSphere/6.7/com.vmware.vsphere.vcsa.doc/GUID-533AE852-A1F9-404E-8AC6-5D9FD65464E5.html

How do I automate data extract from Hive to Windows desktop?

I have to login to bastion Linux host then run kinit and beeline using pbrun, then sftp csv file to Windows.
Query sample:
"SELECT * FROM db.table WHERE id > 100"
Is there a Python script or tool to automate this?

You could put yours query to file, for example hive_script.sql
and run his from terminal
hive -f hive_script.sql

I want to post my findings.
The most difficult part was figuring out expect+pbrun.
Because there are 2 interactive questions I had to pause for a sec after first question.
My expect code:
#!/usr/bin/expect -f
set timeout 300
set usr [lindex $argv 0];
set pwd [lindex $argv 1];
set query_file [lindex $argv 2];
spawn -noecho pbrun $usr &
expect -re "Password:"
send "$pwd\r"
sleep 1
expect "Enter reason for this privilege access:"
send "test\r"
send "kinit -k -t /opt/Cloudera/keytabs/`whoami`.`hostname -s`.keytab `whoami`/`hostname -f`#YOUR_FQDN_NAME.NET;ssl=true\r"
send "beeline -u 'jdbc:hive2://bigdataplatform-your_dev.net:10000/;principal=hive/bigdataplatform-your_dev.net#YOUR_FQDN_NAME.NET;ssl=true' --outputformat=csv2 --verbose=false --fastConnect=true --silent=true -f $query_file;\r"
expect "*]$\ " {send "exit\r"}
expect eof
Query:
select * from gfocnnsg_work.pytest LIMIT 1000000;
The rest is Python and paramiko.
I create Transport object, execute expect script, and save standard output on Windows OS.
Data access path:
Windows desktop->
SSH->
Linux login->
pbrun service login->
kinit
beeline->
SQL->
save echo on Windows
Here's a Python script with details: hivehoney

Python Program rsync

I am struggling to make a program on Python (Ubuntu) "To get a file from a directly connected Linux machine without prompting for Password"
Right now I am using this command on python but wanna put password in advance so it will not prompt me for password.
import os
os.system("echo 'hello world'")
os.system("rsync -rav pi#192.168.2.34:python ~/")
IP Address of Other Linux Machine is: 192.168.2.34
Password is: raspberry
Hostname: pi

You can achieve this by exchanging private keys. This way you can get a file from a directly connected Linux machine without prompting for Password. Here are the steps to exchange private keys:
Execute command ssh-keygen on your Ubuntu terminal.
Keep on pressing enter until something like this shows up:
The key's randomart image is:
+--[ RSA 2048]----+
| . .*|
| . + +o|
| + * + .|
| o E * * |
| S + + o +|
| o o o |
| . . . |
| |
| |
+-----------------+
After that execute ssh-copy-id pi#192.168.2.34 and enter password i.e., raspberry, if that is the password for the other machine.
Now execute python script as normal and it wont prompt for password.
import os
os.system("echo 'hello world'")
os.system("rsync -rav pi#192.168.2.34:python ~/")

You can try the following using pexpect and subprocess, the pexpect should definitely work, subprocess I am not sure:
cmd = "rsync -rav pi#192.168.2.34:python ~/"
from pexpect import *
run(cmd,events={'(?i)password':'your_password\r'})
from subprocess import PIPE,Popen
cmd = "rsync -rav pi#192.168.2.34:python ~/"
proc = Popen(cmd.split(),stdin=PIPE)
proc.stdin.write('your_pass\r')
proc.stdin.flush()
If you don't have pexpect installed use pip install pexpect

If you are on a private network (it should be as addresses are 192.168..), and if you trust all IP addresses on that network (means that no unauthorized user can spool an IP), you can also use host based authentication.
Extract from man page for ssh (I assume you use it as the underlying protocol for rsync) :
Host-based authentication works as follows: If the machine the user logs in from is listed in /etc/hosts.equiv or /etc/shosts.equiv on the remote machine, and the user names are the same on both sides, or if the files ~/.rhosts or ~/.shosts exist in the user's home directory on the remote machine and contain a line containing the name of the client machine and the name of the user on that machine, the user is considered for login.
That is you put in pi home directory a file .shosts containing one line
name_or_ip_address_of_source_machine user_name_on_source_machine
if the file already exists, just add that line.
But ... you must understand that as for BHAT IRSHAD's solution, it implies that you are now allowed to pass any command on dest machine as user pi without password.

How to get the list of windows administrators from a remote machine using WMI client for Python?

I am writing a Python script to retrieve the list of administrators users from a remote Windows box using the WMI client (https://pypi.python.org/pypi/WMI/1.4.9). This is the code I am using:
import wmi
wql = r'SELECT * FROM Win32_GroupUser WHERE GroupComponent="Win32_Group.Domain=\"MY_MACHINE\",Name=\"Administrators\""'
host = '10.253.6.38'
domain = 'My_Domain'
user = 'My_User'
auth_user = '{0}\{1}'.format(domain, user)
password = 'Secret
wmic = wmi.WMI(host, user=auth_user, password=password)
admins_list = wmic.query(wql)
But when I try running the query I get an exception with this error:
com_error: (-2147217385, 'OLE error 0x80041017', None, None)
I cannot find anywhere what that error means. If I run the exact same script but I use another query like for example SELECT PartComponent FROM Win32_SystemUsers, it works fine. To make it more strange, if I run the same query that is giving me problems directly in the remote machine using the "WMI Tester" it works perfectly. I have spent more than 2 days trying to make this work with no luck, I've run out of ideas. If anyone knows what is going on I could use some help here. thanks.