Accessing HDFS jceks in python script - python

I want to access HDFS jceks (password alias is created) in python script for secure login. Can anyone help with with python code /steps to do so.
Thank you

You may use the hadoop configuration to access the password from python code.
Only thing you would need is to instanciate a hadoop configuration and use the property hadoop.security.credential.provider.path to set and retrieve the password using getPassword
You could use Spark also to read the jecks password from inside your code:
Python:
spark1 = SparkSession.builder.appName("xyz").master("yarn").enableHiveSupport().config("hive.exec.dynamic.partition", "true").config("hive.exec.dynamic.partition.mode", "nonstrict").getOrCreate()
x = spark1.sparkContext._jsc.hadoopConfiguration()
x.set("hadoop.security.credential.provider.path", "jceks://file///localpathtopassword")
a = x.getPassword("<password alias>")
passw = ""
for i in range(a.__len__()):
passw = passw + str(a.__getitem__(i))
In the above code you shall get the password string in passw

Related

Getting and passing an encrypted password between Flask, Ansible, then back to Python using Fernet

I am currently creating a web application with Flask.
My Flask Python gets a password from user input through a webform, then that password is pushed to Ansible using the ansible_runner module.
form.py
from cryptography.fernet import Fernet
from tools import load_key
from ansiblerunner import ansible, inventory_hosts
ansible_role = "role"
tags = "install"
username = request.form.get("username")
dcryptpwd = request.form.get("password")
# Encrypt encpwd to pass to Ansible
refKey = load_key()
key = Fernet(refKey)
pwdbyt = bytes(dcryptpwd, 'utf-8')
encpwdbyt = (key.encrypt(pwdbyt))
encpwd = encpwdbyt.decode('utf-8')
inventory = inventory_hosts(username)
runner = ansible(hostname, ansible_role, encpwd, tags)
return render_template("form/form.html")
tools.py
def load_key():
with open('refKey.txt') as f:
refKey = ''.join(f.readlines())
refKeybyt = bytes(refKey, 'utf-8')
f.close()
return refKeybyt
Ansible is then using it on my role in main.yml
- block:
- name: Run setup.py
command: "{{ setup_py_run }} --username {{ username }} --encpwd {{ encpwd }}
no_log: false
delegate_to: localhost
tags:
- rebuild
Then setup.py decrypts the password and uses it here:
# Arguments imported from ansible
username = args.username
encpwd = bytes(args.encpwd,'utf-8')
# Decrypt password
refKey = load_key()
key = Fernet(refKey)
encpwdbyt = key.decrypt(encpwd)
password = encpwdbyt.decode('utf-8')
Currently this solution works, in the Ansible logs the plaintext password no longer shows up and the password gets decrypted successfully.
A few questions here:
As this stands the private key is hidden in my server in a random directory, and the server is protected with SSH keys. Is this a secure way of 'hiding' a private key?
Instead of having one key stored that encrypts and decrypts, would it be more secure to generate a new key each time, overwriting the previous key used?
Is encryption even necessary or am I overthinking this? Should obfuscation be used instead?
I am new to programming and even newer to program security, any insight would be greatly appreciated, thank you!

pass password to a Python script on prompt

How can I pass a password to my python script when it prompts for a password. I do not want the user to enter the password when it prompts. It should be passed directly from the script.
subprocess.run(['ansible-vault', 'decrypt', 'main.yml', 'linux.yml','--ask-vault-pass'])
While running the script it prompts for the password. I want the password to be passed from script only not by the user or passing a password file.
Is there a way to pass the password here? I have other ansible vault option like vault-password etc but that doesn't serve my purpose.
Instead of using the cmdline ansible-vault, you can use the Python package - ansible-vault which will allow you to hardcode the password inside the script itself:
from ansible_vault import Vault
vault = Vault('password')
main_data = vault.load(open('main.yml').read())
linux_data = vault.load(open('linux.yaml').read())
As you are hardcoding the password inside the code, ensure you don't commit this code anywhere or send it to anyone, it's a serious security risk.

Make an AD/LDAP query without storing username / password as cleartext

Hi I'm creating Cloudera environment, and want to share this to other teams. This script should be querying our Active directory server. But unfortunately I don't want to put my password / username inside the script.
from ldap3 import Server, Connection, ALL, NTLM, ALL_ATTRIBUTES, ALL_OPERATIONAL_ATTRIBUTES, AUTO_BIND_NO_TLS, SUBTREE
serverName = 'internal.imsglobal.com'
server = Server(serverName)
conn = Connection(server, read_only=True, user='',password='', auto_bind=True)
person='somebody'
conn.search('dc=internal,dc=mydomain,dc=com', '(&(givenName=*)(sAMAccountName=*{person}*))'.format(person=person), attributes=[ALL_ATTRIBUTES, ALL_OPERATIONAL_ATTRIBUTES])
no_entries = len(conn.entries)
What can be done to avoid hardcoding credentials ?
Unfortunately Cloudera doesn't allow to use input function or anything interactive.
You can use dotenv
pip install python-dotenv
Create a .env file that consists of key/values like this:
USERNAME=foo
PASSWORD=bar
Then load the .env file and access the values.
dotenv_path = join(dirname(__file__), $location_of_.env_file)
load_dotenv(dotenv_path)
username = os.getenv("USERNAME")
password = os.getenv("PASSWORD")

Using username and password safely in Python code

I'm using the Python requests library to make a call to an API that requires Windows Authentication. In C# I have always used the Directory Services, which has allowed me to avoid putting passwords in any of my code or configurations. From what I have found online, it seems that my only option in Python is to have a password somewhere. I have a service account that I will use, but I need to store the password securely. What is the best way to securely store and retrieve a service account password in Python without hard coding plain text?
The code that I am currently using is below. I have the username and password stored in plain text in my configuration:
auth = HttpNtlmAuth(
config.ServiceAccount["Username"],
config.ServiceAccount["Password"]
)
content = requests.post(call_string, json=parameters, auth=auth)
Edit: I should mention that this will not be a user-facing application. It will run as a batch job. So there will not be any way for a user to enter the username/password while running the application.
You could just not store the password at all and require the user to provide the password at runtime
import getpass
user = getpass.getuser()
password = getpass.getpass()
Otherwise, you could do something similar to git and just have the user store their password in plaintext in a config file in their home directory that you then read at runtime.
I know I asked this question a while ago, but I found a better solution to the NTLM/Windows authentication. I used the requests_negotiate_sspi library to avoid any passwords:
from requests_negotiate_sspi import HttpNegotiateAuth
auth = HttpNegotiateAuth()
content = requests.post(call_string, json=parameters, auth=auth)

Python VCenter API connection string

I am writing a python script to automate deletion of svmi netapp snapshots using PySphere library.
to create a connection instance i do :
vc_server = pysphere.VIServer()
vc_server.connect("vc.example.com", "example.com\dd432", "password")
Everything works the expected way, the only question I have is there a way for me to hide the password in the script, maybe some hashing mechanism, to have that password argument do not specify the password in clear text in the script?
Thanks.
For now i found only way to put password under file system into .cf file and restrict permissions for read on that file. Sure there should be some better way.
#classmethod
def setUpClass(cls):
config_path = os.path.join(os.path.dirname(__file__), "config_conn.cf")
cls.config = ConfigParser.ConfigParser()
cls.config.read(config_path)
host = cls.config.get("READ_ONLY_ENV", "host")
user = cls.config.get("READ_ONLY_ENV", "user")
pswd = cls.config.get("READ_ONLY_ENV", "password")
cls.server = VIServer()
cls.server.connect(host, user, pswd)

Categories