We Keep Coding

Retrieve Azure IoT connection string using Python

I have the following code
conn_str = "HostName=my_host.azure-devices.net;DeviceId=MY_DEVICE;SharedAccessKey=MY_KEY"
device_conn = IoTHubDeviceClient.create_from_connection_string(conn_str)
await device_conn.connect()
This works fine, but only because I've manually retrieved this from the IoT hub and pasted it into the code. We are going to have hundreds of these devices, so is there a way to retrieve this connection string programmatically?
It'll be the equivalent of the following
az iot hub device-identity connection-string show --device-id MY_DEVICCE --hub-name MY_HUB --subscription ABCD1234
How do I do this?

The device id and key are you give to the each device and you choose where to store/how to load it. The connection string is just a concept for easy to get started but it has no meaning in the actual technical level.
You can use create_from_symmetric_key(symmetric_key, hostname, device_id, **kwargs) to direct pass key, id and hub uri to sdk.

I found it's not possible to retrieve the actual connection string, but a connection string can be built from the device primary key
from azure.iot.hub import IoTHubRegistryManager
from azure.iot.device import IoTHubDeviceClient
# HUB_HOST is YOURHOST.azure-devices.net
# SHARED_ACCESS_KEY is from the registryReadWrite connection string
reg_str = "HostName={0};SharedAccessKeyName=registryReadWrite;SharedAccessKey={1}".format(
HUB_HOST, SHARED_ACCESS_KEY)
device = IoTHubRegistryManager(reg_str).get_device("MY_DEVICE_ID")
device_key = device.authentication.symmetric_key.primary_key
conn_str = "HostName={0};DeviceId={1};SharedAccessKey={2}".format(
HUB_HOST, "MY_DEVICE_ID", device_key)
client = IoTHubDeviceClient.create_from_connection_string(
conn_str)
client.connect()
# Remaining code here...

Other options you could consider include:
Use the Device Provisioning service to manage provisioning and connecting your device to your IoT hub. You won't need to generate your connection strings manually in this case.
Use X.509 certificates (recommended for production environments instead of SAS). Each device has an X.509 cert derived from the root cert in your hub. See: https://learn.microsoft.com/azure/iot-hub/tutorial-x509-introduction

testing locally managed indentity with python

I was trying to setup code using python to test the azure managed identity services and with C# I can able to test the code locally. Is there any way to test the python code locally?
Enabled managed identity in azure appservice
Added the application user(appservice) in azure SQL server and gave permissions.
this is my sample python code to connect to azure sql with managed identity
conn = db.connect('Driver={ODBC Driver 17 for SQL Server};'
'Server=testdb.database.windows.net;'
'Database=studentdb;'
'Authentication=ActiveDirectoryIntegrated;'
)
query = pd.read_sql_query('SELECT * FROM STUDENT', conn)
frame = pd.DataFrame(query)
return func.HttpResponse(frame.to_json(orient="index"), status_code=200)
Can anyone help me to test this code locally? as i do not have permissions on azure to deploy this code and test.

You can use this Moto library which allow you to test services. You can run the Lambda functions in the same way you would run python script.
if __name__ == "__main__":
event = []
context = []
lambda_handler(event, context)
If you are in a virtual environment then this will ensure that all the required dependencies installed properly for the lambda function with correct python function.

If you check this document from Microsoft, then you will find that -
Managed Identity cannot be used to authenticate locally-running applications. Your application must be deployed to an Azure service that supports Managed Identity.
It's important to understand that Managed Identity feature in Azure is ONLY relevant when, in this case, the App Service is deployed.
As an alternative you can use DefaultAzureCredential() from the Azure.Identity library which is compatible both when running locally and for the deployed web app. You can read more about how Azure.Identity works from the official docs.
Most of the time we use Azure MSI to connect Azure SQL in Azure function with python. We can can use Azure MSI to get Azure AD access token then you can use the token to connect Azure SQL.
Once you enabled system assigned identity on your Azure Web App and gave SQL permissions, you can get access to the database directly from python as shown in the snippet below.
import pyodbc
from logzero import loggerwith pyodbc.connect(
"Driver=" + driver + ";Server=" + server + ";PORT=1433;Database=" + database
+ ";Authentication=ActiveDirectoryMsi",
) as conn:
logger.info("Successful connection to database")
with conn.cursor() as cursor:
cursor.execute(“select ##version")).fetchall()
Following are the parameters used above:
Driver: We should use : “{ODBC Driver 17 for SQL Server}”
Server: The sql server on which is your database
Database: The name of your database
Authentication: To specify the connection method “ActiveDirectoryMsi”
Check the SQL database access with managed identity from Azure Web App document and Configure your local Python dev environment for Azure document for more information.

Access Minio from outside the Kubernetes cluster

I deployed Minio on Kubernetes by using the new operator:
helm repo add minio https://operator.min.io/
helm install --namespace minio-operator --create-namespace --generate-name minio/minio-operator
kubectl apply -f https://raw.githubusercontent.com/minio/operator/master/examples/tenant-tiny.yaml
sudo -E kubefwd svc
Port-Forward: 127.1.27.1 minio:80 to pod minio-ss-0-0:9000
Port-Forward: 127.1.27.2 minio-hl:9000 to pod minio-ss-0-0:9000
Port-Forward: 127.1.27.3 minio-ss-0-0.minio-hl:9000 to pod minio-ss-0-0:9000
The Tenant is working and I can access the web console both at minio:80 and minio-hl:9000
I'd like to access the storage from outside the cluster (Jupyter notebook running on Docker); this is where I receive the error "ConnectionRefusedError: [Errno 111] Connection refused":
def main():
client = Minio(
"minio:80", #minio-hl:9000 leads to the same error
access_key="minio",
secret_key="minio123!",
secure=False
)
found = client.bucket_exists("posts")
if not found:
client.make_bucket("posts")
print("Bucket 'posts' created")
else:
print("Bucket 'posts' already exists")
I also tried using pyspark with s3a jars for writing-reading objects but the connection hangs for a long time, finally receiving a similar error.
Can someone help me please? Thanks a lot!

Hope it's not too late to provide the info below:
First of all, make sure you can get access to the service after port-forward. Then try to use minio python client to do similar thing.
Actually, I have got similar issue with yours recently and finally solved by using custom http client after double confirming the service is accessible after port-forward.
from minio import Minio
from minio.error import InvalidResponseError
import urllib3
def main():
httpClient = urllib3.PoolManager(
cert_reqs="CERT_NONE")
client = Minio('my-domain:443',
access_key='xxxxx',
secret_key='xxxxx',
http_client=httpClient
)
try:
objects = client.list_objects('bucket_name', prefix='/', recursive=True)
for obj in objects:
print(obj.bucket_name)
except InvalidResponseError as err:
print("error", err)

Getting Registered IPs from Eureka Server

Is there any way we can get the service name and ips that are registered in Eureka server using python client?
I checked the python-eureka-client tutorial page but couldn't find anything related to this!

If you are using https://github.com/keijack/python-eureka-client , please check
import py_eureka_client.eureka_client as eureka_client
client = eureka_client.get_client()
for app in client.applications.applications:
for ins in app.instances:
# for ins in app.up_instances:
print("[%s] %s:%d" % (ins.instanceId, ins.ipAddr, ins.port.port))

Databricks SQL Server connection using integrated authentication

I'm trying to connect my Databricks cluster to an existing SQL Server database using python. I will like to leverage the integrated authentication method. Getting error com.microsoft.sqlserver.jdbc.SQLServerException: This driver is not configured for integrated authentication.
jdbcHostname = "sampledb-dev.database.windows.net"
jdbcPort= 1433
jdbcDatabase = "sampledb-dev"
jdbcUrl = "jdbc:sqlserver://{0}:{1}; database={2}".format(jdbcHostname, jdbcPort, jdbcDatabase)
connectionProperties={
"integratedSecurity" : "true",
"driver" : "com.microsoft.sqlserver.jdbc.SQLServerDriver"
}
print(jdbcUrl)
query ="(SELECT * FROM TABLE1.Domain)"
domains = spark.read.jdbc(url = jdbcUrl, table = query, properties = connectionProperties)
display(domains)

You can't use integratedSecurity=true with an Azure PaaS database. IntegratedSecurity is an on-premise construct.
You need to use authentication=ActiveDirectoryIntegrated or authentication=ActiveDirectoryPassword, please see JDBC docs here:
https://learn.microsoft.com/en-us/sql/connect/jdbc/connecting-using-azure-active-directory-authentication?view=sql-server-ver15
You will also need your account to be user with appropriate permissions to that database which is synch'd to Azure AD. If you use multi-factor authentication, then that's not supported for JDBC and your admin will need to provide you with a non-MFA enabled account. You'll know if this is the case because you will get a WSTrust error when trying to connect.