I'm using python3. i need to use certificates file to make secure connection.
In this case, i used Httpsconnection class from http.client...
this class get certs file path and use it. like this:
import http.client
client=http.client.HTTPSConnection\
("epp.nic.ir",key_file="filepath\\nic.pem",cert_file="filepath\\nic.crt")
As you see, this class get path of files and works correctly.
But I need to give contents of these files. Because I want to put contents of crt file and pem file into DB. the reason is that maybe files path changes...
so i tried this:
import http.client
import base64
cert = b'''
content of cert file
'''
pem = b'''
content of pem file
'''
client=http.client.HTTPSConnection("epp.nic.ir" ,pem, cert)
as expected, i got this error:
TypeError: certfile should be a valid filesystem path
is there any way to make this class to get content of file instead of file path ?!
Or Is it possible to make changes in source codes of http for this purpose ?!
It is possible to modify Python source code, but it is not the recommended way as it definitely brings about portability, maintainability and other issues.
Consider you want to update Python version, you have to apply your modification each time you update it.
Consider you want to run your code in another machine, again the same problem.
Instead of modifying the source code, there is a better and preferrable way: extending the API.
You can subclass the existing HTTPSConnection class and override its constructor method by your own implementation.
There are plenty of ways to achieve what you need.
Here is a possible solution with subclassing:
import http.client
import tempfile
class MyHTTPSConnection(http.client.HTTPSConnection):
"""HTTPSConnection with key and cert files passed as contents rather than file names"""
def __init__(self, host, key_content=None, cert_content=None, **kwargs):
# additional parameters are also optional so that
# so that this class can be used with or without cert/key files
# as a replacement of standard HTTPSConnection
self.key_file = None
self.cert_file = None
# here we write the content of cert & pem into a temporary file
# delete=False keeps the file in the file system
# but, this time we need to remove it manually when we are done
if key_content:
self.key_file = tempfile.NamedTemporaryFile(delete=False)
self.key_file.write(key_content)
self.key_file.close()
# NamedTemporaryFile object provides 'name' attribute
# which is a valid file name in the file system
# so we can use those file names to initiate the actual HTTPSConnection
kwargs['key_file'] = self.key_file.name
# same as above but this time for cert content and cert file
if cert_content:
self.cert_file = tempfile.NamedTemporaryFile(delete=False)
self.cert_file.write(cert_content)
self.cert_file.close()
kwargs['cert_file'] = self.cert_file.name
# initialize super class with host and keyword arguments
super().__init__(host, **kwargs)
def clean(self):
# remove temp files from the file system
# you need to decide when to call this method
os.unlink(self.cert_file.name)
os.unlink(self.pem_file.name)
host = "epp.nic.ir"
key_content = b'''content of key file'''
cert_content = b'''content of cert file'''
client = MyHTTPSConnection(host, key_content=key_content, cert_content=cert_content)
# ...
Related
Currently crating inf_conf from entry script (score.py) and environment however, I have a json file that i also want to include in this.
Is there a way i can do this?
I have seen source_directory argument but json file is not in the same folder as score.py file. https://learn.microsoft.com/en-us/python/api/azureml-core/azureml.core.model.inferenceconfig?view=azure-ml-py
inf_conf = InferenceConfig(entry_script="score.py",environment=environment)
Currently, it is required that all the necessary files and objects related to the endpoint be placed in the source_directory:
inference_config = InferenceConfig(
environment=env,
source_directory='./endpoint_source',
entry_script="./score.py",
)
One workaround is to upload your JSON file somewhere else, e.g., on the Blob Storage, and download it in the init() function of your entry script. For example:
score.py:
import requests
def init():
"""
This function is called when the container is initialized/started,
typically after create/update of the deployment.
"""
global model
# things related to initializing the model
model = ...
# download your JSON file
json_file_rul = 'https://sampleazurestorage.blob.core.windows.net/data/my-configs.json'
response = requests.get(json_file_rul)
open('my_configs.json', "wb").write(response.content)
This is the part of the mailer.py script:
config = pyfig.Pyfig(config_file)
svnlook = config.general.svnlook #svnlook path
sendmail = config.general.sendmail #sendmail path
From = config.general.from_email #from email address
To = config.general.to_email #to email address
what does this config variable contain? Is there a way to get the value for config variable without pyfig?
In this case config = a pyfig.Pyfig object initialised with the contents of the file named by the content of the string config_file.
To find out what that object does and contains you can either look at the documentation and/or the source code, both here, or you can print out, after the initialisation, e.g.:
config = pyfig.Pyfig(config_file)
print "Config Contains:\n\t", '\n\t'.join(dir(config))
if hasattr(config, "keys"):
print "Config Keys:\n\t", '\n\t'.join(config.keys())
or if you are using Python 3,
config = pyfig.Pyfig(config_file)
print("Config Contains:\n\t", '\n\t'.join(dir(config)))
if hasattr(config, "keys"):
print("Config Keys:\n\t", '\n\t'.join(config.keys()))
To get the same data without pyfig you would need to read and parse at the content of the file referenced by config_file within your own code.
N.B.: Note that pyfig seems to be more or less abandoned - no updates in over 5 years, web site no longer exists, etc., so I would strongly recommend converting the code to use a json configuration file instead.
I've got a log file on external computer in my LAN network. Log is an XML file. File is not accessible from http, and is updating every second.
Currently i'm copying log file into my computer and run parser, but I want to parse file directly from external host.
How can I do it in Python? Is it possible, to parse whole file once, and later parse only new content added to the end in future versions?
You can use paramiko and xml.sax's default parser, xml.sax.expatreader, which implements xml.sax.xmlreader.IncrementalParser.
I ran the following script on local virtual machine to produce XML.
#!/bin/bash
echo "<root>" > data.xml
I=0
while sleep 2; do
echo "<entry><a>value $I</a><b foo='bar' /></entry>" >> data.xml;
I=$((I + 1));
done
Here's incremental consumer.
#!/usr/bin/env python
# -*- coding: utf-8 -*-
import time
import xml.sax
from contextlib import closing
import paramiko.client
class StreamHandler(xml.sax.handler.ContentHandler):
lastEntry = None
lastName = None
def startElement(self, name, attrs):
self.lastName = name
if name == 'entry':
self.lastEntry = {}
elif name != 'root':
self.lastEntry[name] = {'attrs': attrs, 'content': ''}
def endElement(self, name):
if name == 'entry':
print({
'a' : self.lastEntry['a']['content'],
'b' : self.lastEntry['b']['attrs'].getValue('foo')
})
self.lastEntry = None
def characters(self, content):
if self.lastEntry:
self.lastEntry[self.lastName]['content'] += content
if __name__ == '__main__':
# use default ``xml.sax.expatreader``
parser = xml.sax.make_parser()
parser.setContentHandler(StreamHandler())
client = paramiko.client.SSHClient()
# or use ``client.load_system_host_keys()`` if appropriate
client.set_missing_host_key_policy(paramiko.AutoAddPolicy())
client.connect('192.168.122.40', username = 'root', password = 'pass')
with closing(client) as ssh:
with closing(ssh.open_sftp()) as sftp:
with closing(sftp.open('/root/data.xml')) as f:
while True:
buffer = f.read(4096)
if buffer:
parser.feed(buffer)
else:
time.sleep(2)
I am assuming that another process of which you don't have access to is maintaining the xml as an object being updated every so often, and then dumping the result.
If you don't have access to the source of the program dumping the XML, you will need a fancy diffing between the two XML versions to get an incremental update to send over the network.
And I think you would have to parse the new XML each time to be able to have that diff.
So maybe you could have a python process watching the file, parsing the new version, diffing it (for instance using solutions from this article), and then you can send that difference over the network using a tool like xmlrpc. If you want to save bandwidth it'll probably help. Although I think I would send directly the raw diff via network, patch and parse the file in the local machine.
However, if only some of your XML values are changing (no node deletion or insertion) there may be a faster solution. Or, if the only operation on your xml file is to append new trees, then you should be able to parse only these new trees and send them over (diff first, then parse in the server, send to client, merge in client).
Following advice from Jean-Paul Calderone here on SO, I'm trying to modify the twisted "starttls_server" sample below to support the use of ssl.ClientCertificateOptions, to allow me to specify my private key, certificate, and trusted roots, as per http://twistedmatrix.com/documents/14.0.0/api/twisted.internet.ssl.CertificateOptions.html
from twisted.internet import ssl, protocol, defer, task, endpoints
from twisted.protocols.basic import LineReceiver
from twisted.python.modules import getModule
class TLSServer(LineReceiver):
def lineReceived(self, line):
print("received: " + line)
if line == "STARTTLS":
print("-- Switching to TLS")
self.sendLine('READY')
self.transport.startTLS(self.factory.options)
def main(reactor):
certData = getModule(__name__).filePath.sibling('server.pem').getContent()
cert = ssl.PrivateCertificate.loadPEM(certData)
factory = protocol.Factory.forProtocol(TLSServer)
factory.options = cert.options()
endpoint = endpoints.TCP4ServerEndpoint(reactor, 8000)
endpoint.listen(factory)
return defer.Deferred()
if __name__ == '__main__':
import starttls_server
task.react(starttls_server.main)
My understanding is that I effectively need to replace the cert = ssl.PrivateCertificate... and cert.options = ssl.PrivateCertificate.... lines with something like certopts = ssl.CertificateOptions(privateKey=pKeyData, certificate=certData, trustRoot=caCertsData) (having read the appropriate files in to certData, caCertsData, and pKeyData) and then pass this in to factory.options - but without pasting every variant of code I've tried, I've yet to work this out correctly - my efforts have produced varying results from the classic "OpenSSL.crypto.Error: []" - through to seemingly just dumping the contents of my 3 PEM files to screen and exiting!
Can anyone enlighten me? Thank you :)
cert.options() is already returning a CertificateOptions. The problem is that options takes authorities (as Certificate objects) as positional args, and doesn't let you pass through all the other configuration values through, so you probably want to construct a CertificateOptions directly.
Just change the factory.options = cert.options() line to factory.options = ssl.CertificateOptions(...).
However, CertificateOptions takes a pyOpenSSL PKey object as its privateKey, not the key data. So you'll need to use OpenSSL APIs to load that key, or you can extract it from a PrivateCertificate.
If you read the signature of CertificateOptions very carefully, the required types should be fairly clear. You may also need to consult the pyOpenSSL documentation as well.
I want to generate for my users temporary download link.
Is that ok if i use django to generate link using url patterns?
Could it be correct way to do that. Because can happen that I don't understand some processes how it works. And it will overflow my memory or something else. Some kind of example or tools will be appreciated. Some nginx, apache modules probably?
So, what i wanna to achieve is to make url pattern which depend on user and time. Decript it end return in view a file.
A simple scheme might be to use a hash digest of username and timestamp:
from datetime import datetime
from hashlib import sha1
user = 'bob'
time = datetime.now().isoformat()
plain = user + '\0' + time
token = sha1(plain)
print token.hexdigest()
"1e2c5078bd0de12a79d1a49255a9bff9737aa4a4"
Next you store that token in a memcache with an expiration time. This way any of your webservers can reach it and the token will auto-expire. Finally add a Django url handler for '^download/.+' where the controller just looks up that token in the memcache to determine if the token is valid. You can even store the filename to be downloaded as the token's value in memcache.
Yes it would be ok to allow django to generate the urls. This being exclusive from handling the urls, with urls.py. Typically you don't want django to handle the serving of files see the static file docs[1] about this, so get the notion of using url patterns out of your head.
What you might want to do is generate a random key using a hash, like md5/sha1. Store the file and the key, datetime it's added in the database, create the download directory in a root directory that's available from your webserver like apache or nginx... suggest nginx), Since it's temporary, you'll want to add a cron job that checks if the time since the url was generated has expired, cleans up the file and removes the db entry. This should be a django command for manage.py
Please note this is example code written just for this and not tested! It may not work the way you were planning on achieving this goal, but it works. If you want the dl to be pw protected also, then look into httpbasic auth. you can generate and remove entries on the fly in a httpd.auth file using htpasswd and the subprocess module when you create the link or at registration time.
import hashlib, random, datetime, os, shutil
# model to hold link info. has these fields: key (charfield), filepath (filepathfield)
# datetime (datetimefield), url (charfield), orgpath (filepathfield of the orignal path
# or a foreignkey to the files model.
from models import MyDlLink
# settings.py for the app
from myapp import settings as myapp_settings
# full path and name of file to dl.
def genUrl(filepath):
# create a onetime salt for randomness
salt = ''.join(['{0}'.format(random.randrange(10) for i in range(10)])
key = hashlib('{0}{1}'.format(salt, filepath).hexdigest()
newpath = os.path.join(myapp_settings.DL_ROOT, key)
shutil.copy2(fname, newpath)
newlink = MyDlink()
newlink.key = key
newlink.date = datetime.datetime.now()
newlink.orgpath = filepath
newlink.newpath = newpath
newlink.url = "{0}/{1}/{2}".format(myapp_settings.DL_URL, key, os.path.basename(fname))
newlink.save()
return newlink
# in commands
def check_url_expired():
maxage = datetime.timedelta(days=7)
now = datetime.datetime.now()
for link in MyDlink.objects.all():
if(now - link.date) > maxage:
os.path.remove(link.newpath)
link.delete()
[1] http://docs.djangoproject.com/en/1.2/howto/static-files/
It sounds like you are suggesting using some kind of dynamic url conf.
Why not forget your concerns by simplifying and setting up a single url that captures a large encoded string that depends on user/time?
(r'^download/(?P<encrypted_id>(.*)/$', 'download_file'), # use your own regexp
def download_file(request, encrypted_id):
decrypted = decrypt(encrypted_id)
_file = get_file(decrypted)
return _file
A lot of sites just use a get param too.
www.example.com/download_file/?09248903483o8a908423028a0df8032
If you are concerned about performance, look at the answers in this post: Having Django serve downloadable files
Where the use of the apache x-sendfile module is highlighted.
Another alternative is to simply redirect to the static file served by whatever means from django.