We Keep Coding

EJBCA get certificate revocation status

I am using an EJBCA pki to store certificates ... I implemented connection with EJBCA using soap and rest interfaces .
My problem now that i want to get the revocation status of certificates.
I tried two methods :
first one was with soap like below :
revocation_status = client.service.checkRevokationStatus(cert.issuer.rfc4514_string(), cert.serial_number)
and second one was using rest url but it didin't even worked :
/v1/certificate/{issuer_dn}/{certificate_serial_number}/revocationstatus
I have to mention that I am implementing all these using python
I want to know , if someoneknows how to get the revocation status of certificates on EJBCA in an easy way .
thank you very much .

You can use either of the above methods (REST only in Enterprise), or use thet standard OCSP protocol (RFC6960) to check revocation status. OCSP is the most standardized approach, and will work against any CA that has an OCSP responder.
If yo want to try WS, so you can check (debug log) the SOAP messages and look so that it implements the same in python, you can use cientToolBox.
./ejbcaClientToolBox.sh EjbcaWsRaCli checkrevocationstatus
What does a rfc4514_string look like? Some of these methods mess up the DN instead of giving a good string.

How to replicate python's `ssl.get_default_context()` in Windows C++?

My final goal is to port over a simple mqtt-paho-python script to C++ for integration within a large application.
The python example using paho is quite simple:
client = mqtt.Client(transport="websockets")
client.username_pw_set(settings['username'], password=settings['password'])
client.tls_set_context(context=ssl.create_default_context())
They set up the default TLS context, authenticate with a username and password, and then connect. This works great!
However, now I want to try to get the same secure configuration using paho-mqtt-cpp. The basic example, borrowing from their async examples, goes like this:
mqtt::connect_options connOpts;
connOpts.set_keep_alive_interval(20);
connOpts.set_clean_session(true);
connOpts.set_user_name("username");
connOpts.set_password("password123");
mqtt::ssl_options sslOpts;
connOpts.set_ssl(sslOpts);
mqtt::async_client client("wss://test.mosquitto.org:8081", "myClient");
callback cb(client, connOpts);
client.set_callback(cb);
However, ssl.get_default_context() in python's ssl library seems to do quite a bit of setup for me that isn't replicated in C++; from python's own documentation:
"For client use, if you don’t have any special requirements for your security policy, it is highly recommended that you use the create_default_context() function to create your SSL context. It will load the system’s trusted CA certificates, enable certificate validation and hostname checking, and try to choose reasonably secure protocol and cipher settings."
Most WSS connections I've tried require a certificate, and create_default_context() seems to be able to provide the proper certificates without me generating any myself.
So my questions:
(1) Where are Windows' System Default Certificates that I can use for secure connections? and
(2) What other settings do I need to manually configure that create_default_context() might be setting up for me under the hood?
I've tried looking at the source, but it's not easily discernible where the OS-specific options are.

Creating a CA certificate (.pem) file to connect to crate database

I am trying to connect to a crate database with python
from crate import client
url = '434.342.435.2:4400' # Faked these numbers for purposes of this post
conn = client.connect(url)
It seems like I need to pass the cert_file and key_file arguments to client.connect which point to my .pem and .key files. Looking in the documentation, I cannot find any resource to create or download these files.
Any advice? Even a comment pointing me to a good resource for beginners would be appreciated.

So cert and key files are part of the TLS encryption of a HTTP(S) connection that are required if you use a self-signed certificate :)
This seems to be a very good explanation of the file types
As mfussenegger explained in the comment, these files are optional and only required if your CrateDB instance is "hidden" behind a reverse proxy server like NGINX or Apache with a self-signed certificate.
A small green lock on the far left of your browser's address bar indicates HTTPS (and therefore TLS) with known certificates.
Typically certificates signed by an unknown CA - like yourself - result in a warning page and a red indicator:
Since you are also referring to username and password, they usually indicate some sort of auth (maybe basic auth) which is not yet supported by crate-python :(

Access a page that require Safenet USB Token from urllib2 ot httplib

When I have a software certificate I do like this.
import httplib
CLIENT_CERT_FILE = '/path/to/certificate.pem'
connection = httplib.HTTPSConnection('url-to-open', cert_file=CLIENT_CERT_FILE)
connection.request('GET', '/')
response = connection.getresponse()
print response.status
data = response.read()
print data
How can I do the same with a Safenet USB Token ?

TL;DR there are significant caveats and security issues with doing this in Python.
A working "solution" involves using a PKCS#11 library to read the certificate from the key, then somehow persisting the certificate on the disk, and finally passing the resulting file path to the request object.
There will also be differences with each security stick's particularities. Some sticks do not offer to store a certificate along with its private key (aka a .pfx or .p12 file) which will essentially make this solution unworkable. I didn't have access to a Safenet stick, so used my own, please bear this in mind.
A solution for this requires quite a bit of work. Your use of a security dongle means that your client certificates are located onto the dongle itself. So, in order to achieve the same level of functionality, you need to write code to extract the certificate from there and feed it to your request object.
1. HTTPS-capable libraries in Python
Your requirement of using httplib (http.client for python 3.x) or urllib introduces a big caveat that the certificate used in the request has to be a file on the disk (and the same can be said of all libraries building in top of them, e.g. requests). See cnelson's answer to How to open ssl socket using certificate stored in string variables in python for the reason (in short: it's because python's ssl library makes use of a native C library which does not offer passing in-memory objects as the certificate). Also see the next answer from Dima Tisnek detailing possible workarounds with varying degrees of hackmanship.
If writing your certificate (even temporarily) on the disk is a non-starter for you, as it may very well be since you use a security stick, then it's not starting off looking good.
2. Getting the certificate from the security stick
Your biggest challenge is to get your hand on the certificate, which is currently nestled inside the security stick. Safenet sticks, like many others, are at the core a PKCS#11 capable SmartCard. I suggest you familiarise yourself with the concepts, but in essence, SmartCard is a standardised chip design, and PKCS#11 is a standardised protocol to interface with it. "Standardised" comes with caveats of course since many vendors come up with their own implementations, but it could probably be standardised enough for your purpose. The trick here will be to use available PKCS#11 interfaces on the stick to extract the certificate's attributes. This is what web browsers essentially do when using the stick to authenticate on websites using the stored certificate, so you need to have your python program do a similar thing.
2.1 Selecting a PKCS#11 library
Unfortunately, there are only a few libraries that come up when searching for "python pkcs11". I have no vested interest in either of them, and there may exist other less prominent ones.
python-pkcs11 (pypi, github, reference) offers a "high level, pythonic implementation of PKCS#11". It may be easier to use overall, but may lack compatibility and/or features depending on what you want to do, however I suspect simply retrieving certificates may be alright.
PyKCS11 (pypi, github, reference) on the other hand is a wrapper around a native PKCS#11 library, to which it will defer the calls. This one is lower-level, but looks more complete, plus may have the advantage to offer using your particular vendor's implementation if relevant.
2.2 Example code
For the example, I'll be using the user-friendlier API of python-pkcs11. Please bear in mind that this code is not thoroughly tested (and has been simplified in parts) and serves as illustrating the general idea.
import pkcs11
import asn1crypto.pem
import urllib.request
import tempfile
import ssl
import os
# this is OpenSC's implementation of PKCS#11
# other security sticks may come with another implementation.
# choose the most appropriate one
lib = pkcs11.lib('/usr/lib/pkcs11/opensc-pkcs11.so')
# tokens may be identified with various names, ids...
# it's probably rare that more than one at a time would be plugged in
token = lib.get_token(token_serial='<token_serial_value>')
pem = None
with token.open() as sess:
pkcs11_certificates = sess.get_objects(
{
pkcs11.Attribute.CLASS: pkcs11.ObjectClass.CERTIFICATE,
pkcs11.Attribute.LABEL: "Cardholder certificate"
})
# hopefully the selector above is sufficient
assert len(pkcs11_certificates) == 1
pkcs11_cert = pkcs11_certificates[0]
der_encoded_certificate = pkcs11_cert.__getitem__(pkcs11.Attribute.VALUE)
# the ssl library expects to be given PEM armored certificates
pem_armored_certificate = asn1crypto.pem.armor("CERTIFICATE",
der_encoded_certificate)
# this is the ugly part: persisting the certificate on disk
# i deliberately did not go with a sophisticated solution here since it's
# such a big caveat to have to do this...
certfile = tempfile.mkstemp()
with open(certfile[1], 'w') as certfile_handle:
certfile_handle.write(pem_armored_certificate.decode("utf-8"))
# this will instruct the ssl library to provide the certificate
# if asked by the server.
sslctx = ssl.create_default_context()
sslctx.load_cert_chain(certfile=certfile[1])
# if your certificate does not contain the private key, find it elsewhere
# sslctx.load_cert_chain(certfile=certfile[1],
# keyfile="/path/to/privatekey.pem",
# password="<private_key_password_if_applicable>")
response = urllib.request.urlopen("https://ssl_website", context=sslctx)
# Cleanup and delete the "temporary" certificate from disk
os.remove(certfile[1])
3. Conclusion
I'd say that Python is not going to be the best bet for doing ssl client authentication using security sticks. The fact that most ssl libraries require the certificate to be present on the disk works directly against the benefits (and sometimes, requirements) of the use of a security stick in the first place. I'm well aware that this answer does not provide a full solution to this problem, but hopefully exposes the challenges in enough detail to make an educated decision on whether to pursue this further or to find another way.
In any case, good luck.

Httplib2 ssl error

Today I faced one interesting issue.
I'm using the foursquare recommended python library httplib2 raise
SSLHandshakeError(SSLError(1, '_ssl.c:504: error:14090086:SSL routines:SSL3_GET_SERVER_CERTIFICATE:certificate verify failed'),)
while trying to request an oauth token
response, body = h.request(url, method, headers=headers, body=data)
in
_process_request_with_httplib2 function
does anyone know why this happens?

If you know that the site you're trying to get is a "good guy", you can try creating your "opener" like this:
import httplib2
if __name__ == "__main__":
h = httplib2.Http(".cache", disable_ssl_certificate_validation=True)
resp, content = h.request("https://site/whose/certificate/is/bad/", "GET")
(the interesting part is disable_ssl_certificate_validation=True )
From the docs:
http://bitworking.org/projects/httplib2/doc/html/libhttplib2.html#httplib2.Http
EDIT 01:
Since your question was actually why does this happen, you can check this or this.
EDIT 02:
Seeing how this answer has been visited by more people than I expected, I'd like to explain a bit when disabling certificate validation could be useful.
First, a bit of light background on how these certificates work. There's quite a lot of information in the links provided above, but here it goes, anyway.
The SSL certificates need to be verified by a well known (at least, well known to your browser) Certificate Authority. You usually buy the whole certificate from one of those authorities (Symantec, GoDaddy...)
Broadly speaking, the idea is: Those Certificate Authorities (CA) give you a certificate that also contains the CA information in it. Your browsers have a list of well known CAs, so when your browser receives a certificate, it will do something like: "HmmmMMMmmm.... [the browser makes a supiciuous face here] ... I received a certificate, and it says it's verified by Symantec. Do I know that "Symantec" guy? [the browser then goes to its list of well known CAs and checks for Symantec] Oh, yeah! I do. Ok, the certificate is good!
You can see that information yourself if you click on the little lock by the URL in your browser:
However, there are cases in which you just want to test the HTTPS, and you create your own Certificate Authority using a couple of command line tools and you use that "custom" CA to sign a "custom" certificate that you just generated as well, right? In that case, your browser (which, by the way, in the question is httplib2.Http) is not going to have your "custom" CA among the list of trusted CAs, so it's going to say that the certificate is invalid. The information is still going to travel encrypted, but what the browser is telling you is that it doesn't fully trust that is traveling encrypted to the place you are supposing it's going.
For instance, let's say you created a set of custom keys and CAs and all the mambo-jumbo following this tutorial for your localhost FQDN and that your CA certificate file is located in the current directory. You could very well have a server running on https://localhost:4443 using your custom certificates and whatnot. Now, your CA certificate file is located in the current directory, in the file ./ca.crt (in the same directory your Python script is going to be running in). You could use httplib2 like this:
h = httplib2.Http(ca_certs='./ca.crt')
response, body = h.request('https://localhost:4443')
print(response)
print(body)
... and you wouldn't see the warning anymore. Why? Because you told httplib2 to go look for the CA's certificate to ./ca.crt)
However, since Chrome (to cite a browser) doesn't know about this CA's certificate, it will consider it invalid:
Also, certificates expire. There's a chance you are working in a company which uses an internal site with SSL encryption. It works ok for a year, and then your browser starts complaining. You go to the person that is in charge of the security, and ask "Yo!! I get this warning here! What's happening?" And the answer could very well be "Oh boy!! I forgot to renew the certificate! It's ok, just accept it from now, until I fix that." (true story, although there were swearwords in the answer I received :-D )

Recent versions of httplib2 is defaulting to its own certificate store.
# Default CA certificates file bundled with httplib2.
CA_CERTS = os.path.join(
os.path.dirname(os.path.abspath(__file__ )), "cacerts.txt")
In case if you're using ubuntu/debian, you can explicitly pass the path to system certificate file like
httplib2.HTTPSConnectionWithTimeout(HOST, ca_certs="/etc/ssl/certs/ca-certificates.crt")

Maybe this could be the case:
I got the same problem and debugging the Google Lib I found out that the reason was that I was using an older version of httplib2(0.9.2). When I updated to the most recent (0.14.0) it worked.
If you already install the most recent, make sure that some lib is not installing an older version of httplib2 inside its dependencies.

When you see this error with a self-signed certificate, as often happens inside a corporate proxy, you can point httplib2 to your custom certificate bundle using an environment variable. When, for example, you don't want to (or can't) modify the code to pass the ca_certs parameter.
You can also do this when you don't want to modify the system certificate store to append your CA cert.
export HTTPLIB2_CA_CERTS="\path\to\your\CA_certs_bundle"