is there any way to specify dns server should be used by socket.gethostbyaddr()?
Please correct me, if I'm wrong, but isn't this operating system's responsibility? gethostbyaddr is just a part of libc and according to man:
The gethostbyname(), gethostbyname2() and gethostbyaddr() functions each return a
pointer to an object with the following structure describing an internet host refer-
enced by name or by address, respectively. This structure contains either the infor-
mation obtained from the name server, named(8), or broken-out fields from a line in
/etc/hosts. If the local name server is not running these routines do a lookup in
/etc/hosts.
So I would say there's no way of simply telling Python (from the code's point of view) to use a particular DNS, since it's part of system's configuration.
Take a look at PyDNS.
Related
I want to bypass proxy for domains like:
http://server-1:5000
http://server-2:5000
NO_PROXY=server-1, server-2
server-1 and server-2 are basically services attached to kube pods, so they can change dynamically during runtime.
I want to bypass proxy for any domains in this format. For example, at any point it can even reach to server-124.
It would have been easier if domain name was in this format : subdomain.domain.com
like 1.server.com , 2.server.com.
I believe, NO_PROXY=.server.com would have worked in this case.
But my current scenario is a little different. So, can it be done ?
Yes it can be done. NO_PROXY=*.server.*
This will help to bypass proxy for any domain like .server. or server-124.
This wild card match will handle that.
In some cases, setting no_proxy to * effectively disables proxies altogether, but this is not a universal rule.
No implementation performs a DNS lookup to resolve a hostname to an IP address when deciding if a proxy should be used. Do not specify IP addresses in the no_proxy variable unless you expect that the IPs are explicitly used by the client.
The same holds true for CIDR blocks, such as 18.240.0.1/24. CIDR blocks only work when the request is directly made to an IP address.
The libraries that support the http_proxy environment variable generally also support a matching no_proxy that names things that shouldn't be proxied. The exact syntax seems to vary across languages and libraries but it does seem to be universal that setting no_proxy=example.com causes anything.example.com to not be proxied either.
This is relevant because the Kubernetes DNS system creates its names in a domain based on the cluster name, by default cluster.local. The canonical form of a Service DNS name, for example, is service-name.namespace-name.svc.cluster.local., where service-name and namespace-name are the names of the corresponding Kubernetes objects.
I am trying to a simple make a http request to a server inside my company, from a dev server. I figured out that depending on the origin / destination server, I might, or not, to be forced to use qualified name of the destination server, like srvdestination.com.company.world instead of just srvdestination.
I am ok with this, but I don't understand how come my DB connection works?
Let's say I have srvorigin. Now, to make http request, I must use qualified name srvdestination.com.company.world. However, for database connection, the connection string with un-qualified name is enough psycopg.connect(host='srvdestination', ...) I understand that protocols are different, but how psycopg2 does to resolve the real name?
First it all depend on how the name resolution subsystem of your OS is configured. If you are on Unix (you did not specify), this is governed by /etc/resolv.conf. Here you can provide the OS with a search list: if a name has not "enough" dots (the number is configurable) then a suffix is added to retry resolution.
The library you use to do the HTTP request may not query the OS for name resolution and do its DNS resolution itself. In which case, it can only work with the information you give it (but it could as well re-use the OS /etc/resolv.conf and information in it), hence the need to use the full name.
On the contrary, the psycopg2 may use the OS resolution mechanism and hence dealing with "short" names just fine.
Both libraries should have documentation on how they handle hostnames... or otherwise you need to study their source code. I guess psycopg2 is a wrapper around the default libpq standard library, written in C if I am not mistaken, which hence certainly use the standard OS resolution process.
I can understand the curiosity around this difference but anyway my advice is to keep short names when you type commands on the shell and equivalent (and even there it could be a problem), but always use FQDNs (Fully Qualified Domain Names) in your program and configuration files. You will avoid a lot of problems.
How do I get the remote IP address in Python?
I tried searching Google but couldn't find any useful results. os.environ['REMOTE_ADDR'] is giving KeyError: 'REMOTE_ADDR'
You're accessing the operating system's os environment, not the request's.
The WSGI callable should be passed two variables, environ and start_response, and that environ variable will have the variables you're looking for.
Those variables would only be present in the actual os.environ if you were running a CGI app.
Depending on the web framework you're using, you might not have access to this. If you're passed a request object, this will likely end up in request.META or something similar.
If you're not using any framework, that will be in the environ dict that is passed to your wsgi callable.
As noted in another answer, REMOTE_ADDR doesn't have to be their as per the spec, but if you're using Apache's mod_wsgi, it should be there.
If you have access to the TCP socket, you can use socket.getpeername() to get the remote address. Docs are here.
It seems obvious that it would use the twisted names api and not any blocking way to resolve host names.
However digging in the source code, I have been unable to find the place where the name resolution occurs. Could someone point me to the relevant source code where the host resolution occurs ( when trying to do a connectTCP, for example).
I really need to be sure that connectTCP wont use blocking DNS resolution.
It seems obvious, doesn't it?
Unfortunately:
Name resolution is not always configured in the obvious way. You think you just have to read /etc/resolv.conf? Even in the specific case of Linux and DNS, you might have to look in an arbitrary number of files looking for name servers.
Name resolution is much more complex than just DNS. You have to do mDNS resolution, possibly look up some LDAP computer records, and then you have to honor local configuration dictating the ordering between these such as /etc/nsswitch.conf.
Name resolution is not exposed via a standard or useful non-blocking API. Even the glibc-specific getaddrinfo_a exposes its non-blockingness via SIGIO, not just a file descriptor you can watch. Which means that, like POSIX AIO, it's probably just a kernel thread behind your back anyway.
For these reasons, among others, Twisted defaults to using a resolver that just calls gethostbyname in a thread.
However, if you know that for your application it is appropriate to have DNS-only hostname resolution, and you'd like to use twisted.names rather than your platform resolver - in other words, if scale matters more to you than esoteric name-resolution use-cases - that is supported. You can install a resolver from twisted.names.client onto the reactor, appropriately configured for your application and all future built-in name resolutions will be made with that resolver.
I'm not massively familiar with twisted, I only recently started used it. It looks like it doesn't block though, but only on platforms that support threading.
In twisted.internet.base in ReactorBase it looks like it does the resolving through it's resolve method which returns a deferred from self.resolver.getHostByName.
self.resolver is an instance of BlockingResolver by default which does block, but it looks like that if the platform supports threading the resolver instance is replaced by ThreadedResolver in the ReactorBase._initThreads method.
I want to provide dbus methods and signals on a custom bus (i.e. not SessionBus or SystemBus). If I start a test copy of the dbus-daemon from the command line, as described in dbus-daemon man page, like so:
dbus-daemon --session --print-address
then this returns for example the address:
unix:abstract=/tmp/dbus-vthAiAw4am,guid=60da6b6ef244a0dbdb9710a800002218
I can use this address in d-feet to "Connect to Other Bus", and there is nothing there. Now, I would like to claim a name on that bus and export objects to provide dbus methods and signals using Python. I have tried reading the code behind dbus.service.BusName where I would normally pass in the Session or System bus, but I simply get lost. Anyone know how to do this (if even possible)?
Looking at the source code for d-feet was of course an easier way to find the answer than browsing the entire dbus-python lib. An address like the one in the question could be used when claiming a bus name, by passing in a dbus.bus.BusConnection object with the address as argument, like so:
bus_name = dbus.service.BusName('my.testbus.test',
dbus.bus.BusConnection('unix:abstract=/tmp/dbus-vthAiAw4am'))
I can then export methods and emit signals on this bus.