Quantcast

Android and reverse DNS lookup issues

classic Classic list List threaded Threaded
15 messages Options
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Android and reverse DNS lookup issues

Michael Pujos
There's an issue with Android concerning reverse DNS lookups.

On Android, if there is no DNS or if it is not working, reverse DNS requests (for example done by InetAddress.getHostByName()) will timeout after 10s. The Dalvik VM caches failed DNS queries for only 10 seconds and this is not configurable at all (I checked the code), as it doesn't have a SecurityManager. The Android java doc on InetAddress is wrong on this topic.
If you start a cling app on Android without a DNS, it will likely not work as expected. At best it will work but very slowly (because of blocking DNS queries), or you'll get ANRs etc.
There's a few hidden reverse DNS lookups in cling, most notably in the constructor of RouterImpl(), in the init() functions of the StreamServerImpl and DatagramIOImpl servers (IIRC, as I don't have my dev PC at hand to verify).
For example, the bindAddress that is passed to InetSocketAddress will trigger a reverse DNS lookup (which sucks).
Hopefull those 2 cases can be fixed with:
new ServerSocket(0)   (in StreamServerImpl)
and:
localAddress = new InetSocketAddress(0)   (in DatagramIOImpl)
I think there's a few other places with hidden lookups  but I can recall it right now (in toString() of some network related classes I think)

After spotting reverse lookups I've got my app working mostly fine (ie no bloking for 10seconds), except that HttpClient.execute() will sometimes block (for 10secs, indicating a probable DNS lookup),  but only for some UPnP devices! I suspect httpclient to have hidden reverse lookups somewhere but I have not tackled the task to find them.

Adding the ip of this device to /etc/hosts (on Android) will solve the problem. However modifying /etc/hosts requires root privilege and I modified it only to test DNS theories.

This message is just to start discussion on this important topic (cling shouldn't require a working DNS). It doesn't seem easy to totally fix and got me a few WTFs and lots of googling while troubleshooting it :)

Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: Android and reverse DNS lookup issues

James Wysynski
Interesting, that definitely makes sense.  That's a nasty issue for a mobile device, and I agree that avoiding DNS lookups is probably a good idea on Android if this is the case.  At the very least they shouldn't be running on the main thread at all, which would cause the ANR.

One other data point: I'm not using the StreamServer server portion, and disabling it via a stubbed out interface seems to have fixed all the slowdown issues for me.  I think DatagramIO is still used under this scenario, though, so I might run into some issues with more testing.

-James
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: Android and reverse DNS lookup issues

Michael Pujos
This is indeed a nasty issue and likely the cause of the infamous Router lock timeout on resume.

And it probably happens more often than we think: I recently changed my WiFi router and now every time the DHCP lease runs off, Internet connectivity is lost, there's no working DNS and this problem occur (well, it doesn't happen in my fixed version).

We definitely should eliminate all DNS lookups as they are not required for UPnP. I will investigate HttpClient which seems to do some lookups under the hood.
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: Android and reverse DNS lookup issues

Michael Pujos
As if it wasn't complicated enough, Android has all kind of issues with DHCP that can lead to loss of connectivity or weird things happening:

http://www.net.princeton.edu/android/android-stops-renewing-lease-keeps-using-IP-address-11236.html

My new router is giving me so many problems with DHCP every few hours (loss of connectivity) that I had to resort to static IP.

Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: Android and reverse DNS lookup issues

Christian Bauer
Administrator
In reply to this post by Michael Pujos
So... what can we do about this issue? If Apache HttpCrap has hidden DNS lookups, we can never fix this.
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: Android and reverse DNS lookup issues

Michael Pujos
It is not the fault of HttpClient but more of the Android platform sucking in the handling of reverse DNS lookups.

The problem happens when InetSocketAddress.InetSocketAddress(InetAddress address, int port) is called either directly or indireclty. InetSocketAddress.InetSocketAddress(int port) will not do the lookup as it binds to Inet4Address.ANY.

This trigger a revese DNS lookup as you can see by looking at InetSocketAddress.java code which will block and timeout after 10 seconds if the lookup fails (you cannot configure or disable this timeout, it is hardcoded).

In fact the lookup will not fail in all cases with this constructor, but from my experience it always fail with the WiFi IP of the android device as parameter, and for some UPnP devices IPs on the LAN (for example my Linn DS exhibit this problem, but not my PC). It seems to have something to do with how the router register reverse DNS records (or not) for particular device IPs, but I'm not expert on that (it might even be specific to the router involved).

To summarize, when the (hardware) router has no Internet access:

1. InetSocketAddress.InetSocketAddress(<android wifi ip>, port) will always timeout

This constructor is called indirectly when in the construction of the Router (the cling class) in the init function of the different Servers.

Here's the changes I've made, getting rid of the bindAddress so Inet4Address.ANY is used intead.
All these changes are OK on Android since there is only 1 bind address to bind to, but less so in the general case (PCs, etc).

in StreamServermpl.init(), replace

this.serverSocket =  new ServerSocket(configuration.getListenPort(), configuration.getTcpConnectionBacklog(), bindAddress);

with

this.serverSocket =  new ServerSocket(configuration.getListenPort(), configuration.getTcpConnectionBacklog());

int DatagramIOImpl, replace

(note that bindAddress.toString() will do a reverse DNS lookup!)

   log.info("Creating bound socket (for datagram input/output) on: " + bindAddress);
   localAddress = new InetSocketAddress(bindAddress, 0);

with

 log.info("Creating bound socket (for datagram input/output) on: " + bindAddress.getHostAddress());
 localAddress = new InetSocketAddress(0);


2. InetSocketAddress.InetSocketAddress(<any UPnP LAN device ip>, port) may timeout for some devices

This constructor is called by HttpClient when doing a http request,  when it opens the connection.
There's not much we can do here for devices that fails at reverse DNS. I think such devices cannot be handled in cling as Android cache failed reverse DNS lookup only for 10 seconds, so it is going to timeout all the time.




Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: Android and reverse DNS lookup issues

Christian Bauer
Administrator
The code in InetSocketAddress constructor that triggers the reverse lookup seems to be this:

  60     public InetSocketAddress(InetAddress address, int port) {
  61         if (port < 0 || port > 65535) {
  62             throw new IllegalArgumentException();
  63         }
  64         if (address == null) {
  65             addr = Inet4Address.ANY;
  66         } else {
  67             addr = address;
  68         }
  69         hostname = addr.getHostName();
  70         this.port = port;
  71     }
  72 

The problem is addr.getHostName(): "If this InetAddress was created with a host name, this host name will be remembered and returned; otherwise, a reverse name lookup will be performed and the result will be returned based on the system configured name lookup service. "

To prevent the lookup, we could simply provide a fake hostname to InetAddress with this constructor: InetAddress.getByAddr(String, byte[])

That would be a simple change in NetworkAddressFactory, we just have to copy/wrap the discovered bind addresses.

No?
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: Android and reverse DNS lookup issues

Michael Pujos
Thta's an interesting approach

I will try and let you know
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: Android and reverse DNS lookup issues

Michael Pujos
I've upgraded from Android 2.2 to 2.3.4 and in the later,  InetSocketAddress(<wifi ip>, int port) do not fail anymore like it used to do on 2.2, so something has changed, at least in the handling of ip = device ip.

Google has made a fix in InetSocketAddress(InetAddress address, int port) so it doesn't call getHostName() for numeric IPs.
But I couldn't find this change in Gingerbread source so it must be only in Honeycomb.

I tried to replicate my old 2.2 setup in the emulator, but it doesn't fail (timeout) either, probably because the network setup is different than on a real device.

Anyway I could test that rewriting getInetAddress() like this is harmless (but I couldn't test it on a real device having the problem):

 protected List<InetAddress> getInetAddresses(NetworkInterface networkInterface) {
    	List<InetAddress> addresses = new ArrayList<InetAddress>(); 
    	for(InetAddress address :  Collections.list(networkInterface.getInetAddresses())) {
    		try {
    			addresses.add(InetAddress.getByAddress("localhost", address.getAddress()));
			} catch (UnknownHostException e) {
				log.severe("cannot create IpAddress: " + e);
			}
    	}
    	return addresses;
   }

Finally, even on 2.3.4, InetSocketAddress(<UPnP device lan ip>, int port)  still fail like before for my LinnDS IP (which doesn't have a hostname) and after trying to workaround it (in HttpClient) with code below did not work (still timeouting):

  InetAddress address2 = InetAddress.getByAddress("fail", address.getAddress());
  InetSocketAddress remoteAddress = new InetSocketAddress(address2, port);

Which let me think that the getByAddress() trick do not work...

So it looks like we're unable to have Cling working on a LAN without Internet access in all cases.
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: Android and reverse DNS lookup issues

Christian Bauer
Administrator
So the upshot is that you have a reproducible isolated problem at least? Can you guide me through the steps necessary for duplicating it, so I can have a go at it?

This is now one of two issues that block the 1.0.3 release. (The other is the initial GENA event bug, which I can easily write a test for.)
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: Android and reverse DNS lookup issues

Michael Pujos
The condition to reproduce is as follow

- a router not connected to the Internet
- an Android device running 2.2 (as I could not reproduce anymore with 2.3.4 to which I've upgraded my phone since - one possible reason is that the device has now a hostname, which it didn't in 2.2).
- it is also possible this issue router dependant in how it handles reverse DNS

With this setup, StreamServerImpl.init(), DatagramIOImpl.init()  will block 10 seconds on some direct or indirect call to InetSocketAddress.InetSocketAddress(<android wifi ip>, port) trying to reverse DNS the ip

binding to any address (InetSocketAddress.InetSocketAddress(0) ), fixed the issue for me.

But later there still the problem that UPnP devices on the LAN that fails at reverse DNS (most likely because they don't have a hostname like my Linn DS, but could also be other Android phones), will have each http request take 10seconds which make them unusuable. But there's nothing we can do about that.

Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: Android and reverse DNS lookup issues

Christian Bauer
Administrator
I don't understand what hostnames have to do with the problem. The hostname of the client host that initiates a DNS request shouldn't matter. It also doesn't matter if the host to which the IP belongs has a configured hostname, it's not involved in the resolution at all. (It would be if talk about mDNS but that's not the case anyway. Which is btw another thing that Android sucks at, it doesn't support mDNS like all the Apple systems.)
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: Android and reverse DNS lookup issues

Michael Pujos
There's so many wifi related issues with Android between Android versions, wifi chipsets, driver customizations by manufacturers, wifi sleep modes, DHCP issues, WiFi routers not playing nice,  etc  that I don't pretend anymore to understand it all.

For mDNS, the JmDNS library works very well.
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: Android and reverse DNS lookup issues

Lasse V. Hansen
Hi there

I was struggling with this exact same problem (10sec delays when no internet connection/DNS server).
I eventually found a solution here: http://code.google.com/p/android/issues/detail?id=13117#c14 

The trick is the injectHostname(..) function. Adding this function to my custom SSLSocketFactory in the createSocket fixed everything. Just thought you might be interested.

Best
Lasse
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: Android and reverse DNS lookup issues

Michael Pujos
I think I finally have a workaround for this.

The trick is to inject a value for the hostName field of InetAddress. That way, when InetAddress.getHostName() is called, the reverseDNS is skipped because of the non null hostName.

Injecting the hostName field can be done when enumerating the usable InetAddresses  in AndroidNetworkAddressFactory:

      InetAddress address = ...

       try {
                hostName = address.getHostAddress();
            	Field field = InetAddress.class.getDeclaredField("hostName");
            	field.setAccessible(true);
            	field.set(address, hostName);
            } catch(Exception e) {
            	log.error("failed injecting hostName for InetAddress: " + hostName + ": " + e);
            }

Now, next time these InetAddresses will be passed to the constructor of InetSocketAddress,
 InetAddress.getHostName() that is called in this constructor will do nothing thanks to the injected hostName.

Apache httpclient must also be modified so all http requests bypass the reverse DNS. InetAddress used for socket creation can be hacked as described above in DefaultClientConnectionOperator.openConnection()
This probably can also be achieved with a custom SocketFactory but it is left as an exercise to anyone who has mad httpclient skills.

Loading...