Having no idea why does this happen?
Check the code that creates the address. Did you use java.net.InetSocketAddress.createUnresolved(String, int) to create it?
Do NOT! Just new java.net.InetSocketAddress and it should be fixed.
P.S. This is kind of a post I write here after spending hours on a stupid bug.
So people can google it out and spend less time on it.