does anybody know how to actually fix the problem of Ubuntu Cloud VM instances in OpenStack not being able to import public keys from Nova-api's meta-data server with the following message:
2012-07-18 11:05:45,409 - util.py[WARNING]: 'http://169.254.169.254/2009-04-04/meta-data/instance-id' failed [113/120s]: url error [[Errno 111] Connection refused]
2012-07-18 11:05:52,419 - DataSourceEc2.py[CRITICAL]: giving up on md after 120 seconds
I've found numerous mentions of the problem (e.g. here or here) and tried turning this iptables rule on/off (using iptables-save and iptables-apply), but it doesn't work. The funny rule seems to be:
$ iptables -t nat -L -v | grep -n3 169.254.169.254
48-
49-Chain nova-network-PREROUTING (1 references)
50- pkts bytes target prot opt in out source destination
51: 32 1920 DNAT tcp -- any any anywhere 169.254.169.254 tcp dpt:http to:128.131.172.155:8775
52- 0 0 DNAT udp -- any any anywhere sneezy.infosys.tuwien.ac.at udp dpt:1000 to:10.0.0.2:1194
53-
54-Chain nova-network-float-snat (1 references)
Is there any good way to manually debug this, by the way?
Best Answer
Fixing the issue on an all-in-one deployment
With the help of the good folks at the IRC channel #openstack (zynzel, livemoon) we solved this by simply restarting nova-api:
You can verify that the metadata server is indeed up by issuing:
from your controller node or
with your controller node's IP address and checking that port 8775/tcp is open and listened to.
It might be the case that this is a problem with Dodai-deploy, as nova-api has to be installed after nova-compute is installed for the meta-data server to be initialized correctly.
Fixing the issue on a multinode doployment
If you run a multihost deployment and want to use Ubuntu cloud instances that pull in public keys from a metadata server you need this in your controller's nova.conf:
and this in your compute node's nova.conf:
Without this you'll get errors in your instance's console output (nova console-log test-instance) about it not being able to reach the metadata server and you won't be able to ssh into them.
Another solution is to tell your instance who you are on Launchpad through the user-data Dashboard form (or file in the terminal) so that it can pull in your public keys from there. The syntax is (not explained anywhere else but in the source code):