Node Controller (NC) fail to register

Bug #684816 reported by brian mullan
6
This bug affects 1 person
Affects Status Importance Assigned to Milestone
eucalyptus (Ubuntu)
Triaged
Medium
Unassigned

Bug Description

I had been having problems with UEC on ubuntu 10.10 getting the Node Controllers (NC) to register correctly.

Finally, after searching both the Ubuntu and Eucalyptus forums (and launchpad bugs) I found the following message on the Eucalyptus forum:

= = = = = = =

 Posted: Mon, 09/20/2010 - 13:08
 when you did 'euca_conf --register-nodes 192.168.1.100', did you type in your password for either root or eucalyptus account on NC?

 If this continue to fail, then you can perform credential exchange steps manually.

 1. Add a line to /opt/eucalyptus/etc/eucalyptus/eucalyptus.conf
 NODES="IP_ADDRESS_OF_NCs" (eg., NODES="10.0.0.2 10.0.0.3")

 2. Copy followings files from CLC's /opt/eucalytpus/var/lib/eucalyptus/keys to the same directory of NC.
 cc-client-policy.xml cloud-cert.pem cluster-cert.pem nc-client-policy.xml node-cert.pem node-pk.pem

 3. Make sure the owner of above files on NC is eucalyptus.

 4. Restart CLC/CC/NC.

= = = = = =

Here is what I found after reading the above in particular the following statement:

 2. Copy followings files from CLC's /opt/eucalytpus/var/lib/eucalyptus/keys to the same directory of NC.
 cc-client-policy.xml cloud-cert.pem cluster-cert.pem nc-client-policy.xml node-cert.pem node-pk.pem

That statement made me go look to see where those PEM key files actually were on the CLC and it appears that after a default install of UEC on 10.10 the PEM key files are placed into==> /var/lib/eucalyptus/keys/cluster1

 $ sudo ls -lg ./var/lib/eucalyptus/keys/cluster1
 total 24
 -rw-r--r-- 1 eucalyptus 1147 2010-09-28 18:43 cloud-cert.pem
 -rw-r--r-- 1 eucalyptus 1147 2010-09-28 18:43 cluster-cert.pem
 -rw-r--r-- 1 eucalyptus 1675 2010-09-28 18:43 cluster-pk.pem
 -rw-r--r-- 1 eucalyptus 1147 2010-09-28 18:43 node-cert.pem
 -rw-r--r-- 1 eucalyptus 1675 2010-09-28 18:43 node-pk.pem
 -rw-r--r-- 1 eucalyptus 512 2010-09-28 18:43 vtunpass

I know there are differences with UEC and Eucalyptus but are these files perhaps supposed to be in the /var/lib/eucalyptus/keys

On my CLC if I try to manually register an NC the error messages returned seem to think the fils should be? (see below)

>> $ euca_conf --register-nodes 172.17.34.31
>> cat: /var/lib/eucalyptus/.ssh/id_rsa.pub: Permission denied
>>
>> INFO: We expect all nodes to have eucalyptus installed in //var/lib/eucalyptus/keys for key synchronization.
>> Warning: cannot file file node-cert.pem in //var/lib/eucalyptus/keys/
>> Warning: cannot file file cluster-cert.pem in //var/lib/eucalyptus/keys/
>> Warning: cannot file file node-pk.pem in //var/lib/eucalyptus/keys/
>> Warning: cannot file file cloud-cert.pem in //var/lib/eucalyptus/keys/

If so then this may be where the NC registration problems that myself and others have been reporting are occurring.

brian mullan (bmullan)
description: updated
description: updated
Revision history for this message
C de-Avillez (hggdh2) wrote :

Thank you for opening this bug and helping make Ubuntu better.

'euca_conf' is expected to be run under 'sudo': the files under /var/lib/eucalyptus are owned by the 'eucalyptus' user. Please re-run the node registration as:

sudo euca_conf --register-nodes ...

Now, if you have your environment correctly set up, the installation of a new NC should capture the Eucalyptus basic configuration (CLC, SC, Walrus, and CC) from the -registration services -- so you might have a problem somehwere there.

Additionally, I do not like the messages stating "Warning: cannot file file ..." -- these are not clear (what does 'file file' mean?). I would like you to open a new bug on this, and add in the command and full output.

Finally, it is usually a better option to try to open bugs via 'ubuntu-bug eucalyptus' -- this will also upload logs, and allow for an easier (and hopefully faster) issue triage.

Changed in eucalyptus (Ubuntu):
importance: Undecided → Medium
status: New → Incomplete
Revision history for this message
brian mullan (bmullan) wrote : Re: [Bug 684816] Re: Node Controller (NC) fail to register

I've done that. (use sudo to register the nodes).. the NC does not register
correctly..

From google searches It seems a lot of people are having this same problem ?

thanks tho'.

On Mon, Dec 6, 2010 at 11:00 AM, C de-Avillez <email address hidden> wrote:

> Thank you for opening this bug and helping make Ubuntu better.
>
> 'euca_conf' is expected to be run under 'sudo': the files under
> /var/lib/eucalyptus are owned by the 'eucalyptus' user. Please re-run
> the node registration as:
>
> sudo euca_conf --register-nodes ...
>
> Now, if you have your environment correctly set up, the installation of
> a new NC should capture the Eucalyptus basic configuration (CLC, SC,
> Walrus, and CC) from the -registration services -- so you might have a
> problem somehwere there.
>
> Additionally, I do not like the messages stating "Warning: cannot file
> file ..." -- these are not clear (what does 'file file' mean?). I would
> like you to open a new bug on this, and add in the command and full
> output.
>
> Finally, it is usually a better option to try to open bugs via 'ubuntu-
> bug eucalyptus' -- this will also upload logs, and allow for an easier
> (and hopefully faster) issue triage.
>
> ** Changed in: eucalyptus (Ubuntu)
> Importance: Undecided => Medium
>
> ** Changed in: eucalyptus (Ubuntu)
> Status: New => Incomplete
>
> --
> You received this bug notification because you are a direct subscriber
> of the bug.
> https://bugs.launchpad.net/bugs/684816
>
> Title:
> Node Controller (NC) fail to register
>

Revision history for this message
C de-Avillez (hggdh2) wrote :

OK. Let's get on to the basics, then. Please add/attach here:

1. a description of your topology (on which machines the CLC/CC/SC/Walrus are running);
2. on each of the machines above, the output of 'initctl list | grep euca';
3. run another try of registering the NC, then:
4. from the CLC, the contents of /var/log/eucalyptus (please tar -z the file before uploading);
5. if the CC is on a different computer, please also attach the tarball for this machine /var/log/eucalyptus.

And we will go from there.

Revision history for this message
C de-Avillez (hggdh2) wrote :

Brian answered me directly on an email:

Topology is simple, 2 machines on same VLAN of a 1Gbps etherswitch. I can ping and ssh to/from CLC and NC

Ubuntu 10.10 x64 on both machines.

CLC/CC/SC/Walrus ----------
                                       |----- 1GB Ethernet Switch
NC ----------------------------------

On CLC/CC/SC/Walrus machine output of "initctl list | grep euca"

clc21$ initctl list | grep euca
eucalyptus start/running, process 2161
eucalyptus-cloud start/running
eucalyptus-network stop/waiting
eucalyptus-sc start/running
eucalyptus-walrus start/running
eucalyptus-walrus-publication start/running, process 2097
eucalyptus-cc-publication start/running, process 2098
eucalyptus-cloud-publication start/running, process 3703
eucalyptus-sc-publication start/running, process 2099
eucalyptus-cc start/running, process 2232

On NC machine output of "initctl list | grep euca"

nc31:~$ initctl list | grep euca
eucalyptus-nc start/running, process 2049
eucalyptus start/running, process 2233
eucalyptus-nc-publication start/running, process 2178
eucalyptus-network stop/waiting

I'm going to re-install again and see if it gets cleared up. I've done this once already but who knows.

Revision history for this message
xteejx (xteejx-deactivatedaccount) wrote :

Hi Brian,
There doesn't seem to have been any updates on this bug report for a while. Can you tell us if this is still an issue in the latest version of Ubuntu (10.10)? Thank you.

Revision history for this message
brian mullan (bmullan) wrote :

I'd forgotten about this because I'd gotten an email that someone had junked
it already? That was a couple months ago.

I would have to go back and retest but from watching some of the lists there
still seems to be a fair number of people describing
the same symptoms.

On Fri, Mar 25, 2011 at 3:37 PM, Teej <email address hidden> wrote:

> Hi Brian,
> There doesn't seem to have been any updates on this bug report for a while.
> Can you tell us if this is still an issue in the latest version of Ubuntu
> (10.10)? Thank you.
>
> --
> You received this bug notification because you are a direct subscriber
> of the bug.
> https://bugs.launchpad.net/bugs/684816
>
> Title:
> Node Controller (NC) fail to register
>
> To unsubscribe from this bug, go to:
> https://bugs.launchpad.net/ubuntu/+source/eucalyptus/+bug/684816/+subscribe
>

Revision history for this message
xteejx (xteejx-deactivatedaccount) wrote :

Actually I seemed to have asked about 10.10, but looking at the bug description again you already stated it was 10.10 that you were using, sorry about that. I have marked this bug as Triaged as there is quite a bit of information here and I believe there is enough information from yourself to pass this on to the developers.
Eucalyptus uses Launchpad as its bug tracker, so there is no need to send this anywhere else, and a developer should be looking into this issue soon. If you get no response from a developer, nudge us at #ubuntu-bugs on irc.freenode.net and we can look into it further.
Thank you again for reporting this to us, and please continue to report any other bugs you may find!

Changed in eucalyptus (Ubuntu):
status: Incomplete → Triaged
To post a comment you must log in.
This report contains Public information  
Everyone can see this information.

Other bug subscribers

Remote bug watches

Bug watches keep track of this bug in other bug trackers.