[nailgun] Addition of new Compute node cause puppet run on all controllers (fuel cli)

Could you provide diagnostic snapshot of the environment?

Diagnostic snapshot http://www.filedropper.com/fuel-snapshot-2014-02-1417-31-31

You've redefined deployment data and then sent all of your nodes on /api/v1/clusters/5/orchestrator/deployment/ handler and us result deployment was started on all nodes.

If you wants deploy only compute then you need to send on this handler only your compute nodes.

Run

     fuel deployment --env-id 3 --delete

command between 3 and 4 step.

It's not obvious and we need to rethink usecase of custom settings functional.
Maybe we can keep settings for nodes in nodes models.

Workaround:

psql -U nailgun -W -h 127.0.0.1
update clusters set is_customized=false where id=${ID};

Shouldn't be high priority if there's a safe and simple workaround.

Added to Known Issues in 5.0 Release Notes

AFAIU the issue takes place because of some features in serialization process for deployment: if deployment data has been changed via fuel-cli it is stored in 'replaced_deployment_info' attribute of Cluster model and it is that attribute which will be checked in first turn against deployment info retrieving: https://github.com/stackforge/fuel-web/blob/master/nailgun/nailgun/task/task.py#L138-L139

So as you can see this data will be using in all future cluster redeployment regardless how many nodes you have added to cluster after settings change they are not deployed.

As Evgeniy L has noticed clearing of mentioned attribute will solve the issue, but it must be done explicitly by user via fuel-cli or via direct interaction with db (and btw, workaround that was introduced by Roman Sokolkov doesn't fix the problem so corresponding 5.0 Release Notes entry should be updated).

My proposal is to modify nailgun code so that 'replaced_deployment_info' is purged after success deployment of cluster.

Of course such solution is not transparent for user and maybe as Evgeniy said we should reorganize our workflow of managing deployment data for cluster but still it will be fix.

Artem,

I've checked this:

"My proposal is to modify nailgun code so that 'replaced_deployment_info' is purged after success deployment of cluster."

So finally i've succeed with this (Fuel 5.0)

- is_customized='f'
- replaced_deployment_info='{}'

Roman,

I want to clarify situation with the issue one more time so that no one has any misunderstandings left: first of all, 'is_customized' flag controls displaying of ui notification about that cluster is changed via fuel-cli and I think one should not reset it if his or her environment is actually changed using fuel client because this information has some value, hasn't it?

And second: only purging 'replaced_deployment_info' makes newly added nodes deployed properly and is fix for the problem introduced in bug upon the table.

Listed as "Known Issue" in 5.0.1 Release Notes.

Meg,

Yes, it is listed in "Known Issues" as I mentioned in my previous comment such workaround won't fix particularly nothing. I think we should update our documentation from

psql -U nailgun -W -h 127.0.0.1
update clusters set is_customized=false where id=${ID};

to following snippet:

psql -U nailgun -W -h 127.0.0.1
update clusters set replaced_deployment_info='{}' where id=${ID};

with mention that user must do this before deployment of newly added nodes to already deployed cluster if he/she has modified deployment info for that cluster via cli.

So, this is by design and if user changed cluster configuration once, with help of fuel client, he loses ability to modify it via UI in future. I am not sure is this really that bad, but as for now fields:

- replaced_deployment_info
- is_customized

wont be cleared by itself

As fix for this bug we can provide ability to reset fields by fuel client

Or just provide Artems workouround in known issues section:

psql -U nailgun -W -h 127.0.0.1
update clusters set replaced_deployment_info='{}' where id=${ID};

Fix proposed to branch: master
Review: https://review.openstack.org/112333

Actually there is already such command that i wanted to implement previously,

fuel deployment --delete --env 5

Reviewed: https://review.openstack.org/112333
Committed: https://git.openstack.org/cgit/stackforge/fuel-web/commit/?id=4cd225ca9d0eb0b1dd398b07ad8202169f9c3ea4
Submitter: Jenkins
Branch: master

commit 4cd225ca9d0eb0b1dd398b07ad8202169f9c3ea4
Author: Dima Shulyak <email address hidden>
Date: Wed Aug 6 18:48:44 2014 +0300

    Store replaced info on node instead of cluster

    - add necessery columns to node model
    - add replace/get provisioning/deployment info on objects.Cluster
    - api interaction with cli is not changed
    - added migration from old data model to new

    DeploymentInfo treated everywhere as list, because it will consist of
    - multiple yaml files per node, on role basis

    ProvisioningInfo treated as dict
    - one yaml for cluster section (engine)
    - yaml per node with provisioning params of this node

    It is possible to manage cluster from UI and cli

    Change-Id: I9345a5e9adadead2c149e85fab139ae4e5615cf1
    Closes-Bug: #1280318

verified on {

    "build_id": "2014-09-17_21-40-34",
    "ostf_sha": "64cb59c681658a7a55cc2c09d079072a41beb346",
    "build_number": "11",
    "auth_required": true,
    "api": "1.0",
    "nailgun_sha": "eb8f2b358ea4bb7eb0b2a0075e7ad3d3a905db0d",
    "production": "docker",
    "fuelmain_sha": "8ef433e939425eabd1034c0b70e90bdf888b69fd",
    "astute_sha": "f5fbd89d1e0e1f22ef9ab2af26da5ffbfbf24b13",
    "feature_groups": [
        "mirantis"
    ],
    "release": "5.1",
    "release_versions": {
        "2014.1.1-5.1": {
            "VERSION": {
                "build_id": "2014-09-17_21-40-34",
                "ostf_sha": "64cb59c681658a7a55cc2c09d079072a41beb346",
                "build_number": "11",
                "api": "1.0",
                "nailgun_sha": "eb8f2b358ea4bb7eb0b2a0075e7ad3d3a905db0d",
                "production": "docker",
                "fuelmain_sha": "8ef433e939425eabd1034c0b70e90bdf888b69fd",
                "astute_sha": "f5fbd89d1e0e1f22ef9ab2af26da5ffbfbf24b13",
                "feature_groups": [
                    "mirantis"
                ],
                "release": "5.1",
                "fuellib_sha": "d9b16846e54f76c8ebe7764d2b5b8231d6b25079"
            }
        }
    },
    "fuellib_sha": "d9b16846e54f76c8ebe7764d2b5b8231d6b25079"

}