Fails to translate the derived polices
Affects | Status | Importance | Assigned to | Milestone | |
---|---|---|---|---|---|
Heat Translator |
New
|
High
|
Sahdev Zala |
Bug Description
Tried to translate https:/
while i investigate the issue, found that heat-translator expects the policy as tosca.policies.
Changed in heat-translator: | |
importance: | Undecided → High |
assignee: | nobody → Sahdev Zala (spzala) |

Sahdev Zala (spzala) wrote : | #1 |

Bob Haddleton (bob-haddleton) wrote : | #2 |
The tacker code automatically imports the tacker-specific definitions files based on the version of the template.
See https:/
https:/
https:/
Standalone tosca-parser and heat-translator cannot parse Tacker VNFDs/NSDs without including the associated tacker imports.

Sahdev Zala (spzala) wrote : | #3 |
Hi Bob, thanks for the input, it's helpful!! As you noticed, the stand alone tosca-parser requires the type definition which typically is provided either via normative type (as defined in the TOSCA toscaparser/

Sahdev Zala (spzala) wrote : | #4 |
I tried several things but seems like tosca-parser works fine if type definition is provided. For example, if I update NFV definition file in the tosca-parser with (policy_types:
tosca.
derived_from: tosca.policies.

Kanagaraj Manickam (kanagaraj-manickam) wrote : | #5 |
I have verified from tacker that, it passes the scaling definitions (https:/

Sahdev Zala (spzala) wrote : | #6 |
OK, cool. Let's continue debug - so for tosca-parser to know the definition it has be via:
1. toscaparser/
2. toscaparser/
Anything else is considered as custom types and needed to be imported in the service template.
3. *.yaml which is then imported in the service template
4. define types in the service template itself
I have verified all those in my local environment and that seems working fine. You aren't doing #1, #3 or #4 I guess. How do you pass tacker_defs.yaml to parser and expect the parser to handle it? I guess the issue is happening even before heat-translator, can you invoke tosca-parser and see if it passes parsing of it?

Kanagaraj Manickam (kanagaraj-manickam) wrote : | #7 |
Sahdev, we have not introduced any recent changes in the way tacker passes the TOSCA custom type definitions to HT and been working fine across many releases, which means the HT is parsing the the TOSCA used in tacker. and we have not changed this logic.
In ToscaTemplate, tacker provides both custom type definitions in addition to the actual template. so I feel that HT should consider the custom types provided as part of ToscaTemplate object in addition to the above mentioned #1 and #2.
Kindly let me know if tacker should change the way it passes the custom types definition to HT?

Sridhar Ramaswamy (srics-r) wrote : | #8 |
I concur w/ Kanagaraj's findings.
Custom type definitions is a back bone in making tosca-nfv profile usable. As bobh mentioned earlier we have been doing the imports (#3 above) for a while now and it was working fine.

Sahdev Zala (spzala) wrote : | #9 |
Kanagarj and Sridhar, thanks for digging more into it. I totally agree and can't see TOSCA templates without custom types, and that's we have several template examples created and supported in the tosca-parser and HT. As I said above use of #3 requires an "import" statement in the TOSCA service template. For example, https:/
I am all up to fix this, if it is in issue with TP or HT with the highest priority but if I can't recreate it and see that custom type support is working fine I am helpless :(.
@Bob, you know tosca-parser and Tacker well. If you can jump on here that would be great.
Here is an example of how the custom type works:
Custom type definition as scalingcustom_
tosca_definitio
description: >
Template for deploying servers based on policies.
policy_types:
tosca.
derived_from: tosca.policies.
Service Template (note the imports statement):
tosca_definitio
description: >
Template for deploying servers based on policies.
imports:
- scalingcustom_
topology_template:
node_templates:
my_server_1:
type: tosca.nodes.Compute
capabilities:
host:
os:
# host Operating System image properties
type: Linux
policies:
- asg:
type: tosca.policies.
targets: [my_server_1]
triggers:
properties:

Sahdev Zala (spzala) wrote : | #10 |
Please look at above example and my comments. If that doesn't make sense please help me recreate the issue and since I don't have Tacker environment if you can provide where tosca-parser is failing and more detail that would be very helpful to solve this.

Bob Haddleton (bob-haddleton) wrote : | #11 |
I'll try to look at this later today. I think the first step is to take the existing template and add the two imports that Tacker adds automatically:
https:/
https:/
and see if the template can be processed using standalone TP.
Assuming it CAN then the problem is in Tacker. If it cannot, then the problem is likely in TP.
I should be able to try it after lunch my time.
Bob

Sahdev Zala (spzala) wrote : | #12 |
@Bob, that sounds like a right approach to go forward. Thanks!! I was on IRC and asked Bharath to help just few minutes back too.

bharaththiruveedula (bharath-ves) wrote : | #13 |
@spzala, as per our discussion,the problem is improper ordering of "targets" for scaling resource. The target attribute must be in parallel to "properties", but the template kanagaraj posted contains "targets" inside "properties".
@KanagarajM, please try the below template
tosca_definitio
description: sample-
metadata:
template_name: sample-
topology_template:
node_templates:
VDU1:
type: tosca.nodes.
properties:
image: cirros-
flavor: m1.tiny
CP1:
type: tosca.nodes.
properties:
management: true
order: 0
requirements:
- virtualLink:
node: VL1
- virtualBinding:
node: VDU1
VDU2:
type: tosca.nodes.
properties:
image: cirros-
flavor: m1.tiny
CP2:
type: tosca.nodes.
properties:
management: true
order: 0
requirements:
- virtualLink:
node: VL1
- virtualBinding:
node: VDU2
VL1:
type: tosca.nodes.nfv.VL
properties:
vendor: Tacker
policies:
- SP1:
type: tosca.policies.
targets: [VDU1, VDU2]
properties:
cooldown: 120

Sridhar Ramaswamy (srics-r) wrote : | #14 |
catching up a bit, looks you all are further down in narrowing down the issue.
@spzala: quick note on the imports. The imports doesn't show up in tacker's tosca templates (including the sample you refer to [1]) because they happen implicitly. The imports are unconditionally added by the code in [2] & [3] before sending off to TP.
[1] https:/
[2] https:/
[3] https:/

Sahdev Zala (spzala) wrote : | #15 |
@Sridhar: thanks for looking at it and update, I really appreciate it.
AS Bharath commented above, there seems no problem in resolving custom types. The implicit references you mentioned are working fine. About the second issue of translated template not providing expected result is due to the fact that the Tacker template (https:/
Correct syntax:
policies:
- SP1:
type: tosca.policies.
targets: [VDU1, VDU2]
properties:
cooldown: 120
Incorrect syntax as provided in the https:/
policies:
- SP1:
type: tosca.policies.
properties:
cooldown: 120
targets: [VDU1, VDU2]

Kanagaraj Manickam (kanagaraj-manickam) wrote : | #16 |
sridhar, sahdev, thanks for continously looking into it.
tbh, thanks for spotting the root cause. i will try it and update the findings here

Sahdev Zala (spzala) wrote : | #17 |
@Kanagaraj, no problem! Sure please try it, I know you are going to give it a priority but just a friendly reminder that this is the main issue right now that we all need to make sure we are fine with before TP and HT new release. So I appreciate your priority here. Thanks!

Kanagaraj Manickam (kanagaraj-manickam) wrote : | #18 |
@Sahdev, i tried to test it and pasted the generated templates here
http://
I have highlighted the issues
1. outputs are missing from provider template for scaling unit
2. invalid Alarm is generated. supposed to be removed when as scaling policy does not include it. (there is an separate policy called AlarmPolicy)
Tacker needs these issues to get fixed from HT. Kindly help. thanks

Sahdev Zala (spzala) wrote : | #19 |
@Kanagaraj, thanks!! Sure, looking at it.

Sahdev Zala (spzala) wrote : | #20 |
@Kanagaraj, for #1 there is no 'outputs' section in the TOSCA template itself so translator can't create one. Bharath kindly run translator in Tacker env by adding outputs section in the TOSCA template and that will produce outputs in the main template
e.g. Adding outputs in the template as: policies:
- SP1:
type: tosca.policies.
targets: [VDU1, VDU2]
properties:
cooldown: 120
outputs:
ip_address:
value: { get_attribute: [ VDU1, public_address ] }
will create outputs in the translated template as:
outputs:
ip_address:
value:
get_attr:
- VDU1
- networks
- private
- 0
Hope makes sense.

Sahdev Zala (spzala) wrote : | #21 |
@Kanagaraj, or #2 to not to unnecessary create Alarm resource I have created this patch https:/
https:/
Thanks!

Kanagaraj Manickam (kanagaraj-manickam) wrote : | #22 |
@sahdev, thanks for #2. for #1, i would like HT to provide these details as part of provider template generation, which is required by tacker. and this output should be part of scaling translation to facilitate getting the IP address. and this would be the case for in general, IMO. so when heat-tranaltor command is used to translate it would provide same output.

Sahdev Zala (spzala) wrote : | #23 |
@Kanagaraj, np and thank you much for the quick feedback. So are we good with #2?
About #1, sorry but I didn't get the requirement completely. As we know, in my understanding the 'outputs' section which is optional in Heat or TOSCA or any other orchestration in general is a way to display some information to the user of template by an author of template.
Per original design in translator, if the input template has 'outputs' section the translator reads it and try to produce an available mapping in Heat. I am good changing/enhance this per Tacker's requirement but can you please provide more detail, ideally a blueprint with a small example that we can use to understand it better and later to test. Also, I believe this work probably is time consuming and can impact other other test templates we have too which you may agree with me too, can we consider this as a new feature that can be made available in 0.8.0 (not in 0.7.0 which is about to release). I am good on releasing 0.8.0 or 0.7.1 sooner than later when this feature, along with any other near term need from Tacker, is available. I sincerely hope you would agree with me here. Thanks!

Kanagaraj Manickam (kanagaraj-manickam) wrote : | #24 |
@sahdev, I tried to use the new Python API you have introduced for translate and its generating the HOT with following invalid entries in place:
VDU2:
type: OS::Nova::Server
properties:
user_
availabil
image: cirros-
flavor: m1.tiny
networks:
- port: '{ get_resource: CP2 }'
config_drive: false
Here the port part is having invalid char '. which is causing the template validate failure. so can you please fix it?
Regarding the requirments mentioned above:
Tacker mainly looking for the IP address of the VM created as part of stack. so can you please add output section. similar to below:
{'mgmt_ip-VDU2': 'CP2', 'mgmt_ip-VDU1': 'CP1'}
Here the VDU and CP are self explanatory. in place of CP, pls provide the IP address of VUDs

Kanagaraj Manickam (kanagaraj-manickam) wrote : | #25 |
@sahdev, it seems now HT broken the alarm policy translation.
pls refer http://
kindly check it once.

Sahdev Zala (spzala) wrote : | #26 |
@Kanagaraj, thanks! sure I will be looking at it today on both issues - ' char and #594279 About output translator can't add one if template doesn't have it, again something we need more time to look into this requirement. Hope you agree. Thanks!

Sahdev Zala (spzala) wrote : | #27 |
@Kanagaraj, can you please provide expected alarm translation for #594279? The scaling translation support was created with simple use case with single policy with or without triggers. This one seems a advance one with multiple policies. Thanks!

Sahdev Zala (spzala) wrote : | #28 |
@Kanagaraj, also if your schedule allows and you can contribute to code in heat-translator that would be very welcome too! The project will be benefited with your expertise!

OpenStack Infra (hudson-openstack) wrote : Related fix proposed to heat-translator (master) | #29 |
Related fix proposed to branch: master
Review: https:/

OpenStack Infra (hudson-openstack) wrote : Related fix merged to heat-translator (master) | #30 |
Reviewed: https:/
Committed: https:/
Submitter: Jenkins
Branch: master
commit 404852064587228
Author: sahdev zala <email address hidden>
Date: Mon Jan 9 16:07:17 2017 -0500
Remove invalid char ' from translated nested autoscaling template
The translation of autoscaling template requires to produce nested templates,
which currently injecting invalid char in some properties.
Change-Id: I60a770694c39cc
Related-Bug: 1652928

Kanagaraj Manickam (kanagaraj-manickam) wrote : | #31 |
@sahdev, thank you for quick fix on removing the invalid char.
I tired to test the http://
can you please plan HT release to support tacker req.
To support your recent version deliverable, i hope below strategy would help:
currently we have HT version in tacker as heat-translator
And if your new version 0.7.0 breaks then we could push one patch to exclude this version.
Thank you for actively supporting.

Sahdev Zala (spzala) wrote : | #32 |
@Kanagaraj, you are welcome and thank you too for flexibility. We will give priority of making needed enhancements to support your Tacker use cases and make another release (0.7.1 or 0.8.0) during the Ocata cycle so that Tacker can consume in Ocata release. The 0.7.0 should be out this week.
On a side, I sent email to you and Sridhar this morning on current limited support for policies in parser and HT with some detail on it of why so. I believe the 0.6.0 of HT doesn't support this use case either, so Tacker should be able to use 0.7.0 with the current limited support (which will be better than 0.6.0 as we now added nested template support and APIs etc) without skipping the planned 0.7.0 release, unless it's needed to to skip it. Thanks again!

Sahdev Zala (spzala) wrote : | #33 |
@Kanagaraj, can you please provide ASAP an expected HOT for http://
Hi Kanagaraj,
What I see is in the template you have provided certain custom types doesn't have a definition associated with it or there is no imported template that provides it. For example scaling type. The following test template that I have created has tosca.policies. tacker. Scaling similar to what you have but with added 'policy_types' section and translator is working fine here,
tosca_definitio ns_version: tosca_simple_ yaml_1_ 0
description: >
Template for deploying servers based on policies.
policy_types: policies. tacker. Scaling: Scaling
tosca.
derived_from: tosca.policies.
topology_template:
properties:
num_ cpus: 2
disk_ size: 10 GB
mem_ size: 512 MB
properties:
architectu re: x86_64
distributi on: RHEL
version: 6.5 tacker. Scaling
description: Simple node autoscaling
resize_ compute:
descriptio n: trigger
condition:
constraint: utilization greater_than 50%
period: 60
evaluations: 1
method: average
min_ instances: 2
max_ instances: 10
default_ instances: 3
increment: 1
node_templates:
my_server_1:
type: tosca.nodes.Compute
capabilities:
host:
os:
# host Operating System image properties
type: Linux
policies:
- asg:
type: tosca.policies.
targets: [my_server_1]
triggers:
properties:
Please take a look and let me know if you have question. Thanks!