Mirantis OpenStack

[Sahara] Spark job do not work on Ambari if Swift datasources are used

Bug #1529646 reported by Evgeny Sikachev
6
This bug affects 1 person
Affects Status Importance Assigned to Milestone
Mirantis OpenStack
Fix Released
High
Vitalii Gridnev
Mirantis OpenStack 8.0-updates
8.0.x
Won't Fix
High
Vitalii Gridnev
Mirantis OpenStack 8.0-updates
9.x
Fix Released
High
Vitalii Gridnev
Mirantis OpenStack 9.0

Bug Description

ENVIRONMENT: MOS 8.0 ISO 328

STEPS TO REPRODUCE:
1. Create ambari cluster
2. Run WordCount job using Swift as input datasource from https://github.com/openstack/sahara/blob/master/etc/scenario/sahara-ci/edp.yaml.mako#L72

EXPECTED RESULT:
Job passed

ACTUAL RESULT:
Job has Killed status

See original description
Revision history for this message
Roman Podoliaka (rpodolyaka) wrote :

Sahara team, could you please take a closer look and evaluate the user impact? Feel free to downgrade the importance, if needed.

tags: added: area-sahara
removed: sahara
Revision history for this message
Vitalii Gridnev (vgridnev) wrote :

The problem is that sahara can't correctly handle swift as input or outputs datasource in that job. So, user impact is High and importance is correct right now.

description: updated
Revision history for this message
Fuel Devops McRobotson (fuel-devops-robot) wrote : Fix proposed to openstack/sahara (openstack-ci/fuel-8.0/liberty)

Fix proposed to branch: openstack-ci/fuel-8.0/liberty
Change author: Vitaly Gridnev <email address hidden>
Review: https://review.fuel-infra.org/16034

Revision history for this message
Fuel Devops McRobotson (fuel-devops-robot) wrote : Fix merged to openstack/sahara (openstack-ci/fuel-8.0/liberty)

Reviewed: https://review.fuel-infra.org/16034
Submitter: Pkgs Jenkins <email address hidden>
Branch: openstack-ci/fuel-8.0/liberty

Commit: d6e6939e6b4faac8a7a22a6a62fffad9a1502f3f
Author: Vitaly Gridnev <email address hidden>
Date: Tue Jan 19 12:24:01 2016

Fix issue with Swift on Spark 1.4.1

To solve issues with Swift datasources in Spark
job type we can just put spark.xml inside of
spark wrapper to avoing issues with classpath.

Change-Id: Ic28ccf99af01156897b611c6420bc7e14cc72bc0
Closes-bug: 1529646

Evgeny Sikachev (esikachev)
tags: added: on-verification
Revision history for this message
Evgeny Sikachev (esikachev) wrote : Re: [Sahara] Spark WordCount job do not work on Ambari

reproduced on 529 iso

tags: removed: on-verification
Revision history for this message
Fuel Devops McRobotson (fuel-devops-robot) wrote : Fix proposed to openstack/sahara (openstack-ci/fuel-8.0/liberty)

Fix proposed to branch: openstack-ci/fuel-8.0/liberty
Change author: Vitaly Gridnev <email address hidden>
Review: https://review.fuel-infra.org/16996

Revision history for this message
Vitalii Gridnev (vgridnev) wrote : Re: [Sahara] Spark WordCount job do not work on Ambari

Fix on review: https://review.fuel-infra.org/#/c/16996/

Vitalii Gridnev (vgridnev)
summary: - [Sahara] Spark WordCount job do not work on Ambari
+ [Sahara] Spark job do not work on Ambari if Swift datasources are used
Vitalii Gridnev (vgridnev)
tags: added: release-notes
Olga Gusarenko (ogusarenko)
tags: added: 8.0 release-notes-done
removed: release-notes
Revision history for this message
Sergey Reshetnyak (sreshetniak) wrote :

Fixed in upstream.

Revision history for this message
Fuel Devops McRobotson (fuel-devops-robot) wrote : Fix proposed to openstack/sahara (9.0/mitaka)

Fix proposed to branch: 9.0/mitaka
Change author: Vitaly Gridnev <email address hidden>
Review: https://review.fuel-infra.org/18607

Revision history for this message
Fuel Devops McRobotson (fuel-devops-robot) wrote : Change abandoned on openstack/sahara (9.0/mitaka)

Change abandoned by Vitaly Gridnev <email address hidden> on branch: 9.0/mitaka
Review: https://review.fuel-infra.org/18607
Reason: Fixed in upstream

Revision history for this message
Evgeny Sikachev (esikachev) wrote :

verified on 363 iso, mos 9.0

Revision history for this message
Fuel Devops McRobotson (fuel-devops-robot) wrote : Change abandoned on openstack/sahara (openstack-ci/fuel-8.0/liberty)

Change abandoned by Vitaly Gridnev <email address hidden> on branch: openstack-ci/fuel-8.0/liberty
Review: https://review.fuel-infra.org/16996

To post a comment you must log in.
This report contains Public information  
Everyone can see this information.
Subscribing...

Other bug subscribers

Remote bug watches

Bug watches keep track of this bug in other bug trackers.