sahara job binary copy times out when files is too big
Bug #1705762 reported by
Telles Mota Vidal Nóbrega
This bug affects 1 person
| Affects | Status | Importance | Assigned to | Milestone | |
|---|---|---|---|---|---|
| Sahara |
In Progress
|
Medium
|
Telles Mota Vidal Nóbrega | ||
Bug Description
Sahara times out during the copy of job binary into the cluster if the binary is too big. The size of file tested was around 115MB and it took around 15 minutes to copy.
The copy is done using paramiko sftp on the sahara/
def _write_fl(sftp, remote_file, data):
fl = sftp.file(
fl.write(data)
fl.close()
A little research says that increasing transfer window size could be helpful but we need deeper investigation on it.
Another possible solution is change sftp write for sftp put.
| Changed in sahara: | |
| assignee: | nobody → Telles Mota Vidal Nóbrega (tellesmvn) |
| importance: | Undecided → Medium |
Revision history for this message
| Jeremy Freudberg (jfreud) wrote | #1 |
| Changed in sahara: | |
| status: | New → Triaged |
Revision history for this message
| OpenStack Infra (hudson-openstack) wrote Fix proposed to sahara (master) | #2 |
| Changed in sahara: | |
| status: | Triaged → In Progress |
Revision history for this message
| Telles Mota Vidal Nóbrega (tellesmvn) wrote | #3 |
To post a comment you must log in.
Another possible solution is moving job binary retrieving to be inside the cluster itself instead of inside Sahara.