Percona XtraBackup moved to https://jira.percona.com/projects/PXB

Backup of non-innodb files super slow over NFS

Bug #1197843 reported by Yves Trudeau on 2013-07-04

This bug affects 1 person

	Status	Importance	Assigned to	Milestone
Percona XtraBackup moved to https://jira.percona.com/projects/PXB	Triaged	Medium	Unassigned	Percona XtraBackup moved to https://jira.percona.com/projects/PXB future-2.3-releases
2.0	Won't Fix	Undecided	Unassigned
2.1	Triaged	Medium	Unassigned	Percona XtraBackup moved to https://jira.percona.com/projects/PXB future-2.1-releases
2.2	Triaged	Medium	Unassigned	Percona XtraBackup moved to https://jira.percona.com/projects/PXB future-2.2-releases
2.3	Triaged	Medium	Unassigned	Percona XtraBackup moved to https://jira.percona.com/projects/PXB future-2.3-releases

Bug Description

For a customer with about 16k non innodb files (mostly .frm), the copy of the files to the NFS destination takes more than 20min causing issue with replication because of the lock table. The --rsync option changes nothing to this since it is the stating of the files over NFS that is long. I wrote a patch reusing the rsync logic, doing a local tar archive of the files during the table locks and than a further restore to the destination once the lock is released. It adds the --tar option. The option is good only when there's no large MyISAM (or other non-innodb) tables since that would require a lot of tmpdir space.

With the patch and the --tar option the lock time was reduced from 20+ min to 9s.

Regards,

Yves

Revision history for this message

Yves Trudeau (y-trudeau) wrote on 2013-07-04:

innobackupex-tar.patch Edit (17.4 KiB, text/plain)

Revision history for this message

Alexey Kopytov (akopytov) wrote on 2013-07-05:

Yves,

I wonder if the same could be achieved with this blueprint: https://blueprints.launchpad.net/percona-xtrabackup/+spec/rsync-with-stream-option

I.e. we can implement that with a --rsync-tmpdir option, which instead of rsyncing to the final location would first copy files to the specified temporary directory, and then streaming/copying to the final location after unlocking the server. The requirement in that blueprint is to make rsync usable for streaming backups, but it sounds like it may be useful in your case with just a minor tweak to specification.

What do you think?