Memory leak in Nautilus/Kernel?

Bug #734552 reported by Jim (JR) Harris
6
This bug affects 1 person
Affects Status Importance Assigned to Milestone
Linux
New
Undecided
Unassigned
nautilus (Ubuntu)
Expired
Medium
Unassigned

Bug Description

Yesterday when moving relatively huge amounts of data between hard drives on my system (using the GUI - drag-and-drop) I discovered that after some time I had consumed all available memory and all available swap.

System:
Athlon 64 X2 processor on ASUS A8R32-MVP Deluxe MoBo.
4 gig RAM
1.64 gigs Swap
Ubuntu 10.04 (64 bit) "Desktop" edition.

System configuration:
40 gig IDE mounted as root ( / )
4x 1T SATA mounted as md0 - RAID 5 on /mnt/Storage3/Public
1x 2T eSATA mouonted on /mnt/Storage3/Public2
1x 2T eSATA mounted on /mnt/Backup
1x 2T eSATA mounted on /mnt/Backup2

Sustem is being used as a NAS file server running Samba, Apache, and SWAT

System is fully up-to-date according to Update Manager, with all updates to whatever it wanted to update, kernel, firefox, and a whole laundry-list of other things. . . .

Running "gadgets":
1. HW-monitor running libsensors (monitoring CPU temp, both core temps, CPU and chassis fan speeds, and four voltages - vCore, 3.3v supply, 5v supply and 12v supply)
2. "Weather" gadget.

Desktop configuration:
Running the "Clear" (?) Desktop theme (the "Windows" looking desktop)
I have not tried this using any other desktop configuration.

Possibly relevant?
During a moment of absolutely culpable stupidity, I inadvertantly deleted all the non-dot folders and files from my home directory. Not knowing any clever way to rebuild my home directory, I rebooted into the 10.04 live CD (64 bit), and copied the "Ubuntu" users home directory (the non-dot folders) back to my home directory, changed permissions and owner to match my user, and rebooted. Prior to this copy, I verified that none of the hidden (dot) files or folders had ben touched. I do not believe that this would have any bearing on this, but I mention it just in case.

Scenerio:

My 4T RAID-5 file store was becoming fuller than I could back-up so I decided to re-distribute the files by moving certain directories to Public2 (an additional drive I just added) and replacing them (on Public) with symlinks.

The magnitude of the move was in the order of almost two teryabytes of data, (appx. 1.5-or-so T), to distribute the data more evenly between the two drives.

About half the data was moved using "cp" on the command line. Parts of the data that had non-standard filenames (containing spaces, etc.) caused difficulties moving it with cp, so I used the "file browser" as root (gksu "nautilus. . .") because some of the directories had varying permissions set.

The porocess was as follows:
1. Drag directory from source to destination.
2. Wait for the copy to successfully finish.
3. Delete the original direcotry.
4. Create a symlink (drag with ctl-shift) from the new location back to the original location.
This was done because only the Public directory is shared via Samba, and I wanted to keep it that way.

In some cases, I moved multiple directories at the same time - some being very small (in the megabytes range) and some were relatively large (in the multiples of gigabytes range)

This went on for several hours.

Eventually my system virtually ground to a halt. Attempting to launch any new GUI based process failed with "unable to fork - could not allocate memory" (or something like that).

I eventually got "top" running in a terminal, and discovered that all available RAM and all available swap had been consumed.

I was eventually able to get the computer to shutdown, pause to catch it's breath, and restart.

Performing some directory copies after the re-boot, with "system monitor" running on the desktop, disclosed the following:

1. Under normal operating condiditons, the system is using appx. 450 (plus or minus) megs of RAM and zero swap.

2. Copying of any amount of data using the command-line within a terminal does not affect the system memory used.
There are variations in memory used - but the magnitudes are less than about 50 or so megs - it never gets as high as 500 megs used. Once the process is finished, the memory used returns to very nearly the base value - eventually dropping back to it after a moment or two.

(Update: This actually depends on the amount of data being moved, and the amount of terminal activity - I have subsequently seeen it rise into the 600+megs region, but it always recovers.)

3. If I copy data using the GUI and drag and drop between different file-manager windows - the used memory rises to about 570 (plus or minus) megs used, and when the process finishes, it releases only about 20 or 30 megs - if that much. Additional directory moves cause the memory usage to continue to climb by about 100 megs per copy. Note that the majority of the copies were multi-gigabyte copies that could take from 10 minutes to an hour to complete.

Rebooting the machine always restores memory - as we would expect.

Top shows Nautilus consuming increasing amounts of memory every copy.

I looked all over the web checking for a solution, and I even tried the "apt-get remove appmenu-gtk" fix - apt-get reported that I didn't have the package installed.

I used system monitor to look at running processes, and nothing looked fishy to me. (i.e. I didn't have 30 instances of nautilus running, or other strange things like that.)

I did not try logging off and back on again.

Due to the somewhat "mission critical" nature of this file-store, my ability to experiment with it to try various things may be severely limited.

Any assistance would be gratefuly appreciated.

Jim

Follow-up information as requested:

@ actionparsnip

uname -a:
Linux Storage3 2.6.32-29-generic #58-Ubuntu SMP Fri Feb 11 20:52:10 UTC 2011 x86_64 GNU/Linux

lsb_release -a:
Distributor ID: Ubuntu
Description: Ubuntu 10.04.2 LTS
Release: 10.04
Codename: lucid

echo:
(returned no data)

apt-cache policy nautilus:
nautilus:
  Installed: 1:2.30.1-0ubuntu1.1
  Candidate: 1:2.30.1-0ubuntu1.1
  Version table:
 *** 1:2.30.1-0ubuntu1.1 0
        500 http://us.archive.ubuntu.com/ubuntu/ lucid-updates/main Packages
        100 /var/lib/dpkg/status
     1:2.30.0-0ubuntu4 0
        500 http://us.archive.ubuntu.com/ubuntu/ lucid/main Packages

Tags: leak memory
Revision history for this message
Jim (JR) Harris (jimrh) wrote :

All work described above was done at the "console" (direclty logged into the machine - not from a remote session or Samba)

Jim (JR) Harris (jimrh)
tags: added: leak memory
Revision history for this message
Jim (JR) Harris (jimrh) wrote :

Ref: question #148866 in Launchpad.

I have taken the (admittedly great) liberty of inviting the Kernel team to weigh-in on this since I am beginning to suspect that this may actually be a kernel issue.

Summary:
When moving huge files - hundred+gig to terabyte sized file groups - eventually all available RAM and then all available swap is consumed.

Running huge file/multi-file copies or moves either from Nautilus (drag-and-drop) or from within a terminal window causes large amounts of memory to be consumed that is not released when the copy is done. Subsequent file copies/moves simply add to the memory consumed.

Multiple, simultaneous huge file/group copies simply pour gasoline [petrol] on a raging fire. This has been a problem for me since at least Feb. 22 - 2010 - probably before - and has cost me massive crashes on my file server on multiple occasions.

To duplicate this, you will need massive amounts of storage - in the multiple T range - and it will be necessary to create folders full of multi-gig files in the 200G to fractional-T size to move around. (e.g. If I have two 2T drives, one that has abut 1.5+ T of data on it, and the other empty, and I run a "cp -a -v /mnt/huge_drive/* /mnt/huge_drive_2" - and watch the memory allocation either in SysMon, or via top, you will see memory climb steadily (top shows increasing amounts of buffer used).

If you are really sadistic, you can repeat the command again once it finishes. My box has 4 gig RAM, and almost 2 gig swap, and doing this a few times rapidly runs all available memory and swap - and then some! - right down the tubes.

Please review question #148866 in launchpad: https://answers.launchpad.net/ubuntu/+source/nautilus/+question/148866 for additional, possibly relevant, info on this issue.
.

Revision history for this message
Jim (JR) Harris (jimrh) wrote :

For me this is extremely important, bordering on critical, as I frequently need to move huge amounts of data onto, around within, and off of my multi-T file server based on the needs of the client-/du-jour/.

Thanks!

summary: - Memory leak in Nautilus?
+ Memory leak in Nautilus/Kernel?
Revision history for this message
Pedro Villavicencio (pedro) wrote :

Thank you for taking the time to report this bug and helping to make Ubuntu better. Please try to obtain a valgrind log following the instructions at https://wiki.ubuntu.com/Valgrind and attach the file to the bug report. This will greatly help us in tracking down your problem.

Changed in nautilus (Ubuntu):
importance: Undecided → Medium
status: New → Incomplete
Revision history for this message
Pedro Villavicencio (pedro) wrote :

We're closing this bug since it is has been some time with no response from the original reporter. However, if the issue still exists please feel free to reopen with the requested information. Also, if you could, please test against the latest development version of Ubuntu, since this confirms the bug is one we may be able to pass upstream for help.

Changed in nautilus (Ubuntu):
status: Incomplete → Expired
To post a comment you must log in.
This report contains Public information  
Everyone can see this information.

Other bug subscribers

Related questions

Remote bug watches

Bug watches keep track of this bug in other bug trackers.