NTFS-3G Major Issue: Improper clearing of memory buffer/cache resulting in OOM Kills

Bug #1751675 reported by Ben Roten
6
This bug affects 1 person
Affects Status Importance Assigned to Milestone
ntfs-3g (Ubuntu)
Confirmed
Undecided
Unassigned

Bug Description

I am running a modern gaming rig with Kubuntu 16.10 installed. 16GB ram, 32GB swap. I have an 8TB external that can only be formatted in NTFS (does not behave correctly when formatted any other way). When transferring INDIVIDUAL files to the drive (and reading files), the disk memory cache/buffer as according to the KInfoCenter slowly fills up and is not cleared. After transferring/reading enough INDIVIDUAL files the system runs out of memory and starts OOM killing every app that is currently launched, including the desktop environment after a time (in this case, Plasma). If I use the command

"sudo sync && sudo echo 3 | sudo tee /proc/sys/vm/drop_caches"

when the buffer/cache is nearly full, no crashes occur. SWAP is essentially never used at any point, even if swappiness value is set very high. Relaunching the apps does not clear the memory and typically results in the apps immediately crashing as soon as more memory is needed. Furthermore, I am on kernel version "4.4.0-62-generic" which is necessary in pointing out because a previous kernel version had a bug involving memory and overactive OOM killer.

***However, the distinction for individual files is important because if many files are selected simultaneously and copied over, the memory buffer/cache is cleared systematically as the transfers are completed, preventing any crash. This is useful but still does not prevent the eventual accruing of memory/buffer space when accessing many large files on the drive over time.

As far as I can tell, there is no way to automatically clear the buffer/cache without having a manual script that would have to be run every time. Also worth noting, I have spent sometime trying to nail down the issue and nearly positive that is has nothing to do with any other aspect of how Debian/the kernel manages memory because every memory tweak I have tried has had no effect, short of running the terminal command above to clear the buffer/cache.

It seems that NTFS-3G needs to clear the memory/buffer after transferring individual files or reading them or run a check to clear it after a certain percentage is reached.

Ben Roten (roten-ben)
description: updated
Changed in ntfs-3g (Ubuntu):
status: New → Confirmed
description: updated
To post a comment you must log in.
This report contains Public information  
Everyone can see this information.

Other bug subscribers

Remote bug watches

Bug watches keep track of this bug in other bug trackers.