Abysmal memory leak on tracker-extract

Thank you for your bug report. Could you add your journalctl log to the bug?

I have the exact same issue as gert7. Tracker-extract fills my entire memory until kernel kills the process, after which tracker-extract gets started again with the same situation after a few seconds. The only way to make the system usable again is to forcibly kill all tracker processes.

While clogging up the memory, tracker-extract also gives some errors to journalctl like this, on two specific .dds files.

>Apr 23 17:04:27 apfelgriebs-desktop tracker-extract[3527]: Call to gst_discoverer_discover_uri(file:///[...]/[...].dds) failed: Internal data stream error.

could you add one of those files to the bug? does it stop doing that if you remove those?

Here you go. Those are texture files from a mod for the game "Crusader Kings 2".

>does it stop doing that if you remove those?

Can I exclude the folder from being indexed by tracker? I'll try that (or removing the files) tomorrow.

I have the same game installed as well.

After removing the folder with those .dds files, the memory leak of tracker-extract doesn't occur.

However IMO this can't be regarded as anything as a temporary workaround.

Status changed to 'Confirmed' because the bug affects multiple users.

The issue is an upstream one, could someone report it on https://gitlab.gnome.org/GNOME/tracker/issues ?

Thanks for providing the 'minimap_area.dds' file. How many other files are there in the same directory? And what is the total size of all of these .dds files?

Running `tracker extract` on the given gives the following message:

Call to gst_discoverer_discover_uri(file:///.../minimap_area.dds) failed: Internal data stream error.
file:///.../minimap_area.dds: No metadata or extractor modules found to handle this file

I think this is pretty normal, that GStreamer's type detection code sometimes reports an error when it fails to detect the type of a file. Running with `env GST_DEBUG=2` doesn't show any unexpected messages. Running under `valgrind` doesn't show any dramatic memory leak.

It's possible that this code path blows up on some larger files that are in the same directory, though. Could you run `tracker extract *` inside the problematic directory and see what happens? (This will run in the foreground so you should be able to CTRL+c if it starts to eat all your RAM ... which it hopefully will do so we can narrow down the problem :).

In the directory there are 10666 files, total size 735,5 MB. There are 723 .dds files with a total size of 248.2 MB

>Could you run `tracker extract *` inside the problematic directory and see what happens

There are various sub-directories with .dds files, running tracker inside them gives very similar outputs. Two outputs here https://pastebin.com/NTzha2Ww and here https://pastebin.com/MEiTzZPq

It however doesn't loop forever, it just gives one error message for every single dds file and then exists properly.

The bug is still present in Ubuntu 19.10.

I confirm the bug on 19.10 for DDS files ( from a mod for HOI4)

I get the same error as Alex, but for TGA files:

```
tracker-extract[13828]: Call to gst_discoverer_discover_uri(file://[...]/Ale_col.tga) failed: Internal data stream error.
```

The error only occurs with certain TGAs, I shall attaching one below.

My current workaround is to add '*.tga' to "/org/freedesktop/tracker/miner/files/ignored-files" in `dconf-editor`.

SYSTEM:
```
#lsb_release -rd
Description: Ubuntu 20.04 LTS
Release: 20.04

#apt-cache policy tracker*
tracker-extract:
  Installed: 2.3.3-2
  Candidate: 2.3.3-2

tracker-miner-fs:
  Installed: 2.3.3-2
  Candidate: 2.3.3-2

tracker:
  Installed: 2.3.4-1
  Candidate: 2.3.4-1
```