tags.xml is sometimes deleted

Hello MCast, thanks for reporting a bug, but I'm afraid that without a little context there is not much we can do.

What were you doing when that happened (GTG crashed, X crashed ...)?

I'm afraid I got the same bug... sadly for me I lost the file and I have to create an empty tags.xml with two lines, and lose all the color tags and subtags... :(. After that, I recovered all my tasks with their respective tag, but as I was saying - no color nor sub-tags.

I don't know what happened, I think it was a X crash, but I don't know hot to debug it (the only thing I know that it was today).

It wouldn't be a bad idea to backup the file the same way the xml for the tasks is backed up.

On Thu, May 13, 2010 at 09:22:23PM -0000, Pablo Olmos de Aguilera Corradini wrote:
> I'm afraid I got the same bug... sadly for me I lost the file and I have
> to create an empty tags.xml with two lines, and lose all the color tags
> and subtags... :(. After that, I recovered all my tasks with their
> respective tag, but as I was saying - no color nor sub-tags.
>
> I don't know what happened, I think it was a X crash, but I don't know
> hot to debug it (the only thing I know that it was today).
>
> It wouldn't be a bad idea to backup the file the same way the xml for
> the tasks is backed up.

I have seen this same bug multiple times myself. I haven't reported it
before because I've never been able to identify the specific steps to
reproduce it. It seems to happen when there is a crash while gtg is in
the middle of saving stuff or something. I've taken to doing regular
backups, so when it happens and tags.xml is 0, I can restore that file
from the most recent backup.

I agree it would be most appropriate for gtg to keep a backup of this
file like it does for tasks.

Bryce

Indeed, we should have an automated backup solutions for tag.xml. If the tag.xml cannot be loaded, then the backup should be used.

Maybe there's also some point in the code where we keep the file opened without reason ?

There's already a backup code available in tools/cleanxml.py

I think we could also investigate why the file is empty. Maybe it would be best
to save a temporary file and then "mv" the file to the actual location.
This problem has been already dealt with with most text editing applications
(I think vim does the mv - or did, I'm not sure -). There are articles around
the web on this matter.
Example:
http://bitworking.org/news/390/text-editor-saving-routines
On Fri, May 14, 2010 at 12:49:13PM -0000, Lionel Dricot wrote:
> Indeed, we should have an automated backup solutions for tag.xml. If the
> tag.xml cannot be loaded, then the backup should be used.
>
> Maybe there's also some point in the code where we keep the file opened
> without reason ?
>
>
> There's already a backup code available in tools/cleanxml.py
>
> ** Summary changed:
>
> - Program crashes on start (tags.xml size is 0)
> + tags.xml is sometimes deleted
>
> ** Changed in: gtg
> Status: Incomplete => Confirmed
>
> ** Changed in: gtg
> Importance: Undecided => Critical
>
> --
> tags.xml is sometimes deleted
> https://bugs.launchpad.net/bugs/579189
> You received this bug notification because you are subscribed to Getting
> Things GNOME!.
>
> Status in Getting Things GNOME!: Confirmed
>
> Bug description:
> Getting Things Gnome! 0.2.4
> LinuxMint 7.
>
> yuriy@ynix ~ $ /home/yuriy/YuriyInstalledSoft/gtg-0.2.4/gtg
> Error parsing XML file /home/yuriy/.local/share/gtg/tags.xml: no element found: line 1, column 0
>
> The size of tags.xml file is 0. It is empty.
> Fortunately I have had backup of my config files and restored tags.xml.
>
>

Today when I started gtg I found myself with a similar error, but now referring to the "tasks.xml":

Error parsing XML file /home/arch/.local/share/gtg/c9fdb5af-b37a-4bc8-8811-7b173279e3ac.xml: no element found: line 1, column 0

I checked the .local/share/gtg to found a c9fdb5af-b37a-4bc8-8811-7b173279e3ac.xml empty, and a c9fdb5af-b37a-4bc8-8811-7b173279e3ac.xml (studio's conflicted copy 2010-05-15) and a tags.xml (studio's conflicted copy 2010-05-15).

I replaced them and everything is there, it is somehow related?

Don't pay attention to my last comment... it was Dropbox that was causing the problem creating thos "studio's conflicted copy".

Changes:

1. Now, instead of just rewriting main XML file, we rename main file to temp one and than write to main.
2. Instead of using f.write(), we use os.write(). This enables us to check if writing was successful or not.
3. When reading, if something is wrong to main file, we will use temp one (in case it exists).
4. When reading file, we open it as read only and close it after reading.

What was tested:

1. Simple stuff (adding new tasks, deleting, adding new tags)
2. Copying main file to temp file and than removing main file (emulating error behaviour).
3. Copying main file to temp file and than breaking main file (emulating error behaviour).

That's great!

A couple of things:
- please add a check that the tags.xml file exists.
  A new installation of GTG starts without the tmp/ folder (which contains the user data if you start gtg with ./scripts/debug)
- add yourself to GTG/info.py, AUTHORS and update the CHANGELOG (I always forget to do that, but we should do it)
- create a branch (bzr push lp:~gtg-user/gtg/NAMEOFFANTASY or bzr push lp:~mkevac/gtg/NAMEOFFANTASY)
- open the branch in your browser (either manually or with bzr lp-open)
- ask for a merge request in lp:gtg (this way, your name gets in our repository and you can use stuff like ohloh.com

Welcome between GTG contributors!

New version of patch.

In this version I have fixed issues with creating new (empty) files.

Testing:

cd trunk/
rm -rf tmp/
./scripts/debug.sh

Everything works.

In this patch I have also included changes to info.py, AUTHORS and CHANGELOG files. These changes are in separate commit from bugfix.

I have created branch in launchpad.

I will try to contact you for merging.

I've experienced the following error twice in less than a week. In both cases I noticed it after realizing the notification icon was missing, so I suppose it crashed when trying to write a new task.

It sounds similar to what's described here, but I'm not sure the patch attached here should fix the problem I'm seeing as well? If somebody think it does I'm happy to apply the patch manually and give it a try. Otherwise, should I file a separate bug?

salgado@feioso [gtg]$ gtg -d
2010-10-07 10:41:20,534 - DEBUG - gtg:main:105 - Debug output enabled.
Error parsing XML file /home/salgado/.local/share/gtg/a4eb91e8-50a3-46e5-a712-a7338031b88e.xml: no element found: line 505, column 2