Backup copy of project.

Is this still the case on recent hugin versions?

YES. Hugin-recent crashed on me again yesterday.

When stitching and a crash happens only hugin_stitch_project (or some of the external tools) can crashs. Hugin itself is an own process, so there is no data loss, because the project file is still open.
The second part would be an auto save option. This would be a now option, therefore set importance to wishlist.

Dear Thomas,

I was working on a project. Instead of putting my images around the circle hugin placed all images at 0,0,0. That is not how I shot things. So I went to the "preview" window, and tried dragging the images, so that they wouldn't be on top of eachother. That's when HUGIN ITSELF CRASHED.
IT TOOK ALL MANUAL CONTROLPOINT EDITS WITH IT. NO SAVING POSSIBLE. PROCESS GONE. DATA GONE.

Your assumption that it happened when stitching or that no dataloss happened is WRONG.

This IS a real data-loss problem. I'm FINE with the developers not getting around it before christmas. But PLEASE do not put this as "wishlist" again. It is a significant problem.

This, saving the intermediate hugin file every now and then, is a surefire way of diminishing the consequences of an unexpected crash. If we take this seriously, a day of grudgework will reduce the severity of an unexpected crash to "bad luck, just restore from the autosave". Otherwise all crashes HAVE to be considered a showstopper.

FYI, I just tried recreating the "crash" but I can't So I can't file a useful bugreport on the crash yet. But I do know that Hugin lost some of my data this week. And this was with a recent HG pull. Pre-Release 2010.3.0.ee13737cebbb

Dear Rogier,

I read again the whole bug report thread and I must say that I understand why Thomas misunderstood you. I would have too. Maybe it would have been politically safer to set the bug to incomplete and ask for clarification.

First, the initial report was from 0.7.0. Hugin has changed massively since. Thomas was not even around back then.

Second, I read in the initial report: "I'm most pissed that a pto file got saved to run "nona", but this gets (I think) cleaned up after running nona." Hugin does not save a pto file to run nona until the stitching process is started. Hence Thomas' assumption (and I would have assumed the same) that this happened while stitching, and that this was in hugin_stitch_project.

Now, one hour ago, comes more information. I agree with you that based on the new information this is more serious. However there are two (or three) parts to this ticket:
1. identify and prevent the crash in Hugin.
2. linder the consequences of hugin_stitch_project crashing by enabling the resuming of the stitch process with the makefile (Thomas' assumption)
3. add backup / auto-save functionality to Hugin (your feature request that would have prevented the crash from causing you serious loss)

1. would need to be "confirmed" and set to status "important".
2. & 3. would be "wishlist". Not because they are not important, but because strictly speaking they are new features and not fixes. At the moment we are trying to sort out through the large quantity of tickets outstanding from the migration and the distinction wishlist - not wishlist is to mark what is a feature request (adding new code) from what is a bug (error in existing code).

Please bear with us as we set these flags. They are not an expression of judgment on the importance of your ticket. They are just a way of categorizing the work that needs to be done and getting a clear overview of how much in the tracker is about existing features that need fixing and how much is about new features.

If we drop all bug reports that are old this whole migration of bugs could've been prevented.

Although I agree that formally "wishlist" is what you'd call a new feature like "autosave", it should be considered an essential fix to the system as long as sudden crashes still linger around.

A (hypothetical) bugfix for say: 'it crashes when I click on the 31st image in the preview' can be solved by saying "don't do that then". And "not crashing when clicking on the 31st image in the preview" is a wishlist item that someone would like to see implemented.

I mentioned "nona" and the writing of the PTO file because it probably didn't crash until after I had once made a test-stitch. But in general I've had it crash before I've made my test-stitch as well.

Patch attached.

TODO: change the interval to say 120 seconds.
TODO: Add a preference for the autosave interval. (but 120 will do for now IMHO).
TODO: The file currently lands in the CWD of hugin: /tmp . I think would prefer: current dir when hugin was started. Or user's homedir.
TODO: Think about: should we keep the debug output: "autosaving.... done"?

TODO: Think if we should delete the file on normal exit? I vote: no.

(user's homedir is not a nice place: I sometimes work on different Hugin projects at the same time (doing CPU intensive stuff with one, and doing manual-adjustments on the other), so "CWD" would be better than "user's homedir". )

On November 30, 2010 01:58:32 am rew wrote:
> If we drop all bug reports that are old this whole migration of bugs
> could've been prevented.

Would you volunteer to go through 400 tickets (bugs/features/patches) on the
old/slow SF infrastructure?

> Although I agree that formally "wishlist" is what you'd call a new
> feature like "autosave", it should be considered an essential fix to the
> system as long as sudden crashes still linger around.

I agree that it is an effective workaround. Not a fix. I prefer to see the
sudden crashes fixed and Hugin becoming more robust.

> A (hypothetical) bugfix ...

Not fix. Work around.

> I mentioned "nona" and the writing of the PTO file because it probably
> didn't crash until after I had once made a test-stitch. But in general
> I've had it crash before I've made my test-stitch as well.

The misunderstanding was all on our side. I apologize if it made you feel
that we do not consider your reports important enough.

To be totally explicit: I think the autosave feature request is a great idea
and should be on top of the list. But which list?

At SF we had three lists (bugs/patches/features). Bugs became irrelevant
because submerged by new reports. Features became irrelevant because ignored.
Patches was more or less under control, mainly because there were not that
many submissions.

Now we have a big combined list (around 400 tickets open at time of
migration), and an opportunity to do things well, i.e.
- first sort out what is feature request from what is bug report (there were a
lot of feature requests in the bug tracker)
- then go through them and prioritize them into a single "todo" list.

using the importance:
- feature requests -> wishlisth
- bug reports -> low/medium/high

then they can be mentally merged to a single todo-list again, trying to
balance between making the current software work as advertised (bug fixes) and
adding new features.

once we have sorted this mess, we can go more methodical about the todo-list
by using the Assigned-To and Milestone fileds.

Thank you for bearing with us and helping us sort the backlog
Yuv

Oh, I just noticed the patch. THANK YOU VERY MUCH. I put all reports that have patches on Critical status, simply because they are low hanging fruits worth harvesting. Will check into this and hopefully integrate it into 2010.4beta2

Problem: adding a preference means adding a string. Not a nice thing to do to translators at this point in the release cycle. I'll ask the question on Hugin-PTX

The patch needs some more polishing (I had no time to compile, but some comments from have a look on it):
* The timer is not correctly freed at the end.
* Don't use local variables beginning with m_ (these should only used for member variables to get a better overview)
* Status reports should go to stdout and not to stderr (and no blabla, what should this?? bad style). Also fprintf can make problems with Windows. There are better alternatives: e.g. using cout, DEBUG_TRACE,
* The 2 conversion of the filename should not be necessary.
* It needs to be checked if the path in the project file is correct (the cwd can be an other directory than the dir with the image files, e.g. user navigated to lens or mask files.)
* Don't autosave if the project does not contain any image.
* Maybe the EVT_TIMER macro would be better suit to the existing code than using the eventhandler.

also
* using a fixed filename will cause conflicts if two instances of Hugin are open at the same time

other than that the patch builds here and does what it is expected to do.

also
* when using 2 project file in the same directory results in overriding the backup of the other.
* The timer needs to be reseted when starting a new project, load project or save or project.

Concerning the issues to be fixed, it should not go into 2010.4 (because of the short time table). When the main issues are fixed then it can go into default branch.

I found an instant crash in the beta: Bug #678890

I have a better patch that fixes most of the "problems".

I can't decide on a proper autosave place such that running multiple hugin instances works.

Share the patch please. Can't promise (lot of things on my todo list), but if it still does not have a preference setting I can contribute that.

Place = storage space in the file system? how about taking the time when the Hugin instance is started, make it into an epoch, and save it to something like /etc/${epoch}.pto ?

On Tue, Dec 07, 2010 at 12:47:47AM -0000, Yuv wrote:
> Share the patch please. Can't promise (lot of things on my todo list),
> but if it still does not have a preference setting I can contribute
> that.
>
> Place = storage space in the file system? how about taking the time when
> the Hugin instance is started, make it into an epoch, and save it to
> something like /etc/${epoch}.pto ?

/etc is out of the question.

$HOME/.hugin-autosave.$$.pto

($$ = the PID of the running process)

would work fine for me. The question is: who cleans this up?

How about:

$HOME/.hugin-autosave.nn.pto

where nn is a number from 1 to ten. This way you have 10 backups of at
most ten running hugin processes. You don't have to delete them if you
quit. (Sometimes having a backup from a previous session is useful!)

Thus at startup, you'd
 rename $HOME/.hugin-autosave.9.pto $HOME/.hugin-autosave.10.pto
 rename $HOME/.hugin-autosave.8.pto $HOME/.hugin-autosave.9.pto
 rename $HOME/.hugin-autosave.7.pto $HOME/.hugin-autosave.8.pto
 rename $HOME/.hugin-autosave.6.pto $HOME/.hugin-autosave.7.pto
 rename $HOME/.hugin-autosave.5.pto $HOME/.hugin-autosave.6.pto
etc.
and then start saving in

 $HOME/.hugin-autosave.1.pto

but we'd need to keep the filedescriptor to this file so that we'd
keep on saving to the right file even as our autosave file is moved to
a new name by a new instance....

 Rogier.

--
** <email address hidden> ** http://www.BitWizard.nl/ ** +31-15-2600998 **
** Delftechpark 26 2628 XH Delft, The Netherlands. KVK: 27239233 **
*-- BitWizard writes Linux device drivers for any device you may have! --*
Q: It doesn't work. A: Look buddy, doesn't work is an ambiguous statement.
Does it sit on the couch all day? Is it unemployed? Please be specific!
Define 'it' and what it isn't doing. --------- Adapted from lxrbot FAQ

On Tue, Dec 07, 2010 at 12:47:47AM -0000, Yuv wrote:
> Share the patch please. Can't promise (lot of things on my todo list),
> but if it still does not have a preference setting I can contribute
> that.
>
> Place = storage space in the file system? how about taking the time when
> the Hugin instance is started, make it into an epoch, and save it to
> something like /etc/${epoch}.pto ?

Oh.. .I think I checked it in to my local mercurial storage, but how do
I get the patch out?

Last time "hg diff" gave me exactly what I wanted, but now that I've
checked things in it seems to have become more difficult.

 Roger.

--
** <email address hidden> ** http://www.BitWizard.nl/ ** +31-15-2600998 **
** Delftechpark 26 2628 XH Delft, The Netherlands. KVK: 27239233 **
*-- BitWizard writes Linux device drivers for any device you may have! --*
Q: It doesn't work. A: Look buddy, doesn't work is an ambiguous statement.
Does it sit on the couch all day? Is it unemployed? Please be specific!
Define 'it' and what it isn't doing. --------- Adapted from lxrbot FAQ

The filename are all very unixy and does not look good on windows.

Furthermore, if you hold a handle to a file you can not rename the same file in a second instance. So this will break.
Also this will show unpredictable behaviour if you open more than 10 instances (unlikely, but possible).

Also it would be nice to have a clean up routine and to have the possiblity to deactivate this function.

use hg export to produce the patch. hg help export is your friend.

for example, if the code was committed in one go, then hg export -g ${REV} > feature.patch will produce a patch. If it was committed in multiple steps, then hg export -g ${STARTREV}:${ENDREV} > feature.patch will do the trick. You will need to identify the revision number at which you did commit. hg view is your friend there. you will need to activate the hgk extension in your ~/.hgrc settings.

There is a place in the code where a random temporary file is generated for stitching. I don't recall anymore where it is. You could orient yourself to that for both place and filename - AFAIK it works well on all systems.

Thomas, I'm a Unix-guy. Apparently you're not. I have to depend on you to make suggestions as to usable names on Windows, and on where to look in the source code on how to do it. Please make a suggestion, besides telling me that "$HOME/...." sounds unix-y. I already know that.

As I see it, having 10 backup project files around is acceptable. Deleting them when hugin exits succesfully negates part of the "having a backup". (We of course got into this because "hugin crashes", but a backup can also protect against: "I had it linedup just fine yesterday, and now I can't get it to line up correctly! aargh!". For this last situation, removing the file on succesful exit is not desirable).

Using a "random name" means we can't leave them lying around, because there might end up lots of them.

On Unix, if you open a file, and then rename (or delete) the file, it will remain linked to the file you originally opened.

I find "the backup of the first instance of hugin disappears if I open 11 instances of hugin" an acceptable compromise between disk space and usability. (you'll have up to five backups if you don't open more than two hugin instances at the same time). A preference setting may be implemented for this as well. 10 is not a magic number.

the electronics design software "eagle" keeps old versions of your design around as backups with a number. Every time you save, it moves the previous versions up one number. This way you have up to 10 backups of your project should something go wrong.

How about: We save under the "normal" name, with: '.0" attached for the autosave. When an explicit save comes about, we remove .9, move .8 to .9, .7 to .8 etc, and then save under the given name. (before naming your project, the autosave will be in "unnamed.pto.0")

Current patch attached. I didn't manage to get the start-rev right, so I manually removed the other stuff. On the other hand, I hadn't tried "hg view" yet, which I have yet to learn how to use. :-)

Just like with Email I manage to hit "send" before attaching the attachment.

> On Unix, if you open a file, and then rename (or delete) the file,
> it will remain linked to the file you originally opened.

Are you sure that you can delete a file during you are working on it in an other program? This does definitely not work on windows, as I wrote already.

It seems you are mixing 2 issues/features: One is auto save to prevent data loss when hugin crashes. The second one is to provide a history or backup function.

The patch implements an autosave, because only the last project is saved. (Even if you are working on different projects in one session, only the last one is finally kept on disc. The older autosaves are overwritten.) But it needs more fixing, currently the patch does not work with 2 instances. Also the time intervall is too short. It saves every 10 seconds, which create too much disc activity for me. This is too short and can not changed by the user (or even deactivated).

Your proposed name schemes in the comments are more dealing with history/backup.

Thomas, you have to learn to distinguish between a work-in-progress patch and a final "suggested fix" patch.

Yes, I'm sure that on Unix you can keep an open filedescriptor to a file and rename it. The common way to prevent temporary files from cluttering the filesystem is to open the temporary file, and then delete it. Now when the program exits (clean or otherwise) it is the OS'es responsibility to clean up the file now that there are no longer any references to it.

Yes, now that we're talking autosaves, we can think about backups too. Maybe we can solve two issues in one go.

Yes, my patch doesn't yet implement any of my suggested backup ideas. The patch proves that making hugin do an autosave can be done in less than 20 lines-of-code. Of course before the patch goes final, the interval will be increased. But we're not that far yet. So, it provides good debugging opportunities if it triggers the autosave every 10 seconds, as opposed to every 10 minutes.

If we've decided that history/backup might be useful, and if the autosave "fits in" with the naming convention, then maybe we can unify these. So whereas "$HOME/.hugin-autosave" is a Unix-only filename for the auto-save, if we'd use the name of the project we'd immediately have a valid name on whatever OS this runs on. "untitled-autosave.pto" should work on most OSes too.

Proposal for windows filenames:

* Autosave

Save in c:\Documents and Profiles\Usersname\Local Settings\Application Data\hugin or c:\Users\Usersname\Local Settings\Application Data\hugin (depending on version). On Linux this could be Home/.hugin; but not sure about MacOS.
Filename is random and unique
Automatic deletion when closing hugin
Provide automatic loading of autosave file

* Backup
in the same folder as project file
On saving file rename file on disc to filename_date_time.pto
Save current state under filename.pto
User is responsible for cleanup

I set importance to 'High' because I would like to use 'Critical' only for things that we'd like to see in the next release (the next tarball of the current release cycle, not the next release cycle)