bzr: ERROR: [Error 145] The directory is not empty: u'C:/Users/exarkun/twistedbot/windows7-64-py2.6-select/Twisted/.bzr/checkout/limbo/new-7/words'

Bug #597686 reported by Jean-Paul Calderone
18
This bug affects 4 people
Affects Status Importance Assigned to Milestone
Bazaar
Fix Released
Medium
Martin Packman

Bug Description

On Windows 7, a "bzr checkout" command failed with this error, though a later one succeeded.

159.151 Traceback (most recent call last):
  File "bzrlib\commands.pyo", line 853, in exception_to_return_code
  File "bzrlib\commands.pyo", line 1055, in run_bzr
  File "bzrlib\commands.pyo", line 661, in run_argv_aliases
  File "bzrlib\commands.pyo", line 665, in run_direct
  File "bzrlib\cleanup.pyo", line 122, in run_simple
  File "bzrlib\cleanup.pyo", line 156, in _do_with_cleanups
  File "bzrlib\builtins.pyo", line 1328, in run
  File "C:/Program Files (x86)/Bazaar/plugins\svn\branch.py", line 337, in create_checkout
  File "C:/Program Files (x86)/Bazaar/plugins\svn\branch.py", line 283, in _create_heavyweight_checkout
  File "bzrlib\bzrdir.pyo", line 1616, in create_workingtree
  File "bzrlib\workingtree_4.pyo", line 1462, in initialize
  File "bzrlib\transform.pyo", line 2253, in build_tree
  File "bzrlib\transform.pyo", line 2365, in _build_tree
  File "bzrlib\transform.pyo", line 1103, in finalize
  File "bzrlib\osutils.pyo", line 1038, in delete_any
  File "bzrlib\osutils.pyo", line 1058, in _delete_file_or_dir
WindowsError: [Error 145] The directory is not empty: u'C:/Users/exarkun/twistedbot/windows7-64-py2.6-select/Twisted/.bzr/checkout/limbo/new-7/words'

Related branches

Revision history for this message
Jean-Paul Calderone (exarkun) wrote :
Andrew Bennetts (spiv)
tags: added: treetransform win32
description: updated
Revision history for this message
Andrew Bennetts (spiv) wrote :

This seems to be from the delete_any call in DiskTreeTransform that isn't wrapped in a try/finally.

I wonder if it would be acceptable to catch the OSError and just emit a warning rather than fail in this situation?

I also wonder about the correctness of this code on a case-insensitive filesystem:

            entries = [(self._limbo_name(t), t, k) for t, k in self._new_contents.iteritems()]
            entries.sort(reverse=True)
            for path, trans_id, kind in entries: delete_any(path)

(formatting tweaked to avoid LP munging it too badly)

Probably it's ok to assume that the various entries will be returned with consistent case... but if it's not a safe assumption then this code might try to delete 'a' before 'A/B'. [I suppose an other Windows-specific issue is that path separator is \ on Windows, but A\B and A/B should both before A after a sort(reverse=True).]

It might be good to catch OSError around that delete_any, and try to report more details about the failing path if it does happen, e.g. perms, contents if a dir, etc.

Changed in bzr:
importance: Undecided → Medium
status: New → Confirmed
Revision history for this message
Martin Packman (gz) wrote :

I can reproduce this pretty reliably with bzr-dev by branching locally and hitting ctrl+c near the end of the "adding file contents" stage.

With a couple of tries and different python versions I got errors like:
OSError: [Errno 41] Directory not empty: u'.../.bzr/checkout/limbo/new-18/ru/user-guide/images'
WindowsError: [Error 145] The directory is not empty: u'.../.bzr/checkout/limbo/new-16/tests/blackbox'

So, at the least the backout code needs fixing, and perhaps modifying so it doesn't completely mask the original error as well?

Revision history for this message
Martin Packman (gz) wrote :

Catching one of these in pdb, looks like this isn't related to case-sensitivity. The file remaining in the directory isn't in the entities list at all, but is zero bytes in size. If the interrupt happened after the file was created, but before it was closed, would it not appear in the list of files that need removing?

Revision history for this message
Martin Packman (gz) wrote :

Yup, it's pretty clear that the logic DiskTreeTransform.create_file is wrong if open() throws but the file has been created. This looks like a cross-platform bug, it's probably just harder to hit on posix filesystems with a smaller window for the race. More details in this double post-mortem session:

> c:\bzr\bzr\dev\bzrlib\transform.py(1270)create_file()
-> f = open(name, 'wb')
(Pdb) w
  c:\bzr\bzr\dev\bzrlib\transform.py(2559)_build_tree()
-> accelerator_tree, hardlink)
  c:\bzr\bzr\dev\bzrlib\transform.py(2634)_create_files()
-> tt.create_file(contents, trans_id, sha1=text_sha1)
> c:\bzr\bzr\dev\bzrlib\transform.py(1270)create_file()
-> f = open(name, 'wb')
(Pdb) name
u'C:/bzr/bzr/tmptest/.bzr/checkout/limbo/new-16/treebuilder.py'
(Pdb) os.path.exists(name)
True
(Pdb) l
1265 Otherwise, we will try to preserve mode bits of an existing file.
1266 :param sha1: If the sha1 of this content is already known, pass it in.
1267 We can use it to prevent future sha1 computations.
1268 """
1269 name = self._limbo_name(trans_id)
1270 -> f = open(name, 'wb')
1271 try:
1272 try:
1273 unique_add(self._new_contents, trans_id, 'file')
1274 except:
1275 # Clean up the file, it never got registered so
(Pdb) c
> c:\bzr\bzr\dev\bzrlib\osutils.py(1079)_delete_file_or_dir()
-> os.rmdir(path)
(Pdb) path
u'C:/bzr/bzr/tmptest/.bzr/checkout/limbo/new-16'
(Pdb) os.listdir(path)
[u'treebuilder.py']
(Pdb) w
  c:\bzr\bzr\dev\bzrlib\transform.py(1180)finalize()
-> delete_any(path)
  c:\bzr\bzr\dev\bzrlib\osutils.py(1059)delete_any()
-> _delete_file_or_dir(path)
> c:\bzr\bzr\dev\bzrlib\osutils.py(1079)_delete_file_or_dir()
-> os.rmdir(path)
(Pdb) u
> c:\bzr\bzr\dev\bzrlib\osutils.py(1059)delete_any()
-> _delete_file_or_dir(path)
(Pdb) u
> c:\bzr\bzr\dev\bzrlib\transform.py(1180)finalize()
(Pdb) [e for e in entries if e[0].endswith("/treebuilder.py")]
[]

Andrew Bennetts (spiv)
Changed in bzr:
assignee: nobody → Martin [gz] (gz)
status: Confirmed → In Progress
Martin Packman (gz)
Changed in bzr:
milestone: none → 2.4b4
status: In Progress → Fix Released
Martin Packman (gz)
tags: added: affects-twisted
To post a comment you must log in.
This report contains Public information  
Everyone can see this information.

Other bug subscribers

Bug attachments

Remote bug watches

Bug watches keep track of this bug in other bug trackers.