Bazaar Git Plugin

Failed to branch git repository (invalid record 0x03)

Bug #868981 reported by Hibou57 (Yannick Duchêne)
6
This bug affects 1 person
Affects Status Importance Assigned to Milestone
Bazaar Git Plugin
Invalid
Undecided
Unassigned

Bug Description

The following command (copy/paste from log console of Bazaar Explorer):

> Run command: bzr branch http://gcc.gnu.org/git/gcc.git /home/yannick/Applications/GCC/gcc.git --use-existing-dir

Terminated with this error:

> bzr: ERROR: Unknown record type: '\x03'

This was at the second attempt. I would also like to note at the first attempt, it fails due to lack of memory. It seems it tried to download all data into memory instead of to a temporary file or folder. Is that normal behavior ? Whatever for that latter point, the former is an error log anyway.

Have a nice day.

7342.039 Traceback (most recent call last):
  File "/usr/lib/pymodules/python2.6/bzrlib/commands.py", line 946, in exception_to_return_code
    return the_callable(*args, **kwargs)
  File "/usr/lib/pymodules/python2.6/bzrlib/commands.py", line 1150, in run_bzr
    ret = run(*run_argv)
  File "/usr/lib/pymodules/python2.6/bzrlib/commands.py", line 699, in run_argv_aliases
    return self.run(**all_cmd_args)
  File "/usr/lib/pymodules/python2.6/bzrlib/commands.py", line 721, in run
    return self._operation.run_simple(*args, **kwargs)
  File "/usr/lib/pymodules/python2.6/bzrlib/cleanup.py", line 135, in run_simple
    self.cleanups, self.func, *args, **kwargs)
  File "/usr/lib/pymodules/python2.6/bzrlib/cleanup.py", line 165, in _do_with_cleanups
    result = func(*args, **kwargs)
  File "/usr/lib/python2.6/dist-packages/bzrlib/plugins/qbzr/lib/commands.py", line 821, in run
    return run_subprocess_command(cmd, bencoded)
  File "/usr/lib/python2.6/dist-packages/bzrlib/plugins/qbzr/lib/subprocess.py", line 888, in run_subprocess_command
    return commands.run_bzr(argv)
  File "/usr/lib/pymodules/python2.6/bzrlib/commands.py", line 1150, in run_bzr
    ret = run(*run_argv)
  File "/usr/lib/pymodules/python2.6/bzrlib/commands.py", line 699, in run_argv_aliases
    return self.run(**all_cmd_args)
  File "/usr/lib/pymodules/python2.6/bzrlib/commands.py", line 721, in run
    return self._operation.run_simple(*args, **kwargs)
  File "/usr/lib/pymodules/python2.6/bzrlib/cleanup.py", line 135, in run_simple
    self.cleanups, self.func, *args, **kwargs)
  File "/usr/lib/pymodules/python2.6/bzrlib/cleanup.py", line 165, in _do_with_cleanups
    result = func(*args, **kwargs)
  File "/usr/lib/pymodules/python2.6/bzrlib/builtins.py", line 1307, in run
    source_branch=br_from)
  File "/usr/lib/python2.6/dist-packages/bzrlib/plugins/git/dir.py", line 169, in sprout
    mapping=source_branch.mapping)
  File "/usr/lib/python2.6/dist-packages/bzrlib/plugins/git/fetch.py", line 678, in fetch_objects
    target_git_object_retriever, wants, pb, limit)
  File "/usr/lib/python2.6/dist-packages/bzrlib/plugins/git/fetch.py", line 476, in import_git_objects
    target_git_object_retriever, trees_cache)
  File "/usr/lib/python2.6/dist-packages/bzrlib/plugins/git/fetch.py", line 375, in import_git_commit
    allow_submodules=getattr(repo._format, "supports_tree_reference", False))
  File "/usr/lib/python2.6/dist-packages/bzrlib/plugins/git/fetch.py", line 283, in import_git_tree
    allow_submodules=allow_submodules)
  File "/usr/lib/python2.6/dist-packages/bzrlib/plugins/git/fetch.py", line 283, in import_git_tree
    allow_submodules=allow_submodules)
  File "/usr/lib/python2.6/dist-packages/bzrlib/plugins/git/fetch.py", line 283, in import_git_tree
    allow_submodules=allow_submodules)
  File "/usr/lib/python2.6/dist-packages/bzrlib/plugins/git/fetch.py", line 295, in import_git_tree
    (child_base_mode, child_mode), store_updater, lookup_file_id)
  File "/usr/lib/python2.6/dist-packages/bzrlib/plugins/git/fetch.py", line 139, in import_git_blob
    pie = pinv[file_id]
  File "/usr/lib/pymodules/python2.6/bzrlib/inventory.py", line 1906, in __getitem__
    self.id_to_entry.iteritems([StaticTuple(file_id,)]).next()[1])
  File "/usr/lib/pymodules/python2.6/bzrlib/chk_map.py", line 1047, in iteritems
    for node, node_filter in self._iter_nodes(store, key_filter=key_filter):
  File "/usr/lib/pymodules/python2.6/bzrlib/chk_map.py", line 1189, in _iter_nodes
    for record in stream:
  File "/usr/lib/pymodules/python2.6/bzrlib/groupcompress.py", line 1474, in get_record_stream
    orig_keys, ordering, include_delta_closure):
  File "/usr/lib/pymodules/python2.6/bzrlib/groupcompress.py", line 1635, in _get_remaining_record_stream
    for factory in batcher.yield_factories(full_flush=True):
  File "/usr/lib/pymodules/python2.6/bzrlib/groupcompress.py", line 1188, in yield_factories
    block_read_memo, block = blocks.next()
  File "/usr/lib/pymodules/python2.6/bzrlib/groupcompress.py", line 1430, in _get_blocks
    zdata = raw_records.next()
  File "/usr/lib/pymodules/python2.6/bzrlib/repofmt/pack_repo.py", line 2027, in get_raw_records
    for names, read_func in reader.iter_records():
  File "/usr/lib/pymodules/python2.6/bzrlib/pack.py", line 290, in _iter_records
    for record in self._iter_record_objects():
  File "/usr/lib/pymodules/python2.6/bzrlib/pack.py", line 309, in _iter_record_objects
    raise errors.UnknownRecordTypeError(record_kind)
UnknownRecordTypeError: Unknown record type: '\x03'

0.544 bazaar version: 2.4.1

See original description
Tags: branch git
Revision history for this message
Hibou57 (Yannick Duchêne) (yannick-duchene) wrote :

Oops, forget versions informations:

bzr --version returned

> Bazaar (bzr) 2.4.1
> Python interpreter: /usr/bin/python 2.6.6
> Python standard library: /usr/lib/python2.6
> Platform: Linux-2.6.31-10-rt-i686-with-Ubuntu-10.10-maverick
> bzrlib: /usr/lib/pymodules/python2.6/bzrlib
> Bazaar configuration: /home/yannick/.bazaar
> Bazaar log file: /home/yannick/.bzr.log

Bazaar Explorer version is

> Bazaar Explorer — Version Control for Human Beings
> Version 1.2.1 "Leif Ericson"
>
> QBzr 0.21.1, bzrlib 2.4.1, PyQt 4.7.4, Qt 4.7.0, Python 2.6.6

Attachment: compressed content of .bzr.log

Martin Pool (mbp)
description: updated
description: updated
Revision history for this message
Martin Pool (mbp) wrote :

that error seems to indicate corruption in the bzr packs; it's not obvious to me it would be connected to the git import and also not obvious what we could do with the report, if it happens after a large download it may be due to some local environmental factor.

memory usage should not grow without bound but that's a separate issue; you could file one against bzr-git with the output of -Dmem_dump

Revision history for this message
Jelmer Vernooij (jelmer) wrote :

What version of bzr-git are you running?

affects: bzr → bzr-git
Revision history for this message
Hibou57 (Yannick Duchêne) (yannick-duchene) wrote :

Jelmer,

bzr-git version is 0.6.2

I've also just installed bzr-fastimport version 0.11.0 for another test (cloning the repository with git and then will try to import it to see if it works).

Revision history for this message
Hibou57 (Yannick Duchêne) (yannick-duchene) wrote :

I don't know what caused it to behave differently.: I tried to run bzr from the command line instead of from Bazaar Explorer. Created a directory, then from this directory, invoked “bzr branch git://gcc.gnu.org/git/gcc.git”.

I did not get this error, while it finally fails (see later). The sole difference is that I was invoked from the command line, and I used git://gcc.gnu.org/git/gcc.git instead of http://gcc.gnu.org/git/gcc.git, because git://gcc.gnu.org/git/gcc.git caused an error from Bazaar Explorer. May be something wrong with integration in Bazaar Explorer or when invoked from Bazaar Explorer ? Don't know.

Well, there was no error, except it was horribly long at fetching revisions. After abut 30 hours (thirty hours), running alone, with no other application running and the hard drive scratching all that time (not for swap), it was near to terminates, when it stop complaining there was no more space left on device. The gc.git repository was 6.5G at that moment, and there was 6G of free space remaining on hard drive.

I don't know why it was so horribly long, but at least, something may be good: an incremental version which you could stop and start again later, if ever this really cannot be faster and consume less memory and storage. It could also try to estimate the time required a to branch a repository, based on some average measure. With this one, a repository with about 112000 revisions, and a duration of near to 30 hours before reaching the near end, an average time could be inferred: about a seconds per revision to fetch. Would be useful too, to to an estimation of required memory and disk space, but have bot idea for that.

I insisted with this test even is this was overlong, because I wanted to know how it will terminate (will not know, and don't know neither how much of disk space it attempted to reserve when it fails).

Also, may be this gcc repository could be used for a good benchmarking of bzr-git.

May have to drop Bazaar if its so long and resource consuming (git imported the same within about 45 minutes and using 1G of hard disk space).

Revision history for this message
Jelmer Vernooij (jelmer) wrote :

What version of bzr-git are you using?

Revision history for this message
Jelmer Vernooij (jelmer) wrote :

Sorry, just saw you're using 0.6.2. I'm going to have a look at reproducing this locally.

Revision history for this message
Hibou57 (Yannick Duchêne) (yannick-duchene) wrote :

A detail I forget to state: for this last test test, I invoked “bzr branch git://gcc.gnu.org/git/gcc.git” straight away, without any prior “bzr init”. While I don't know it this relevant or not.

Another difference I forget to notice, is that when I did the first test from Bazaar Explorer, bzr did not used a temporary file, and exhausted memory probably for that reason. However, during the second test, from the command line, it did use a temporary file for downloaded data. There as well, I don't know it its due to something I did and did not noticed, or if this is due to bzr integration in its GUI.

I've made a copy of that temporary file named “tmpINQPMV.temp”, and if this ever that can help to get it, just tell me (just note it is a 500M bytes files).

Revision history for this message
Jelmer Vernooij (jelmer) wrote :

It shouldn't be necessary to run bzr init beforehand.

The memory issues should be gone or at least significantly less problematic than with 0.6.2.

I can't get to this repository, even with git. It just seems to hang. Should it still work?

Changed in bzr-git:
status: New → Incomplete
Revision history for this message
Jelmer Vernooij (jelmer) wrote :

I'm closing this bug as it has been open without sufficient information for more than 3 months. Feel free to reopen it with the requested information.

Changed in bzr-git:
status: Incomplete → Invalid
To post a comment you must log in.
This report contains Public information  
Everyone can see this information.
Subscribing...

Other bug subscribers

Related questions

Remote bug watches

Bug watches keep track of this bug in other bug trackers.