log can show too few revs

Bug #1072513 reported by Matthew Fuller
6
This bug affects 1 person
Affects Status Importance Assigned to Milestone
Bazaar
Confirmed
High
Unassigned
Breezy
Triaged
High
Unassigned

Bug Description

Depending on the shape of the history graph, doing "log" on a directory can yield very unexpected results.

This is a very long explanation. I don't have time to make it shorter :(

--

The graph is convoluted enough you probably want to use qlog to look at it. However, here's a quick overview:

    7 Matthew Fuller 2012-10-28
      Nada

    6 Matthew Fuller 2012-10-28 [merge]
      Merge C:1

          1.2.4 Matthew Fuller 2012-10-28 [merge]
                Merge A:2

          1.2.3 Matthew Fuller 2012-10-28
                Nada

          1.2.2 Matthew Fuller 2012-10-28 [merge]
                Merge A:1
                A foo/
                A foo/bar

          1.2.1 Matthew Fuller 2012-10-28
                Nada

    5 Matthew Fuller 2012-10-28
      Nada

    4 Matthew Fuller 2012-10-28
      Empty to force merge

    3 Matthew Fuller 2012-10-28
      Edit bar
      M foo/bar

    2 Matthew Fuller 2012-10-28 [merge]
      Merge B:1
      A foo/
      A foo/bar

          1.1.1 Matthew Fuller 2012-10-28
                Create foo
                A foo/
                A foo/bar

    1 Matthew Fuller 2012-10-28
      Init

C started at (1). The A:1 merge landed (2,3,4), and A:2 landed (5).

--

Now, depending on minor alterations I make, mostly to revs that don't touch foo, I get differing results from log.

The various permutations of running the script below look like this:

------
------

Without -n0, I only ever see

  "Edit bar" (3)
  "Merge B:1" (2)

which is probably what's expected.

------
------

If I uncomment the mk_dir= at [3] to make foo a file rather than a
directory, I get

  "Edit bar" (3)
  "Merge B:1" (2)
    "Create foo" (1.1.1)

which may possibly be the "right" answer (though see below), and none of the below changes have any effect.

--

If, however, I run a "log -v" instead, I see only

    "Merge A:1" (1.2.2)

just like the directory version below, and the described alterations have the same effects.

------
------

However, with foo/ as a directory (the presence/absence of -v makes no difference in this case):

Running the whole thing, I get only

    "Merge A:1" (1.2.2)

which is the *least* useful possible answer, since it tells me nothing at all about most of the revs that touched files in the dir, only the merge into C of the "nothing" -> "final state" transition.

--

If I take out any one of the Nada's labelled [1] I get

    "Merge A:1" (1.2.2)
  "Edit bar" (3)

which means I see the above "everything rolled up" merge, plus the edit rev. But I never see the rev that added foo/bar in its original form.

--

If I take out any 2 of [1], or uncomment the "exit" to skip the [2] block, I get
    "Merge A:1" (1.2.2)
  "Edit bar" (3)
  "Merge B:1" (2)

which means in addition to the above, I now see the merge of the original creation, though not the creation itself.

--

If I take out any 3 or more of [1], or 1+ of the [1] AND the whole [2] block, I
get
    "Merge A:1" (1.2.2)
  "Edit bar" (3)
  "Merge B:1" (2)
    "Create foo" (1.1.1)

which now gives me every rev that changes, relative to its left-hand parent, something in foo/.

--

I believe either the 3-rev version shown without -v for the file version (3, 2, 1.1.1) OR the 4-rev version for dirs with several deletions (1.2.2, 3, 2, 1.1.1) are "correct"; I'm not sure which is "better".

However, I'm _quite_ sure all the 1- and 2-rev versions, and that the 3-rev in the deletions (1.2.2, 3, 2) are wrong wrong wrong. And very unpleasant to run across, since you may not realize bzr is lying to you and take log at face value.

I checked this with bzr.dev, 2.5.1, and the latest 2.0; all acted identically.

------
------

Creation script (inline):

#!/bin/sh -e
#bzr="/home/fullermd/src/bzr/bzr.dev/bzr --no-plugins --no-aliases"
bzr="/usr/local/bin/bzr --no-plugins --no-aliases"

if [ -d "test" ]; then
 echo "test/ already exists"
 exit
fi

mk_dir=true
# [3]
#mk_dir=false

${bzr} init-repo test
(
 cd test

 # First, create a branch with a rev
 ${bzr} init A
 (
  cd A
  ${bzr} ci --unchanged -m 'Init'
 )

 # Now work on a file
 ${bzr} branch A B
 (
  cd B

  if $mk_dir; then
   mkdir foo
   echo bar > foo/bar
  else
   touch foo
  fi

  ${bzr} add
  ${bzr} ci -m 'Create foo'
 )

 # Merge that
 (
  cd A
  ${bzr} merge ../B
  ${bzr} ci -m "Merge B:1"
  if $mk_dir; then
   echo baz > foo/bar
  else
   echo baz > foo
  fi
  ${bzr} ci --unchanged -m 'Edit bar'
  # [1]
  ${bzr} ci --unchanged -m 'Nada'
 )

 # Now make another branch from the start of A, and merge later stuff
 # from A
 ${bzr} branch -r1 A C
 (
  cd C
  # [1]
  ${bzr} ci --unchanged -m 'Nada'
  ${bzr} merge ../A
  ${bzr} ci -m 'Merge A:1'
  # [1]
  ${bzr} ci --unchanged -m 'Nada'
 )

 # Make another change, and merge it into C
 # [2]
 (
  #exit
  cd A
  ${bzr} ci --unchanged -m 'Empty to force merge'

  cd ../C
  ${bzr} merge ../A
  ${bzr} ci -m 'Merge A:2'
 )

 # Now land those changes back on A
 (
  cd A
  # [1]
  ${bzr} ci --unchanged -m 'Nada'
  ${bzr} merge ../C
  ${bzr} ci -m "Merge C:1"
  # [1]
  ${bzr} ci --unchanged -m 'Nada'
 )

 # Now see when foo was touched.
 (
  cd A
  ${bzr} log -n0 foo
  #${bzr} log -n0 -v foo
 )
)

Tags: log
Revision history for this message
Matthew Fuller (fullermd) wrote :

Repro script as attachment, for easier grabbing.

Changed in bzr:
importance: Undecided → High
description: updated
Revision history for this message
Matthew Fuller (fullermd) wrote :

I have a sneaky fear there may be at least 2 bugs in here, given the way -v alters the behavior for the file version.

description: updated
description: updated
Revision history for this message
Matthew Fuller (fullermd) wrote :

OK, that desc got too long and convoluted, with the problem mixed in with a lot of exploration. Let me reiterate the base _problem_.

With the history shown at the top, running "bzr log foo/" should probably show

  1.2.2 (merge all 3 below into second branch)
 3 (make a change)
 2 (merge the add)
  1.1.1 (add)

as those are the revs that edited stuff under foo. However, it _only_ shows

  1.2.2

which is the least useful answer, since the 1.2.1..1.2.2 change is a merge of essentially the entire history of foo/* (all 3 of the other revs). Same result obtained with both plain "log foo" and "log -v foo".

Revision history for this message
Matthew Fuller (fullermd) wrote :

In the case where we make foo a file rather than a dir, we get the same bad result (only 1.2.2) when using "log -v foo", though with "log foo" we instead get

 3 (make a change)
 2 (merge add)
  1.1.1 (add)

but _don't_ see the 1.2.2. So "log" shows one rev, "log -v" shows 3, and there's no intersection between the sets!

Revision history for this message
Matthew Fuller (fullermd) wrote :

In the real case where I ran across this bug, the one change it showed actually turned out to include an edit of one of the files under the dir, which means the diff showed some --- and some +++ to a file that, according to log, never got touched in any other rev (and so was never created with the content this rev was ---'ing!)

Revision history for this message
Matthew Fuller (fullermd) wrote :

The extra weirdness found along the way is that removing the "bzr ci --unchanged -m 'Nada'" commits in the script, one at a time, causes the extra revs that SHOULD be shown to start showing up, one at a time, until removing any 3 of them causes all 4 revs to show up.

This may or may not be related, but it's very weird stuff that showed up in the process of writing the reproduction script, so I mentioned it.

Revision history for this message
Matthew Fuller (fullermd) wrote :

The confusion seems to have come up a few times that this bug is saying "'log DIR' should show changes to files under DIR, not just DIR itself". That's not the case; for one thing, that's already long since the way bzr works (bug 97715, bzr.dev:4206, released in 1.14).

This bug is that, given particular graph shapes, revs that SHOULD show up don't. By taking out some of the --unchanged commits (and thus changing the shape), they DO show up, showing that it's something specific to the history shape, not a general issue in the command.

Also see the case where uncommenting the "mk_dir=false" in the script (and thus working with a file instead of a dir) gives the same result with "log -v", though not without. This shows that it's not specific to dirs either; consideration of the similarities between "log -v FILE" and "log [-v] DIR" may point in a fruitful direction.

Revision history for this message
Matthew Fuller (fullermd) wrote :

In further investigation, it seems that the extra merge in the block at [2] is not a necessary part of this, which simplifies things a bit. Just adding 2 Nada revs has the same effect. Here's the revised full log, when using mk_dir=false (and so having a /foo file rather than a /foo/ dir):

    7 Matthew Fuller 2012-11-05
      Nada

    6 Matthew Fuller 2012-11-05 [merge]
      Merge C:1

          1.2.5 Matthew Fuller 2012-11-05
                Nada

          1.2.4 Matthew Fuller 2012-11-05
                Nada

          1.2.3 Matthew Fuller 2012-11-05
                Nada

          1.2.2 Matthew Fuller 2012-11-05 [merge]
                Merge A:1
                A foo

          1.2.1 Matthew Fuller 2012-11-05
                Nada

    5 Matthew Fuller 2012-11-05
      Nada

    4 Matthew Fuller 2012-11-05
      Nada

    3 Matthew Fuller 2012-11-05
      Edit file
      M foo

    2 Matthew Fuller 2012-11-05 [merge]
      Merge B:1
      A foo

          1.1.1 Matthew Fuller 2012-11-05
                Create foo
                A foo

    1 Matthew Fuller 2012-11-05
      Init

Revised script to be attached.

Revision history for this message
Matthew Fuller (fullermd) wrote :

Revised script without unnecessary merge.

Revision history for this message
Matthew Fuller (fullermd) wrote :

Recap based on new output:

The full set of revs touching foo (from looking at the log -v above) are
    1.2.2
  3
  2
    1.1.1

However, with the script run, only
    1.2.2
shows up (with log -v for FILE, with log with or without -v for DIR).

Taking out _any_ one of the [1]-marked "Nada" commits yields:
    1.2.2
  3

Any two of the [1]'s:
    1.2.2
  3
  2

And any 3 or more:
    1.2.2
  3
  2
    1.1.1

(though watch out for renumberings, depending on which you take out) Doesn't matter which [1]'s you take out; the revs in log are always revealed in that specific order.

--------------------------------------------------

An extra note I came across: See the merge of A into C at [4]. I get the same results whether I merge just rev 2 (the merge of B, which adds /foo), merge rev 3 (the edit of foo), or merge up through rev 4 (the --unchanged "Nada"); only that merge (1.2.2) shows up.

Note however that the presence of the rev 4 "Nada" (marked [4.1] in the script) is required whether or not it's part of the merge into C; taking it out still triggers the first step of "more revs showing up" by unveiling 3 in the log output.

Revision history for this message
Vincent Ladeuil (vila) wrote :

Branch reproducing the failures for 'bzr log --line -n0 foo' attached.

Vincent Ladeuil (vila)
Changed in bzr:
status: New → Confirmed
Jelmer Vernooij (jelmer)
tags: added: check-for-breezy
Jelmer Vernooij (jelmer)
Changed in brz:
status: New → Triaged
importance: Undecided → High
milestone: none → 3.0.0
tags: removed: check-for-breezy
tags: added: log
Jelmer Vernooij (jelmer)
Changed in brz:
milestone: 3.0.0 → 3.1.0
To post a comment you must log in.
This report contains Public information  
Everyone can see this information.

Other bug subscribers

Remote bug watches

Bug watches keep track of this bug in other bug trackers.