Autocomplete fails on embedded newlines

There's been a lot of work on bash-completion since this bug was opened. Can you check again?

If it's still not working it may be that a newer version can be back-ported to 10.04.

It works for some operations, but not for `cd`. Try pressing TAB at the [tab] marked places to verify:

$ touch -- $'--$`!*@\a\b\E\f\r\t\v\\\'"\360\240\202\211 \n' # my test string
$ touch -- --$[tab]
Autocompletes to the correct file.
$ rm -- --$[tab]
Autocompletes to the correct file.
$ mkdir -- $'--$`!*@\a\b\E\f\r\t\v\\\'"\360\240\202\211 \n' # test
$ cd -- --$[tab]
Fails:
$ cd -- --\$\`\!\*@^G^H^[^L^M\ ^K\\\'\"𠂉\
bash: cd: --$`!*

        \'"𠂉 : No such file or directory
$ rmdir -- --$[tab]
Autocompletes to the correct file.

Still broken in 14.04, bash-completion 1:2.1-4, even with the fixes for bug 1387057 and bug 1390061 applied.

Even the fallback minimal completion function fails for that directory from the OP.
mkdir -- $'--$`\! *@ \a\b\e\E\f\r\t\v\\\"\' \n'
echo ./-[TAB]bash: bad substitution: no closing "`" in `\! *@
        \"'
-\$\`\\\!\ \*@\ ^G^H^[^[^L^M\ ^K\\\"\'\

and the completed filename isn't the correct filename. (changing echo to ls or rm gives no such file or directory).

So even making things easy by starting the filename with ./ isn't enough. I guess the problem isn't that it starts with --.

Looked into this some, and I think I might have a fix. The problem is the backquote, and what we're seeing here is related to the reports of completion inside $( failing.

the progcomp code uses a lot of constructs like eval $z=$foo, to assign by reference to a variable whose name is stored as $z.
e.g.
z=myvar
foo='`test'
eval $z="$foo"
bash: unexpected EOF while looking for matching ``'
bash: syntax error: unexpected end of file

 The trick is to single-quote $foo, so parameter expansion doesn't happen on it at the same time as $z.
eval $z='$foo'
echo $myvar
`test

 I'll post a patch after I go through the file and fix what I find.

Hmm, well what's crossing it up is the embedded newline in the filename.

_filedir does
    local IFS=$'\n'
...
       x=$( compgen -f -X "$xspec" -- "$quoted" ) &&
        while read -r tmp; do
            toks+=( "$tmp" )
        done <<< "$x"
...

    if [[ ${#toks[@]} -ne 0 ]]; then
        # 2>/dev/null for direct invocation, e.g. in the _filedir unit test
        compopt -o filenames 2>/dev/null
        COMPREPLY+=( "${toks[@]}" )
    fi

and apparently it's the COMPREPLY+= line where the error pops up. At least, that's the last line that shows in the set -xv output, so maybe the caller, is where it's getting used in a way that lets the shell get at the backquote.

If you leave out the \n in the filename, it works fine. If you leave out the `, but keep the newline, you get weirdness. (completes the filename, but thinks it's ambiguous, and lists all files in the directory.) Probably because you get an empty element in your COMPREPLY array. No idea how the backquote comes into it.

 So I guess if you have embedded newlines in your filenames, best to just use alt-/ to complete them. Working with embedded newlines sucks, in shell programming.

This is upstream https://alioth.debian.org/tracker/index.php?func=detail&aid=311399&group_id=100114&atid=413095

As the comments there show, fixing this is hard, and means you can't even use compgen. (compgen uses newline as its output record separator.)