missing a.out (ZMAGIC) support

This bug is missing log files that will aid in diagnosing the problem. From a terminal window please run:

apport-collect 966472

and then change the status of the bug to 'Confirmed'.

If, due to the nature of the issue you have encountered, you are unable to run this command, please add a comment stating that fact and change the bug status to 'Confirmed'.

This change has been made by an automated script, maintained by the Ubuntu Kernel Team.

(apport-collect hangs reading from a pipe. strace shows that it calls ~/bin/id, a private program of mine. No doubt the intention was to call /usr/bin/id. A bug in apport-collect.)

$ cat /proc/version
Linux version 2.6.32-22-generic (buildd@yellow) (gcc version 4.4.3 (Ubuntu 4.4.3-4ubuntu5) ) #36-Ubuntu SMP Thu Jun 3 19:31:57 UTC 2010
$ cat /etc/lsb-release
DISTRIB_ID=Ubuntu
DISTRIB_RELEASE=10.04
DISTRIB_CODENAME=lucid
DISTRIB_DESCRIPTION="Ubuntu 10.04.4 LTS"

auquicu, this bug was reported a while ago and there hasn't been any activity in it recently. We were wondering if this is still an issue? If so, as per Trusty it would appear enabled for i386:
grep -r CONFIG_HAVE_AOUT= | grep i386
debian.master/config/i386/config.common.i386:CONFIG_HAVE_AOUT=y

Does this work for you? If not, could you please test for this with the latest development release of Ubuntu? ISO images are available from http://cdimage.ubuntu.com/daily-live/current/ .

If it remains an issue, could you please run the following command in the development release from a Terminal (Applications->Accessories->Terminal), as it will automatically gather and attach updated debug information to this report:

apport-collect -p linux <replace-with-bug-number>

> is this still an issue?

Yes.

> latest development release of Ubuntu? ISO images are available

Downloaded the 386 one. It is too big to burn on a CD, 963641344 bytes.

# mount -o loop trusty-desktop-i386.iso /a
# cd /a/caspar
# mount -o loop filesystem.squashfs /b
# cd /b/boot
# grep AOUT conf*
CONFIG_HAVE_AOUT=y
CONFIG_BINFMT_AOUT=m

Looks good. Now find a way to actually test this. Preferably without rebooting.
Some virtual machine? Installed VMware Player, booted this image, and
# ./program
-bash: ./program: cannot execute binary file
# file program
program: Linux/i386 demand-paged executable (ZMAGIC)
# ldd program
        not a dynamic executable
Probably it also requires some libraries, like libc.so.4 and libm.so.4.
And possibly /lib/ld.so.

I recall that there used to be packages like libc4 and maybe aout-binutils
but I cannot find any such thing right now.

Continuing the above report, I found the old libraries, but first

# insmod /lib/modules/3.13.0-8-generic/kernel/fs/binfmt_aout.ko
# ./program
Segmentation fault (core dumped)
# dmesg | tail
fd_offset is not page aligned. Please convert program: program
program[1157]: segfault at 0 ip

So, maybe a present-day Linux kernel is unable to run ZMAGIC binaries.

Continuing the above report, I also found OMAGIC and QMAGIC binaries.
The OMAGIC binaries just segfault at address 0.
The QMAGIC binaries work, when provided with ld.so and the right libraries.
So, CONFIG_HAVE_AOUT=y and CONFIG_BINFMT_AOUT=m are successful,
but only for QMAGIC, the most recent format.

auquicu, regarding the segmentation faults, a file should have appeared in the folder:
/var/crash

which you would be able to file a crash report against via a terminal:
ubuntu-bug CRASHFILENAME.crash

Please report this and subscribe me to the new bug report, and we can take it from there.

I am not quite sure what that would achieve.
We have come to the conclusion that with CONFIG_HAVE_AOUT=y and CONFIG_BINFMT_AOUT=m the latest Ubuntu version successfully handles QMAGIC a.out executables. Also, that all OMAGIC and ZMAGIC files segfault after executing 0 instructions.
It is impossible for a coredump (in case there is one) to contain anything useful. I suspect a kernel bug where the memory mapping setup is faulty.

Hi,
I have identified where the kernel is giving error when trying to load an OMAGIC: lxr.linux.no/#linux+v3.13.5/arch/x86/ia32/ia32_aout.c#L260

It's just after the vm_brk.

Eg. vm_brk(0, 13048);
0 is the txtaddr and 13048 is the size of the sections TEXT and DATA together. The vm_brk() is returning 18446744073709551615, so, the kernel sigkill the current.