no idmapd for nfs4-clients

Hi there,

(the redirect is correct, nfs-utils is the source package that provides nfs-common :)

Why are you trying to run rpc.idmapd by hand? There is an upstart job, /etc/init/idmapd.conf, which is meant to take care of starting idmapd for you at boot time. You can also run it by hand, with 'sudo service idmapd start'. This job automatically triggers the mounting of /var/lib/nfs/rpc_pipefs.

> Why are you trying to run rpc.idmapd by hand?

Because I wanted to see if idmapd is able to start, anyway, and if not, I expected the terminal output of idmapd is useful for bugfixing. :)

Thanks for the quick reply, by the way…

Just to make sure, I don't confuse things here: You mean, if I started idmapd by "sudo service idmapd start", everything would be fine for nfs4?
Or do you agree, there's something wrong with nfs4 in Ubuntu and beside this you wanted to give a hint, how to use upstart right? (Really - I appreciate this! This upstart things are confusing to me - I'm using Debian for quite a while, now.)

On Thu, Jun 02, 2011 at 10:33:37PM -0000, Mitsch wrote:
> Just to make sure, I don't confuse things here: You mean, if I started
> idmapd by "sudo service idmapd start", everything would be fine for nfs4?

To the best of my knowledge, yes, if you start idmapd using the system
interface NFSv4 works fine. I'm using it this way myself without any
trouble.

Please let us know whether you have any problems when starting idmapd via
upstart. As I understand it you've already gotten this working manually on
two of your three systems and that you're still investigating the
configuration on a third.

--
Steve Langasek Give me a lever long enough and a Free OS
Debian Developer to set it on, and I can move the world.
Ubuntu Developer http://www.debian.org/
<email address hidden> <email address hidden>

> I'm using it this way myself without any trouble.

If you have already installed nfs-kernel-server on your system, you are right: dpkg installs several directories under /var/lib/nfs/rpc_pipefs with that package, also the "nfs" directory, needed for idmapd (as server AND client). But if you only install nfs-common, idmapd won't start until you mkdir /var/lib/nfs/rpc_pipefs/nfs by hand. Promise! That's the only way I can make idmapd run on a nfs-client-only-machine with only nfs-common installed, otherwise it fails - this is prooven to work on all my Ubuntu-machines, even on the netbook: idmapd does run, now (- although I get the same funny nonsense about the owners and groups in nautilus and with ls -l in the terminal on the netbook like before. But this is only on the netbook and it seems to be another issue. It may have nothing to do with idmapd on the nfs-client.)

To be more precise: idmapd now is running automatically on startup (no need for manual work), just like configured in /etc/default/nfs-common and just like expected - but that is AFTER I created the needed /var/lib/nfs/rpc_pipefs/nfs directory by hand, which does not exist after a clean Ubuntu 11.04 install with just nfs-common installed.

Hope, that was clear, now! :)

Greets!
Mitsch

The problem seems to be this:

:~# service rpc_pipefs start
start: Unknown parameter: JOB

Doing it by hand:
mount -t rpc_pipefs rpc_piefs /var/lib/nfs/rpc_pipefs/nfs/

solves the idmap and so problem, something seems to be incorrect with upstart and the rpc_pipefs initscript

had the same trouble on natty when setting specific settings in /etc/defaults/nfs-* (yes/no) for the services there (idmap......)

It seems that the autodetection is not doing very well in maveric but in natty

Hope this helps!

That doesn't show anything wrong with the rpc_pipefs upstart job. It's not *meant* to be run by hand; it's meant to trigger when you run 'service idmapd start'.

On Thu, Jun 02, 2011 at 11:55:49PM -0000, Mitsch wrote:
> > I'm using it this way myself without any trouble.

> If you have already installed nfs-kernel-server on your system, you are
> right: dpkg installs several directories under /var/lib/nfs/rpc_pipefs
> with that package, also the "nfs" directory, needed for idmapd (as
> server AND client). But if you only install nfs-common, idmapd won't
> start until you mkdir /var/lib/nfs/rpc_pipefs/nfs by hand. Promise!

This is certainly not the case. First, the nfs-kernel-server package does
not ship, or create at install time, any files under
/var/lib/nfs/rpc_pipefs. Second, /var/lib/nfs/rpc_pipefs is a kernel
*mount point*, so creating directories under it is pointless; they're
created when mounting the filesystem.

Third, I am using this as I described with no nfs-kernel-server package
installed, and can assure you that idmap has no trouble starting at boot
provided that either NEED_IDMAPD is set in /etc/default/nfs-common, or nfs4
mounts are listed in /etc/fstab.

I don't see anywhere that the bug you describe could be hiding. I'm afraid
it looks like you've misdiagnosed the original symptoms caused by trying to
start rpc.idmapd by hand instead of via the upstart job. As such, I am
closing this bug report as invalid. If you can tell us how to reproduce the
bug step-by-step from a clean install, calling rpc.idmapd via the upstart
job and *not* by hand, please reopen this report providing that information.

Fixed branch linked.

1:1.2.5-2ubuntu1 (r49) introduced utils/blkmapd/device-discovery.c and at the
  same time moved /var/lib/nfs/rpc_pipefs to /run/rpc_pipefs. r49 did not modify
  the pre-processor macro BL_PIPE_FILE which remained pointing at the original
  directory.

TJ, please file a separate bug report for the issue you've identified. BL_PIPE_FILE is only used in blkmapd, which is not what this report was about.

No, this is the bug I was experiencing. NFSv4 mounts failing at boot-time (or later when manually attempted) with idmapd failing.

I chased down the location in source of the string "/var/lib/nfs/rpc_pipefs" and found the only place it is hard-coded is in device-discovery.c. I applied the linked patch and the clients can now mount the NFSv4 exports successfully.

Sep 2 06:27:54 hephaestion rpc.idmapd[4449]: main: open(/var/lib/nfs/rpc_pipefs/nfs): No such file or directory
Sep 2 06:27:54 hephaestion rpc.idmapd[4461]: main: open(/var/lib/nfs/rpc_pipefs/nfs): No such file or directory
Sep 2 06:27:54 hephaestion rpc.idmapd[4473]: main: open(/var/lib/nfs/rpc_pipefs/nfs): No such file or directory
Sep 2 06:27:54 hephaestion rpc.idmapd[4485]: main: open(/var/lib/nfs/rpc_pipefs/nfs): No such file or directory
Sep 2 06:27:54 hephaestion rpc.idmapd[4497]: main: open(/var/lib/nfs/rpc_pipefs/nfs): No such file or directory
Sep 2 06:27:54 hephaestion rpc.idmapd[4509]: main: open(/var/lib/nfs/rpc_pipefs/nfs): No such file or directory
Sep 2 06:27:54 hephaestion rpc.idmapd[4521]: main: open(/var/lib/nfs/rpc_pipefs/nfs): No such file or directory
Sep 2 06:27:54 hephaestion rpc.idmapd[4533]: main: open(/var/lib/nfs/rpc_pipefs/nfs): No such file or directory
Sep 2 06:27:54 hephaestion rpc.idmapd[4545]: main: open(/var/lib/nfs/rpc_pipefs/nfs): No such file or directory
Sep 2 06:27:54 hephaestion rpc.idmapd[4557]: main: open(/var/lib/nfs/rpc_pipefs/nfs): No such file or directory
Sep 2 06:27:54 hephaestion rpc.idmapd[4569]: main: open(/var/lib/nfs/rpc_pipefs/nfs): No such file or director

I'm continuing to dig down on this one since it is still biting. The one thing that is certain is that with the patch applied the error from rpc.idmapd doesn't appear in the syslog. Here's logging *without* the patch, using the archive packages.

Working on Precise amd64.

$ apt-cache policy nfs-common
nfs-common:
  Installed: 1:1.2.5-3ubuntu3
  Candidate: 1:1.2.5-3ubuntu3

/etc/fstab contains:

10.254.251.1:/Library /home/all/Library nfs4 _netdev,auto,user 0 0

I wrapped /usr/sbin/rpc.idmapd and blkmapd with shell scripts to capture their launch and environment at boot time into /var/log/rpc.log

$ cat /var/log/rpc.log
rpc.idmapd called from parent PID 1
TERM=linux
PATH=/usr/local/sbin:/usr/local/bin:/usr/bin:/usr/sbin:/sbin:/bin
PWD=/
SHLVL=1
UPSTART_INSTANCE=
UPSTART_EVENTS=local-filesystems
UPSTART_JOB=idmapd
PIPEFS_MOUNTPOINT=/run/rpc_pipefs
_=/usr/bin/env

Sep 2 16:01:23 hephaestion rpc.idmapd.real[3869]: libnfsidmap: using domain: lan.eddie.tj
Sep 2 16:01:23 hephaestion rpc.idmapd.real[3869]: libnfsidmap: Realms list: 'LAN.EDDIE.TJ'
Sep 2 16:01:23 hephaestion rpc.idmapd.real[3869]: libnfsidmap: loaded plugin /lib/libnfsidmap/nsswitch.so for method nsswitch
Sep 2 16:01:23 hephaestion rpc.idmapd.real[3880]: Expiration time is 600 seconds.
Sep 2 16:01:23 hephaestion rpc.idmapd.real[3880]: Opened /proc/net/rpc/nfs4.nametoid/channel
Sep 2 16:01:23 hephaestion rpc.idmapd.real[3880]: Opened /proc/net/rpc/nfs4.idtoname/channel
Sep 2 16:01:23 hephaestion rpc.idmapd.real[3880]: main: open(/var/lib/nfs/rpc_pipefs/nfs): No such file or directory

which confirms the use of the old path.

I ran idmapd with strace to see why it is using that path and found this:

...
5026 open("/etc/idmapd.conf", O_RDONLY) = 3
...
5026 open("/var/lib/nfs/rpc_pipefs/nfs", O_RDONLY) = -1 ENOENT (No such file or directory)

The idmapd.conf is:

$ cat /etc/idmapd.conf
[General]

Verbosity = 0
Pipefs-Directory = /var/lib/nfs/rpc_pipefs
Domain = lan.eddie.tj

[Mapping]

Nobody-User = nobody
Nobody-Group = nogroup

So, "Pipefs-Directory" is the culprit. I don't know when that was set but it is obviously the reason for the issue.

I think the reason that the patched 'blkmapd' solves it is because I build and install updated packages, and the postinst job changes the idmapd.conf file.

I found this which seems to indicate a package upgrade issue but I wasn't notified:

-rw-r--r-- 1 root root 385 Apr 18 10:54 /etc/idmapd.conf.merge-error

[General]

Verbosity = 0
<<<<<<< Current
Pipefs-Directory = /var/lib/nfs/rpc_pipefs
Domain = lan.eddie.tj
||||||| Older
Pipefs-Directory = /var/lib/nfs/rpc_pipefs
Domain = localdomain
=======
Pipefs-Directory = /run/rpc_pipefs
# set your own domain here, if id differs from FQDN minus hostname
# Domain = localdomain
>>>>>>> New

[Mapping]

Nobody-User = nobody
Nobody-Group = nogroup
/etc/idmapd.conf.merge-error (END)

> So, "Pipefs-Directory" is the culprit. I don't know when that
> was set but it is obviously the reason for the issue.

Right. This was the previous default value, that was set in this file as shipped; so what's happened here is that you have a non-default 'Domain' setting, and on upgrade the package tried (and failed) to auto-merge the changes from the new version of the config file. On upgrade (with the default settings) you will have received a prompt asking you what to do with this file, and ultimately you appear to have chosen to keep the original version of the file.

So I think we want some additional upgrade handling to forcibly change the Pipefs-Directory setting, since this is going to be a common merge failure.

The blkmapd change is correct in its own right, but definitely unrelated to idmapd; we don't start blkmapd, and idmapd doesn't invoke this code.

I've moved the blkmapd patch to bug #1045104. Sorry for the noise on this one - just my luck that the observed behaviour after installing the patched package made the warnings go away!

For a time I was thinking that possibly rpcbind and friends were calling blkmapd which was returning the incorrect path which was being passed to idmapd using the -p switch. That was the reason for the shell scripts in place of the ELF daemons to see if that theory might be correct.

I don't recall getting any prompt at upgrade time, but then again the system is mostly doing unattended upgrades so that would explain it being missed by a human.

Hi! I also have the same problem and I have noticed that I configured the parameter NEED_IDMAPD with the word "YES" (uppercase).

So, when I changed to "yes" (lowercase), the idmapd service worked.

WRONG:
NEED_IDMAPD="YES"

CORRECT:
NEED_IDMAPD="yes"

;)

Status changed to 'Confirmed' because the bug affects multiple users.

I struggled with the undesired symptom that names are shown in ls -l output of an NFS filesystem as nobody, instead of the expected user name as known to the NFS server and client.

Background: I'm using 12.10 client to mount an NFS filesystem exported from a 12.04 server. I have the same users on both systems, altho with different numeric user IDs, listed in /etc/passwd. Both systems have proper /etc/hosts entries so that the command dnsdomainname returns the same string. On the client, /etc/default/nfs-common has a line "NEED_IDMAPD=yes". The filesystem mount requests are accepted.

This bug report suggested mounting the rpc-pipe filesystem manually. This was successful for me:

  $ sudo service idmapd stop
  $ sudo mount -t rpc_pipefs rpc_pipefs /run/rpc_pipefs
  $ sudo rpc.idmapd -v -f

And then the idmapd process stayed in the foreground, messages were logged to the window, and then I could debug the id<->name mapping.

When I first installed the nfs-common software on the client, I hit problems much like what is described in this bug report. Google lead me here :) For example syslog showed this entry:

Dec 27 09:58:20 earthquake rpc.idmapd[8446]: main: open(/run/rpc_pipefs/nfs): No such file or directory

Maybe I should have rebooted the machine after doing the apt-get, or some other service needed to be restarted, I just don't know. After a reboot, idmapd starts as I expect.

HTH

oneiric has seen the end of its life and is no longer receiving any updates. Marking the oneiric task for this ticket as "Won't Fix".

quantal has seen the end of its life and is no longer receiving any updates. Marking the quantal task for this ticket as "Won't Fix".

The Precise Pangolin has reached end of life, so this bug will not be fixed for that release