pam configuration for SSH prevents LANG override

Created attachment 1332
client-sent environment overrides PAM-read environment

This bug has been reported and discussed in the Debian BTS, see bugs #313317 and #408029 there.
The environment variables sent by AcceptEnv/SendEnv functionalities
should take precedence over PAM variable settings, especially for
locale and terminal related settings (or commands that are
locale-sensitive or terminal sensitive might give incomprehensible
gibberish as output to the user). TERM is already managed in a special
way, but not LANG or LC_* variables.
Currently, the variables LANG and LC_* are set (in a default debian
installation) by pam (/etc/pamd.d/ssh) which in turn reads
/etc/environment and /etc/default/locale. It happens dans in session.c
(function do_child) the environment of the child process is set as
follows: first, copy the environment set by AcceptEnv/SendEnv, set some
more variables (TERM, TZ, depending on the system), then use pam and
copy the PAM environment inside the child environment, thus clobbering
the useful variables sent through AcceptEnv/SendEnv.

Note that there is no way it could be fixed at the PAM level: PAM
prepares the environment for the child not knowing the sent variables.
It is openssh-server that does the things in the wrong order.

What the patch does: it changes the child_set_env function in
copy_environment to child_set_env_safe (basically the same as
child_set_env but with a twist): any variable which has already been
inserted in the environment is not clobbered by copy_environment.

Since the function copy_environment is the one used to bring the PAM
settings inside the environment, the PAM settings no more clobber the
environment sent by the AcceptEnv/SendEnv mechanism. Which yields
(from a client with LANG unset, and to a server with LANG=fr_FR.UTF-8 in
/etc/default/locale)

$ ssh penpen 'echo $LANG $(locale charmap)'
fr_FR.UTF-8 UTF-8
$ LANG=en_GB.UTF-8 ssh penpen 'echo $LANG $(locale charmap)'
en_GB.UTF-8 UTF-8
$ LANG=fr_FR@euro ssh penpen 'echo $LANG $(locale charmap)'
fr_FR@euro ISO-8859-15
$ LANG=fr_FR ssh penpen 'echo $LANG $(locale charmap)'
fr_FR ISO-8859-1

Since the current behaviour is to enforce the admin-set values, and thus
rendering the AcceptEnv/SendEnv almost useless, since critical variables
set in the environment can be enforced by the administrator by refusing
to accept them (in /etc/ssh/sshd_config) and since the default-accepted
environment variables are only limited to locale-related variables and
a default debian installation does not allow those variables to be used
(the "locales" package always sets LANG in /etc/default/locale), I think
this patch is worth being included in openssh-server. I also think it
free of security holes or memory leaks. I think it is worth being
transmitted upstream. I think some consideration should be given about
whether the "no clobber" behaviour should be the default one
(child_set_env is used several times in session.c and some should
probably consider using child_set_env_safe with the same rationale), but
it is part of a more general reflexion on this and does not interfere in
any way with these two bugs.

Why is PAM setting these variables? Wouldn't it be better to make PAM not set things that you do not want overridden?

It isn't totally obvious why env vars from PAM should not get precedence, as we generally treat SendEnv strings as the lowest priority.

A sysadmin, in a non-English speaking country, should set default values (especially for locale related environment values). The natural way to set these is through PAM (works across many shells). I also implemented a no-clobber pam environment setting, but it will not work for ssh AcceptEnv/SendEnv because (IIRC) the environment passed to pam for initialisation is empty - it does not contain pre-set variables with the ssh AcceptEnv/SendEnv variables set. This solution implies to change 2 major components of a Unix system (openssh + PAM) instead of only one (openssh). I also do not see how being conservative (putting AcceptEnv on low priority) is helpful, especially in the case of locales; if I remotely log into a japanese system, I want to be able to type in French and read messages in French). The locales system degrades correctly if the locale does not exist on the target system; the user does not degrade correctly if the remote language is not understood.

In short, I would like some kind of way to have remotely sent variables override any defaults set by PAM. Maybe not all variables, but imagine that for some users, having some incorrect LANG is the same as having the wrong TERM variable (and TERM is dealt with specially in openssh).
Please consider that LC_CTYPE/LANG is as important as TERM as soon as you are not in a pure ASCII language.

Of course, your position is somehow sensible if those were random variable names. But the names for the locale-setting vars are fixed, must be able to get defaults from pam, and must be overriden by the user.

There is always the possibility of local hacks (AcceptEnv/SendEnv on SSH_LANG, use the shell to copy LANG to SSH_LANG if it is not set and from SSH_LANG to LANG if SSH_LANG is set, using the various level of initialisation), but it is really cumbersome.

It is not also obvious why AcceptEnv/SendEnv should not get precedence.
After all, both sysadmins have to agree on which variables are acceptable for import/export in the environment. The default on all computers that I know of is "LANG LC_*". So if ssh cannot export LANG LC_* as soon as there is some default set in PAM (and almost all distributions do that), there is no use of this feature possible by default. Of course, you are free to think that there should be no default set by PAM, but think that in a non-English speaking environment, something has to be done to put a default language (say, French, Japanese or Latvian) for all "normal" users that use the regular login (so that they can read their mails in the correct encoding, almost all programs now read LANG or LC_CTYPE to know which encoding to use).

It probably makes sense to "generally treat SendEnv strings as the lowest priority". However, it also makes sense to allow clients to request locale settings, especially because it makes the ongoing transition to utf-8 much easier.

I therefore suggest to add SendLocale/AcceptLocale configuration options to OpenSSH. SendLocale would probably just be an alias for "SendEnv LC_* LANG", but AcceptLocale could handle the locale environment variables differently (maybe reset them after pam was called).

On 7 Oct 2004 I publish one article : http://roumenpetrov.info/articles/locale_env_in_ssh_session.html

Now I could not remember the tested linux systems. Some use pam some not, but problem to me was in shell scripts executed at login time.

As examlpe (if I remember well) on redhat based linux problem is in the file /etc/profile.d/lang.sh and not in pam module.
Also I didn't test debian based but most of them set LANG in /etc/environment and this setting override already set variable.

To me this is not OpenSSH bug. The reporters has to resolve vendor issue first, i.e. pam module or login script to preserve environment variables related to locale settings.

P.S. I vote to close - reason not openssh bug

(In reply to comment #4)
> On 7 Oct 2004 I publish one article :
> http://roumenpetrov.info/articles/locale_env_in_ssh_session.html

This article contains incorrect information:

  "Bourne" like shells:
  All shells read system-wide startup file /etc/profile.

With bash, this is wrong. Only *login* shells read it. Non-login shells may read bashrc files, but not always.

(In reply to comment #4)
> To me this is not OpenSSH bug. The reporters has to resolve vendor
> issue first, i.e. pam module or login script to preserve environment
> variables related to locale settings.

Concerning the pam module under Debian, this can be seen as a configuration bug, that can easily be fixed (under Debian 4.0, a.k.a. etch). First, if /etc/environment contains locale-related settings, they should be moved to /etc/default/locale as documented. Then the line

auth required pam_env.so envfile=/etc/default/locale

should be commented out since locale-related environment variables should be passed via SendEnv (I don't think that providing defaults other than POSIX or C is correct anyway, since there's no guarantee that specific locales will be compatible with the user's settings on the client side, and bad settings can trash the terminal).

Now, I'm not convinced that this is not *also* an OpenSSH bug. Said otherwise, what should come first? PAM or SendEnv settings? The current behavior is that SendEnv settings come first, then PAM can override these settings (via sshd-specific configuration, e.g. /etc/pam.d/sshd). If I understand correctly, according to Jean-Christophe Dubacq, PAM settings should come first (then they can be overridden by SendEnv). But I don't see any real reason for that, except that the user has more control with SendEnv, thus these settings should be applied later.

(In reply to comment #6)
> (In reply to comment #4)
> > On 7 Oct 2004 I publish one article :
> > http://roumenpetrov.info/articles/locale_env_in_ssh_session.html
>
> This article contains incorrect information:
>
> "Bourne" like shells:
> All shells read system-wide startup file /etc/profile.
>
> With bash, this is wrong. Only *login* shells read it. Non-login shells
> may read bashrc files, but not always.

What about to read again article ? Take care on subtitle.

If we use shadow passwords the file /etc/login.defs contain a line:
ENVIRON_FILE /etc/environment.
Variables from this file override unconditionally variables already set.
As is reported similar is for ... pam_env.so envfile=

This is the one of issues that block to pass locale variables to remote host. It is vendor/administrator problem.

(In reply to comment #8)
> What about to read again article ? Take care on subtitle.

But then your article is a bit restrictive: one also wants to pass locales with non-login shells (e.g., if a command is given to ssh).

As said by comment #7, the question is whether a user-set variable can override an admin-set default value. /etc/environment is there for default value. Currently, every setting there overrides SendEnv/AcceptEnv passed value. This is what I think is the bug. In non-English speaking countries, all computers are set to have a default value for locale (the local language, be it French or Spanish or Japanese). This value is useful for display login managers, and even some tools that are daemonized. It is also useful for tty login (you do not want each and every of your users to have to set their .somethingshrc to the correct value). So there is (almost always) a default value read (often through pam).

Passing SendEnv/AcceptEnv in priority to environment is important, because of the great split between UTF-8 graphical environments and ISO-8859-1 environments. Most environments are capable to cope with both now, but this requires the locale to be set to the correct value for the terminal of the viewer. And this can only be found by passing prioritarily the value of LANG (and overriding the default local value).

The importance of this bug is probably not seeable for people living in English-speaking countries, or people where all default values use the same encoding.

The usual logic in administration of computer systems is that user can override the admin "default" settings (or the settings are made in "mandatory" mode). Locale setting is not something that should be considered mandatory, but really default.

If locale-related environment variables were all that were handled by SendEnv/PAM then there would be no argument - we would happily allow SendEnv to take precedence. Unfortunately this is not the case and I generally think that env variables specified by a security subsystem (e.g. PAM) should "win". Given that an administrator must explicitly enable AcceptEnv anyway, this doesn't seem too unreasonable.

As a workaround, you can "rename" the locale variables at the client. E.g. by placing SSH_LANG=$LANG in your environment and SendEnv'ing SSH_LANG instead of LANG. At the sshd end, your shell initialisation can look for SSH_LANG and, if it is present, apply it - potentially overriding anything set by the admin.

Finally, if I understand correctly, there is little standardisation of the contents of LANG and LC_* so sending anything in them is bound to be fragile.

'little standardisation'? locales ought to be la_CO[@variant].charset ; with la and CO being ISO639 and ISO3166. The availability of a locale on a given system is not guaranteed; however, ti is very usual to install in some country the locale for this country's main language in both charsets (old 8-bit and UTF-8, really).

pam_environment is perceived more as a default setting than a mandatory one. And the fact that AcceptEnv is set by admin _and_ that the list of accepted variables is a closed list should be enough to convince you that AcceptEnv variables will not override _mandatory_ variables that should be set by the admin only, never to be overriden: the admin would simply remove these variables from the AcceptEnv list. List that, by the way, on all linux systems I know (several distributions), contains by default only... LANG and LC_* (when enabled).

Getting around this bug by using another variable is really not a good way. Many shells have different initialization paths (even for the same shell: login and not login shells). pam_environment is the way to set sane defaults; sshd should override the user-specified variables.

Maybe there should be an option to set, so that the admin can choose whether pam should come first or not. I can code that.

(In reply to comment #13)
> Getting around this bug by using another variable is really not a good
> way. Many shells have different initialization paths (even for the same
> shell: login and not login shells).

Even no initialization at all (e.g. "ssh -t host command" with upstream bash on the remote host).

> Maybe there should be an option to set, so that the admin can choose
> whether pam should come first or not. I can code that.

Shouldn't there be a way to do both? e.g.
1. PAM sets defaults.
2. SendEnv/AcceptEnv variables are overridden by sshd.
3. PAM does locale-name conversions, if possible (because the local name conventions on the local and remote hosts are not the same).

(In reply to comment #10)
> (In reply to comment #8)
> > What about to read again article ? Take care on subtitle.
>
> But then your article is a bit restrictive: one also wants to pass
> locales with non-login shells (e.g., if a command is given to ssh).

"non-login" shells works for me.

(In reply to comment #13)
> 'little standardisation'? locales ought to be la_CO[@variant].charset ;

"little standardization" this is true. Even on linux you could set LANG to full language name, i.e. not only if format above.

> [SNIP]
> pam_environment is perceived more as a default setting than a
> mandatory one.

Sentence above confirm that reported issue is configuration problem - system default setting override already set variable !!!!
[SNIP]

The alias capability is a glibc feature, indeed. I found it quite reliable, but the real locale is la_CO.charset. This works correctly on Linux and MacOSX and, I presume on BSD, since MacOSX is derived from it. Working on Linux/MacOSX/BSD is enough of a standard for me.

> Sentence above confirm that reported issue is configuration problem -
> system default setting override already set variable !!!!

This is because there is no known way to set variable AFTER default setting.

If you look at session.c around line 1265 (openssh 5.1p1), you will see that the variables sent by SendEnv are put in a new environment, THEN the pam environment is copied over this environment. This means that pam never sees the SendEnv variables (which would be another solution to the problem, since pam_env has a syntax for not overriding a preset variable). However, when the pam authentication is done, the variables are not set (and pam does not see them).

(In reply to comment #17)
> The alias capability is a glibc feature, indeed. I found it quite
> reliable, but the real locale is la_CO.charset. This works correctly on
> Linux and MacOSX and, I presume on BSD, since MacOSX is derived from
> it. Working on Linux/MacOSX/BSD is enough of a standard for me.

If remember well OS X (10.2 ) use bulgarian, not bg_BG[..]

> > Sentence above confirm that reported issue is configuration problem -
> > system default setting override already set variable !!!!
>
> This is because there is no known way to set variable AFTER default
> setting.

"default settings" mean : set if is not set !!!

(In reply to comment #15)
> (In reply to comment #10)
> > (In reply to comment #8)
> > > What about to read again article ? Take care on subtitle.
> >
> > But then your article is a bit restrictive: one also wants to pass
> > locales with non-login shells (e.g., if a command is given to ssh).
>
> "non-login" shells works for me.

Upstream bash doesn't with ssh -t: this is a well-known bug^Wfeature.

(In reply to comment #19)
> (In reply to comment #15)
> > (In reply to comment #10)
> > > (In reply to comment #8)
> > > > What about to read again article ? Take care on subtitle.
> > >
> > > But then your article is a bit restrictive: one also wants to pass
> > > locales with non-login shells (e.g., if a command is given to ssh).
> >
> > "non-login" shells works for me.
>
> Upstream bash doesn't with ssh -t: this is a well-known bug^Wfeature.

Since my version bash work fine, please show me the bug report.
If the bash think that this is correct I will set user shell to /bin/sh. If a linux distribution don't work with this there is a trash-can. For the bogus bash many users may prefer something fast.

(In reply to comment #20)
> Since my version bash work fine, please show me the bug report.

The bug I reported on the OpenSUSE BTS:
  https://bugzilla.novell.com/show_bug.cgi?id=345570
OpenSUSE developers eventually fixed it (like what Debian did in the past).

The bug I reported on the bug-bash list (two URL's since they broke the thread):
  http://groups.google.com/group/gnu.bash.bug/browse_thread/thread/d058d7573e051474
  http://groups.google.com/group/gnu.bash.bug/browse_thread/thread/42050a6c470acec8
And you can see that upstream don't want to fix it.

But this broken bash behaviour (to behave differently if SSH2_CLIENT/SSH_CLIENT is defined) was dropped before 6-7 years !

If you execute command with bash then file ~/.bashrc isn't processed.
It is fine bash to do same for ssh command - it is expected to be rejected from bash team. Ask you vendor to drop too.

(In reply to comment #22)
> If you execute command with bash then file ~/.bashrc isn't processed.
> It is fine bash to do same for ssh command - it is expected to be
> rejected from bash team. Ask you vendor to drop too.

There is a difference: when one executes a command with bash, the environment is already set up. When executing a command with ssh, it is useful to set up the environment before executing the command (AcceptEnv/SendEnv are too limited, since one generally wants an environment that is different from the client one -- in particular for the locales, which are not the same on the client side and on the server side). That's why it is useful to process the .bashrc file, and this is what Debian has chosen. This is the only way I can have correct locales (those I really want) on the remote host.

I'm able to use my locale environment on remote host with any shell available on Linux and in all cases without code modification.

(In reply to comment #24)
> I'm able to use my locale environment on remote host with any shell
> available on Linux and in all cases without code modification.

On *your* Linux machines. You can see here that this doesn't always work:

brocoli:~> ssh doublejack locale
Connected to doublejack (from 140.77.14.28)
LANG=POSIX
LC_CTYPE="POSIX"
LC_NUMERIC="POSIX"
LC_TIME=en_DK
LC_COLLATE="POSIX"
LC_MONETARY="POSIX"
LC_MESSAGES="POSIX"
LC_PAPER="POSIX"
LC_NAME="POSIX"
LC_ADDRESS="POSIX"
LC_TELEPHONE="POSIX"
LC_MEASUREMENT="POSIX"
LC_IDENTIFICATION="POSIX"
LC_ALL=

brocoli:~> ssh -t doublejack locale
Connected to doublejack (from 140.77.14.28)
LANG=
LC_CTYPE="POSIX"
LC_NUMERIC="POSIX"
LC_TIME="POSIX"
LC_COLLATE="POSIX"
LC_MONETARY="POSIX"
LC_MESSAGES="POSIX"
LC_PAPER="POSIX"
LC_NAME="POSIX"
LC_ADDRESS="POSIX"
LC_TELEPHONE="POSIX"
LC_MEASUREMENT="POSIX"
LC_IDENTIFICATION="POSIX"
LC_ALL=
Connection to doublejack closed.

No LC_TIME=en_DK this time...

Happy New Year !

Vincent, please confirm that you client (see config) send variables for host doublejack.

Happy New Year!

(In reply to comment #26)
> Vincent, please confirm that you client (see config) send variables for
> host doublejack.

No, but that's not the point (I was using this machine only because this is the only one where I have an account and that has a non-modified bash). On any machine I *need* to modify the variables passed by the client, because their values are system-dependent (even between Linux-based machines). As variable-transformation will never be implemented in OpenSSH, this has to be done via the shell.

No idea what is problem.
Today I perform tests with Ubuntu 8.10 (debuan based distro). Before to run tests LANG was removed from /etc/environment and /etc/default/locale. After modification last file left empty.
Test pass when sshd was run with and without privilege separation and for client - with and without pseudo tty-allocation (-t flag).

First time I see same output as you but it was my failure as I forgot to enable client to send locale variables to my remote host (SendEnv LC_* and SendEnv LANG on separate lines).

Also test was performed with invalid locale (LANG=TESTING ssh ...). In this case remove host warn that locale don't exists.

(In reply to comment #28)
> Also test was performed with invalid locale (LANG=TESTING ssh ...). In
> this case remove host warn that locale don't exists.
Say, locale TESTING exists on the client-side, but you wish to replace this value on the server side (because, as I said, the locale-name conventions are not always the same on the client and on the server). Can you do that?

Anyway, if LANG is not set in /etc/environment and /etc/default/locale, then there is no way the administrator can set a _default_ value for LANG. This may be fine if English is the default language, but not in other parts !

(In reply to comment #30)
> Anyway, if LANG is not set in /etc/environment and /etc/default/locale,
> then there is no way the administrator can set a _default_ value for
> LANG. This may be fine if English is the default language, but not in
> other parts !
This is Debuan design issue. Other Linux distro use /etc/profile.d/lang.{c}sh.

(In reply to comment #31)
> This is Debuan design issue. Other Linux distro use
> /etc/profile.d/lang.{c}sh.

But this means no default LANG settings with "ssh -t" when the remote shell is an unmodified bash. So, whatever solution is chosen, there's currently always a drawback somewhere.

(In reply to comment #29)
> (In reply to comment #28)
[SNIP]

Above (LANG=TESTING...) was just an example to test what is result if
is set an invalid locale on remote (not remove :) host.
Which platform use locale-name conventions different from those
implemented in GNU libc ? How this platform work with X window ?

(In reply to comment #32)
> (In reply to comment #31)
> > This is Debuan design issue. Other Linux distro use
> > /etc/profile.d/lang.{c}sh.
>
> But this means no default LANG settings with "ssh -t" when the remote
> shell is an unmodified bash. So, whatever solution is chosen, there's
> currently always a drawback somewhere.

Its work with all shells !

(In reply to comment #33)
> Which platform use locale-name conventions different from those
> implemented in GNU libc ?

The GNU libc itself uses different conventions that depend on the platform. On Nokia's internet tablets (Maemo), "en_US" means en_US with a UTF-8 charmap. But on other platforms, "en_US" means en_US with an ISO-8859-1 charmap.

Solaris and HP-UX also have different conventions from Linux. For instance, whereas en_US.ISO8859-1 works on most platform, I need iso_8859_1 on Solaris and en_US.iso88591 on HP-UX.

> How this platform work with X window ?

I don't see any problem with X window.

(In reply to comment #34)
> Its work with all shells !

No, not bash. I suppose that you do not have a recent bash or have a modified bash.

Created attachment 1894
first test

this is first attachment and i dont know what i do but only try what hapend

The content of attachment 1894 has been deleted by
    Damien Miller <email address hidden>
who provided the following reason:

Please do not upload malware.

The token used to delete this attachment was generated at 2011-06-06 21:14:43 EST.

So we could define a channel request that could be interpreted by the server to attempt to set the locale-related environment variables. How about something like:

      byte SSH_MSG_CHANNEL_REQUEST
      uint32 recipient channel
      string "<email address hidden>"
      string language
      string charset

Where language might be "en_GB" and charset might be "UTF-8". From this the server could somehow search the local locale database for a good fit.

An alternative would be to special-case LANG and LC_* and do the translation implicitly.

Hello,

is there a workaround for this issue?
I want to be able to send some VAR* variables but it seems impossible at the moment.

I have tried many different configurations.

Is there any PAM conf change I can do to disable this behaviour and allow me to inject variables?

Regards
Alessandro