default Chinese font in snap apps is ugly

We should let Noto CJK has higher priority.

This bug has been reported on the Ubuntu ISO testing tracker.

A list of all reports related to this bug can be found here:
https://iso.qa.ubuntu.com/qatracker/reports/bugs/2017076

It looks like this issue mostly affects Snap apps.
Please see the font config debug output attached.

Hi, indeed I forget the store is also snap.

Status changed to 'Confirmed' because the bug affects multiple users.

@Shengjing Zhu: Can you confirm that it's only in snap applications you see UMing instead of Noto? Btw, is it limited to Snap Store, or is the FF snap affected as well?

Also: Is Noto used as expected
- when browsing web sites with Chinese contents
- in the desktop itself, e.g. Settings

@Gunnar Hjalmarsson:

I have attached the screenshot at https://bugs.launchpad.net/ubuntu/+source/fontconfig/+bug/2017076/comments/2

+ Both Firefox and snap-store are displayed with UMing
+ The webpage in Firefox is displayed with UMing
+ Non-snap applications uses Noto

One thought is to ask why we still install the fonts-arphic-uming package when installing Chinese (Simplified as well as Traditional). Probably it is of historical reasons from the time before Noto existed. But let's consider that to be a matter of its own. Even if AR PL UMing is available, Noto Sans CJK should have higher precedence.

With a Chinese locale, it's not possible to have fontconfig give Noto Sans CJK higher precedence that it currently has:

$ LC_CTYPE=zh_CN.UTF-8 fc-match
NotoSansCJK-Regular.ttc: "Noto Sans CJK SC" "Regular"

But also with non-Chinese locales, Noto Sans CJK ought to be preferred over AR PL UMing:

$ LC_CTYPE=en_US.UTF-8 fc-match -a | grep -iE 'uming|cjk' | head -38
NotoSansCJK-Regular.ttc: "Noto Sans CJK JP" "Regular"
NotoSansCJK-Medium.ttc: "Noto Sans CJK JP" "Medium"
NotoSansCJK-DemiLight.ttc: "Noto Sans CJK JP" "DemiLight"
NotoSansCJK-Light.ttc: "Noto Sans CJK JP" "Light"
NotoSansCJK-Thin.ttc: "Noto Sans CJK JP" "Thin"
NotoSansCJK-Bold.ttc: "Noto Sans CJK JP" "Bold"
NotoSansCJK-Black.ttc: "Noto Sans CJK JP" "Black"
NotoSansCJK-Regular.ttc: "Noto Sans CJK KR" "Regular"
NotoSansCJK-Medium.ttc: "Noto Sans CJK KR" "Medium"
NotoSansCJK-DemiLight.ttc: "Noto Sans CJK KR" "DemiLight"
NotoSansCJK-Light.ttc: "Noto Sans CJK KR" "Light"
NotoSansCJK-Thin.ttc: "Noto Sans CJK KR" "Thin"
NotoSansCJK-Bold.ttc: "Noto Sans CJK KR" "Bold"
NotoSansCJK-Black.ttc: "Noto Sans CJK KR" "Black"
NotoSansCJK-Regular.ttc: "Noto Sans CJK SC" "Regular"
NotoSansCJK-Medium.ttc: "Noto Sans CJK SC" "Medium"
NotoSansCJK-DemiLight.ttc: "Noto Sans CJK SC" "DemiLight"
NotoSansCJK-Light.ttc: "Noto Sans CJK SC" "Light"
NotoSansCJK-Thin.ttc: "Noto Sans CJK SC" "Thin"
NotoSansCJK-Bold.ttc: "Noto Sans CJK SC" "Bold"
NotoSansCJK-Black.ttc: "Noto Sans CJK SC" "Black"
NotoSansCJK-Regular.ttc: "Noto Sans CJK TC" "Regular"
NotoSansCJK-Medium.ttc: "Noto Sans CJK TC" "Medium"
NotoSansCJK-DemiLight.ttc: "Noto Sans CJK TC" "DemiLight"
NotoSansCJK-Light.ttc: "Noto Sans CJK TC" "Light"
NotoSansCJK-Thin.ttc: "Noto Sans CJK TC" "Thin"
NotoSansCJK-Bold.ttc: "Noto Sans CJK TC" "Bold"
NotoSansCJK-Black.ttc: "Noto Sans CJK TC" "Black"
NotoSansCJK-Regular.ttc: "Noto Sans CJK HK" "Regular"
NotoSansCJK-Medium.ttc: "Noto Sans CJK HK" "Medium"
NotoSansCJK-DemiLight.ttc: "Noto Sans CJK HK" "DemiLight"
NotoSansCJK-Light.ttc: "Noto Sans CJK HK" "Light"
NotoSansCJK-Thin.ttc: "Noto Sans CJK HK" "Thin"
NotoSansCJK-Bold.ttc: "Noto Sans CJK HK" "Bold"
NotoSansCJK-Black.ttc: "Noto Sans CJK HK" "Black"
uming.ttc: "AR PL UMing CN" "Light"
uming.ttc: "AR PL UMing HK" "Light"
uming.ttc: "AR PL UMing TW" "Light"

But apparently the snap applications don't query fontconfig dynamically about the default font. Instead a default font seems to be established during installation, based on to me unknown criteria, and that default font does not change.

We need help from the snap folks to sort this out, I think.

Another thought is that many available fontconfig files were moved from /etc/fonts/conf.avail to /usr/share/fontconfig/conf.avail during the lunar development cycle. Can this possibly have created a dissonance between the snaps and the system in this respect? If I understand it correctly...

Adding the language-selector package, since many fontconfig files affecting Chinese rendering are provided by language-selector-common.

I had an IRC conversion with Sebastien Bacher and learned that things no longer work as I thought they did. Unlike what I thought, the snaps do no longer honor the font configuration of the host system, but live in their own world in this respect.

https://github.com/ubuntu/gnome-sdk/issues/49#issuecomment-1216321729

So, what's determining the default font for a snap application is this list:

https://github.com/ubuntu/gnome-sdk/pull/72/files

There is an apparent need to reconsider that list, but that's a separate thing. Let's first try to understand how the specific issue reported in this bug could happen.

The list includes the fonts-noto-cjk package, which also installs 70-fonts-noto-cjk.conf. The latter includes:

    <match target="pattern">
        <test name="lang">
            <string>zh-cn</string>
        </test>
        <test name="family">
            <string>sans-serif</string>
        </test>
        <edit name="family" mode="prepend" binding="strong">
            <string>Noto Sans CJK SC</string>
        </edit>
    </match>

Well, fonts-arphic-uming is also included currently, and it installs 65-fonts-arphic-uming.conf which includes:

    <alias>
        <family>sans-serif</family>
        <prefer>
            <family>AR PL UMing CN</family>
            <family>AR PL UMing HK</family>
            <family>AR PL UMing TW</family>
        </prefer>
    </alias>

Note that this one is not language guarded, so it's effective irrespective of locale.

But still, with the zh_CN.UTF-8 locale 70-fonts-noto-cjk.conf should safely result in Noto Sans CJK SC being preferred over AR PL UMing CN. To confirm that I installed the Chinese (Simplified) language in 23.04 and purged language-selector-common (which made gnome-control-center being uninstalled...). And yes:

$ LC_CTYPE=zh_CN.UTF-8 fc-match
NotoSansCJK-Regular.ttc: "Noto Sans CJK SC" "Regular"

More playing with fc-match:

$ LC_CTYPE=zh_CN.UTF-8 fc-match -a | grep -iE 'cjk sc|uming cn' | head -3
NotoSansCJK-Regular.ttc: "Noto Sans CJK SC" "Regular"
NotoSansCJK-Bold.ttc: "Noto Sans CJK SC" "Bold"
uming.ttc: "AR PL UMing CN" "Light"
$ LC_CTYPE=en_US.UTF-8 fc-match -a | grep -iE 'cjk sc|uming cn' | head -3
uming.ttc: "AR PL UMing CN" "Light"
NotoSansCJK-Regular.ttc: "Noto Sans CJK SC" "Regular"
NotoSansCJK-Regular.ttc: "Noto Sans Mono CJK SC" "Regular"

So I now have a (vague) theory about what happens when you install Ubuntu using the zh_CN.UTF-8 locale:

While it honors the locale when determining which packages get installed (apparently, since fonts-arphic-uming is there), the locale is not properly taken into account at the stage when the default Chinese (Simplified) font of the snaps is determined.

Do you get the same issue on core20 snaps (like firefox from the stable channel)?

@seb128, I think you mean esr/stable channel?

esr channel which uses core20 is using Noto font.

See the attachment.

This is confusing. If you install Ubuntu 23.04, you get FF from latest/stable/23.04 together with core22.

Now I did a test where I removed FF, set the zh_CN.UTF-8 locale, and installed FF again. Whichever channel I pick (tried latest/stable, latest/beta and latest/edge) it pulls core20, and then it defaults to Noto CJK fonts.

So I wasn't able to reproduce the issue with "AR PL UMing" as default this way. I get the same result as Shengjing Zhu got when installing from esr.

@Gunnar, that's because the only channel which has the core22 firefox atm is stable/ubuntu-23.04 (which is specific to Lunar). We transitioned Lunar to core22 but we still need to work with Mozilla on updating the other channels

Your testing suggest that the issue was the one I mentioned earlier that https://github.com/ubuntu/gnome-sdk/pull/72 is missing from the core22 based gnome sdk which I'm going to propose a fix for

Oh, I miss that the lunar desktop image installs firefox snap from latest/stable/ubuntu-23.04 branch.

So, if I install firefox from latest/stable channel (which uses core20), then Noto CJK is rightfully selected.

On 2023-04-21 21:18, Sebastien Bacher wrote:
> @Gunnar, that's because the only channel which has the core22
> firefox atm is stable/ubuntu-23.04 (which is specific to Lunar).

So, then I just learned that if you are on lunar aka 23.04 and install firefox from the latest/stable channel, you get something different compared to installing it from the latest/stable/ubuntu-23.04 channel. Hmm.. I recall someone who talked earlier today about the need to be an expert... ;)

> Your testing suggest that the issue was the one I mentioned earlier
> that https://github.com/ubuntu/gnome-sdk/pull/72 is missing from the
> core22 based gnome sdk

Right, by doing

snap refresh firefox --channel=latest/stable/ubuntu-23.04

(with the Chinese locale) I could finally reproduce the default font issue. So that seems to be it for this bug.

This exercise raised a few other font related issues in my head, but they are less urgent. Let's take those separately later.

I've proposed a fix for the core22 serie now on https://github.com/ubuntu/gnome-sdk/pull/143

This seems to be fixed in Firefox latest/stable/ubuntu-23.04 114.0.1.

@Sebastien: But that doesn't fix the 23.04 installer, does it?

@Gunnar, what's the issue with the installer? it's the first time the installer is mentioned on that report so I'm not sure to understand the question

The installer is mentioned in the first sentence in the bug description. :)

I was thinking that the installer still ships the old snap without the fix. Then, if the user opens FF before refreshing the snaps, they would still have the wrong default in FF.

Maybe I'm wrong. I haven't done any tests.

> The installer is mentioned in the first sentence in the bug description. :)

No it isn't? The description states 'After installs Ubuntu 23.04 with Chinese language' which refers to the installed system and not the installer? (just to be clear to me 'installer' is ubuntu-desktop-installer in lunar which is a flutter UI distributed as a snap that let you install Ubuntu)

The first screenshot after the description is one from firefox showing an issue, reading the comments my understanding is that the report was about the firefox issue which has been resolved now

> The first screenshot after the description is one from firefox showing an issue, reading the comments my understanding is that the report was about the firefox issue which has been resolved now

Yes :)

@Sebastien: Sorry for confusing the installer with the ISO (which is used for installing...).

My point was that at first login after a Chinese install, you have the versions of firefox and core22 from April, and thus see the issue if you open FF before having refreshed the snaps.

One concern of mine was that I feared that the FF default font would be stuck somehow at the value when FF was first opened. ( Yes, I'm a snap illiterate. :/ ) But I just tested, and saw that the FF default font changed to the expected one once I had run "snap refresh".

So yes, the issue has been resolved.

No worry, now things make sense, thanks for testing and confirming it's fixed!

Hi all,

Does this fix affect Japanese fonts?

I installed Ubuntu 23.04 today and encountered a problem where the Japanese in the snap package is incorrectly displayed with Chinese font glyphs.

Specifically, the problem occurs in Firefox and Snap Store, but not in gnome-terminal or Text Editor.

I have attached a screenshot that shows the difference between Ubuntu 22.04 LTS and Ubuntu 23.04. The letters circled in red should originally look the same in 22.04 and 23.04.

@Mitsuya Shibata: Well, yes it *affects* Japanese fonts too, but it makes no sense that *this change* would increase the risk of picking Chinese glyphs under a Japanese locale.

OTOH, if I start Firefox with:

env LC_CTYPE=ja_JP.UTF-8 firefox

the default sans-serif font for me according to FF Settings is Droid Sans Fallback!! That might have something to do with it.

I see now that the fonts-droid-fallback package is shipped on the ISO (did we simply forget to drop it?). So as a test, can you please uninstall that package:

sudo apt purge fonts-droid-fallback

and let us know if it makes a difference.

In any case I think there is a need to reconsider the default font configuation for Japanese. The /usr/share/fontconfig/conf.avail/70-fonts-noto-cjk.conf file includes:

    <match target="pattern">
        <test name="lang">
            <string>ja</string>
        </test>
        <test name="family">
            <string>sans-serif</string>
        </test>
        <edit name="family" mode="prepend">
            <string>Noto Sans CJK JP</string>
        </edit>
    </match>

while the Chinese equivalents have:

        <edit name="family" mode="prepend" binding="strong">

So, Mitsuya Shibata, it would be great if you could submit a new bug where we can focus on the Japanese font configuration. The reconsideration would probably affect multiple packages, but language-selector may be a proper affected package to start with.

@Gunnar

Thanks for the advice. I removed fonts-droid-fallback and tried it, and now Japanese glyphs are displayed.

However, I am concerned that it only affects the snap package, while the non-snap package seems to select the correct glyphs.

I will first check if this also happens in Mantic and then report the problem to language-selector.

> I am concerned that it only affects the snap package, while
> the non-snap package seems to select the correct glyphs.

Yes, that's what it looks like.

$ lsb_release -c
No LSB modules are available.
Codename: mantic
$ dpkg-query -W fonts-droid-fallback
fonts-droid-fallback 1:6.0.1r16-1.1build1
$ snap run --shell firefox
To run a command as administrator (user "root"), use "sudo <command>".
See "man sudo_root" for details.

/home/gunnar$ LC_CTYPE=ja_JP.UTF-8 fc-match sans-serif
DroidSansFallbackFull.ttf: "Droid Sans Fallback" "Regular"
/home/gunnar$ exit
exit
$ LC_CTYPE=ja_JP.UTF-8 fc-match sans-serif
NotoSansCJK-Regular.ttc: "Noto Sans CJK JP" "Regular"

So the font configuration of the snaps prioritizes differently compared to the font configuration of the system. Reason unclear to me ATM.

I also checked in Debian testing, where Firefox is conventionally packaged. Starting FF there with the Japanese locale results in the default sans-serif font "VL Gothic". But that's what's expected from the system font configuration:

$ LC_CTYPE=ja_JP.UTF-8 fc-match sans-serif
VL-Gothic-Regular.ttf: "VL ゴシック" "regular"

(Debian with GNOME has a lot of fonts installed.)

I think the reason why fonts-droid-fallback is installed by default is this dependency chain:

ubuntu-desktop -> ghostscript-x -> ghostscript -> libgs10 -> libgs10-common -> (recommends) fonts-droid-fallback

A fragile workaround might be to somehow stop fonts-droid-fallback from being pulled (change libgs10-common so it Suggests instead of Recommends fonts-droid-fallback). Or maybe we could drop the font configuration file from the fonts-droid-fallback package.

OTOH, neither would address the root cause — why do the snaps behave as they do with respect to font configuration?

Could you use a new bug to discuss the japanese issue?

It could help also to know if you are using the core20 or core22 version of firefox and they have the same issue. Is the configuration in the snap environment matching the real system one?

On 2023-07-03 11:08, Sebastien Bacher wrote:
> Could you use a new bug to discuss the japanese issue?

Yes, Mitsuya plans to submit a new bug once he has checked mantic. (I have used this one in the meantime to share a couple of thoughts.)

> It could help also to know if you are using the core20 or core22
> version of firefox and they have the same issue.

Mitsuya doesn't see it in jammy, and neither do I:

$ snap run --shell firefox
To run a command as administrator (user "root"), use "sudo <command>".
See "man sudo_root" for details.

/home/gunnar$ LC_CTYPE=ja_JP.UTF-8 fc-match sans-serif
NotoSansCJK-Regular.ttc: "Noto Sans CJK JP" "Regular"

Even if fonts-droid-fallback is present there too.

(But I notice another weird issue: In the FF Settings it shows Ubuntu as the default serif font irrespective of locale. How come?)

> Is the configuration in the snap environment matching the real
> system one?

It is not per se, is it? At least not in lunar and mantic, where we use an upgraded fontconfig in the system.

Sorry for late response. I filed bug #2025651.

On 2023-07-03 12:59, Gunnar Hjalmarsson wrote:
> (But I notice another weird issue: In the FF Settings it shows Ubuntu
> as the default serif font irrespective of locale. How come?)

Pls disregard that. User error.