Project

General

Profile

Bug #5434

Locale problem

Added by Nikolay Yankin 7 months ago. Updated 28 days ago.

Status:
Resolved
Priority:
Immediate
Assignee:
-
Category:
-
Target version:
Start date:
02.02.2018
Due date:
% Done:

0%

Estimated time:
Sorting:
Commit Number:
Affected Version:

Description

Same problem as described here https://issues.rhodecode.com/issues/5297 on Gentoo x64

Problem was appeared after upgrading from 4.9.1 to 4.11.1 (4.10.6 also affected)

log.txt (6.43 KB) log.txt Nikolay Yankin, 02.02.2018 14:53

History

#1 Updated by Nikolay Yankin 7 months ago

I tried locales en_US.UTF-8 and ru_RU.UTF-8, both correctly installed in my system.

#2 Updated by Marcin Kuzminski [staff] 7 months ago

have you tried to set LC_ALL ? to specific locale on your machine

What's the output of locale

#3 Updated by Marcin Kuzminski [staff] 7 months ago

there were 0 changes on this code since 4.9, so i expect env changes that caused this... does install 4.9.0 alongside not have this problem?

#4 Updated by Nikolay Yankin 7 months ago

repo ~ # locale
LANG=en_US.utf8
LC_CTYPE="en_US.utf8"
LC_NUMERIC="en_US.utf8"
LC_TIME="en_US.utf8"
LC_COLLATE="en_US.utf8"
LC_MONETARY="en_US.utf8"
LC_MESSAGES="en_US.utf8"
LC_PAPER="en_US.utf8"
LC_NAME="en_US.utf8"
LC_ADDRESS="en_US.utf8"
LC_TELEPHONE="en_US.utf8"
LC_MEASUREMENT="en_US.utf8"
LC_IDENTIFICATION="en_US.utf8"
LC_ALL=

#5 Updated by Nikolay Yankin 7 months ago

Simple script

import locale
loc_list = [(a,b) for a,b in locale.locale_alias.items() ]
loc_size = len(loc_list)
print loc_size,'entries'

for loc in loc_list:
    try:
        locale.setlocale(locale.LC_TIME, loc[1])
        print 'SUCCES set {:12} ({})'.format(loc[1],loc[0])
    except:
        pass

ran by python test.py shows

970 entries
SUCCES set C            (c.ascii)
SUCCES set C            (c.en)
SUCCES set C            (c)
SUCCES set C            (posix-utf2)
SUCCES set C            (c_c)
SUCCES set C            (c_c.c)
SUCCES set en_US.UTF-8  (universal.utf8@ucs4)
SUCCES set C            (posix)
SUCCES set C            (english_united-states.437)
SUCCES set ru_RU.UTF-8  (ru_ru)
SUCCES set en_US.UTF-8  (c.utf8)
SUCCES set ru_RU.UTF-8  (ru)

but throught ishell shows only

970 entries
SUCCES set C            (posix-utf2)
SUCCES set C            (english_united-states.437)
SUCCES set C            (c_c.c)
SUCCES set C            (posix)
SUCCES set C            (c.en)
SUCCES set C            (c_c)
SUCCES set C            (c.ascii)
SUCCES set C            (c)

#6 Updated by Marcin Kuzminski [staff] 7 months ago

can you check

~/.rccontrol/supervisord.ini ?

It can have this:

[supervisord]
minfds = 1024
minprocs = 200
loglevel = info
environment = HOME=/home/ubuntu,LANG=en_US.UTF-8,LC_ALL=en_US.UTF-8
strip_ansi = true
logfile = /home/ubuntu/.rccontrol/supervisor/supervisord.log
pidfile = /home/ubuntu/.rccontrol/supervisor/supervisord.pid

#7 Updated by Nikolay Yankin 7 months ago

[supervisord]
minfds = 1024
minprocs = 200
loglevel = info
environment = HOME=/root,LANG=en_US.UTF-8
strip_ansi = true
logfile = /root/.rccontrol/supervisor/supervisord.log
pidfile = /root/.rccontrol/supervisor/supervisord.pid

#8 Updated by Marcin Kuzminski [staff] 7 months ago

can you add LC_ALL there, and run rccontrol self-stop && rccontrol self-init

#9 Updated by Nikolay Yankin 7 months ago

Nothing changed, but i tested 4.7.0, 4.8.0, 4.9.1 and problem appear on all these versions.

So, problem not only in RhodeCode, probably it appeared after upgrade glibc from 2.25 to 2.26

#10 Updated by Marcin Kuzminski [staff] 7 months ago

might be, we got odd locale issues on multiple of our own server after some recent package updates...

But i'm not sure how to resolve this...

#11 Updated by Nikolay Yankin 7 months ago

Try to update python to 2.7.14 and glibc to 2.25 in you bundles. Do you have night or test builds? Or how i can update this packages manually?

#12 Updated by Marcin Kuzminski [staff] 7 months ago

WE use a nix package manager, and it has pinned older python/glibc

  • we're on upgrading those but it's probably bigger task for next release

This ticket is interesting: https://github.com/NixOS/nix/issues/599

Maybe an option is to set LOCAL_ARCHIVES ?

But anyway it's odd that setting env LC_ALL via supervisor doesn't resolve that problem...

#13 Updated by Nikolay Yankin 7 months ago

Setting LOCALE_ARCHIVE=/usr/lib64/locale/locale-archive in .rccontrol/supervisor/supervisord.ini fixes the problem

#14 Updated by Marcin Kuzminski [staff] 6 months ago

  • Status changed from New to Resolved

#15 Updated by Sefter Aras 4 months ago

i have the same problem on openSuse 42.3, the LOCALE_ARCHIVE=/usr/lib64/locale/locale-archive doesn´t exists

my supervisord.ini:

[supervisord]
minfds = 1024
minprocs = 200
loglevel = info
environment = LOCALE_ARCHIVE=/usr/lib/locale/locale-archive,HOME=/root,LANG=en_US.UTF-8,LC_ALL=en_US.UTF-8
strip_ansi = true

it´s not working for me :-(

do you have any other ideas?

here my log:

Error: unsupported locale setting
[2018-04-12 15:31:08 +0000] [30837] [INFO] Worker exiting (pid: 30837)
Traceback (most recent call last):
File "/opt/rhodecode/store/9k60d775bg7fvbpvvbmxd9hz1lr2558j-python2.7-gunicorn-19.7.1/bin/.gunicorn-wrapped", line 12, in
sys.exit(run())
File "/opt/rhodecode/store/9k60d775bg7fvbpvvbmxd9hz1lr2558j-python2.7-gunicorn-19.7.1/lib/python2.7/site-packages/gunicorn/app/wsgiapp.py", line 74, in run
WSGIApplication("%(prog)s [OPTIONS] [APP_MODULE]").run()
File "/opt/rhodecode/store/9k60d775bg7fvbpvvbmxd9hz1lr2558j-python2.7-gunicorn-19.7.1/lib/python2.7/site-packages/gunicorn/app/base.py", line 203, in run
super(Application, self).run()
File "/opt/rhodecode/store/9k60d775bg7fvbpvvbmxd9hz1lr2558j-python2.7-gunicorn-19.7.1/lib/python2.7/site-packages/gunicorn/app/base.py", line 72, in run
Arbiter(self).run()
File "/opt/rhodecode/store/9k60d775bg7fvbpvvbmxd9hz1lr2558j-python2.7-gunicorn-19.7.1/lib/python2.7/site-packages/gunicorn/arbiter.py", line 231, in run
self.halt(reason=inst.reason, exit_status=inst.exit_status)
File "/opt/rhodecode/store/9k60d775bg7fvbpvvbmxd9hz1lr2558j-python2.7-gunicorn-19.7.1/lib/python2.7/site-packages/gunicorn/arbiter.py", line 344, in halt
self.stop()
File "/opt/rhodecode/store/9k60d775bg7fvbpvvbmxd9hz1lr2558j-python2.7-gunicorn-19.7.1/lib/python2.7/site-packages/gunicorn/arbiter.py", line 393, in stop
time.sleep(0.1)
File "/opt/rhodecode/store/9k60d775bg7fvbpvvbmxd9hz1lr2558j-python2.7-gunicorn-19.7.1/lib/python2.7/site-packages/gunicorn/arbiter.py", line 244, in handle_chld
self.reap_workers()
File "/opt/rhodecode/store/9k60d775bg7fvbpvvbmxd9hz1lr2558j-python2.7-gunicorn-19.7.1/lib/python2.7/site-packages/gunicorn/arbiter.py", line 524, in reap_workers
raise HaltServer(reason, self.WORKER_BOOT_ERROR)
gunicorn.errors.HaltServer:
/opt/rhodecode/store/bc656fqasmx1jmkglarvapy244bbh7dg-bash-4.3-p42/bin/bash: warning: setlocale: LC_ALL: cannot change locale (en_US.UTF-8)
/opt/rhodecode/store/bc656fqasmx1jmkglarvapy244bbh7dg-bash-4.3-p42/bin/bash: warning: setlocale: LC_ALL: cannot change locale (en_US.UTF-8)

#16 Updated by Marcin Kuzminski [staff] 4 months ago

Hi,

Have you rebooted rhodecode after those changes ?

Maybe it's related to supervisord not loading those settings

#17 Updated by Sefter Aras 4 months ago

yes

rccontrol self-stop && rccontrol self-init

it´s correct so? or did you mean server reboot?

#18 Updated by Marcin Kuzminski [staff] 4 months ago

Hmm, not sure then, maybe it's related to different Bash profile.

what's the output of locale when you run it via the profile rhodecode is running?

#19 Updated by Sefter Aras 4 months ago

LANG=de_DE.UTF-8
LC_CTYPE="de_DE.UTF-8"
LC_NUMERIC="de_DE.UTF-8"
LC_TIME="de_DE.UTF-8"
LC_COLLATE="de_DE.UTF-8"
LC_MONETARY="de_DE.UTF-8"
LC_MESSAGES="de_DE.UTF-8"
LC_PAPER="de_DE.UTF-8"
LC_NAME="de_DE.UTF-8"
LC_ADDRESS="de_DE.UTF-8"
LC_TELEPHONE="de_DE.UTF-8"
LC_MEASUREMENT="de_DE.UTF-8"
LC_IDENTIFICATION="de_DE.UTF-8"
LC_ALL=

#20 Updated by Marcin Kuzminski [staff] 4 months ago

yes so problem is that LC_ALL isn't set.

Can you add LC_ALL in your profile maybe?

#21 Updated by Sefter Aras 4 months ago

de_DE.UTF-8 or en_US.UTF-8 ?

#22 Updated by Sefter Aras 4 months ago

if i set LC_ALL then i got this error on rccontrol start vcsserver-1

/opt/rhodecode/store/bc656fqasmx1jmkglarvapy244bbh7dg-bash-4.3-p42/bin/bash: warning: setlocale: LC_ALL: cannot change locale (de_DE.UTF-8)
/opt/rhodecode/store/bc656fqasmx1jmkglarvapy244bbh7dg-bash-4.3-p42/bin/bash: warning: setlocale: LC_ALL: cannot change locale (de_DE.UTF-8)

and i have the same error log entry above

#23 Updated by Nikolay Yankin 4 months ago

Sefter Aras, In your system LOCALE_ARCHIVE need to be set to another path. Maybe to /usr/lib/locale/locale-archive
Try to run 'locate locale-archive'

#24 Updated by Sefter Aras 4 months ago

locate locale-archive
If 'locate' is not a typo you can use command-not-found to lookup the package that contains it, like this:
cnf locate

doesnt work

#25 Updated by Sefter Aras 4 months ago

OK, i have installed the locate util for opensuse

locate locale-archive
/usr/lib/locale/locale-archive

#26 Updated by Sefter Aras 4 months ago

But this path i have already set, this change nothing

#27 Updated by Sefter Aras 4 months ago

i feel that my settings doesn´t work:

my supervisord.ini:

[supervisord]
minfds = 1024
minprocs = 200
loglevel = info
environment = LOCALE_ARCHIVE=/usr/lib/locale/locale-archive,HOME=/root,LANG=en_US.UTF-8,LC_ALL=en_US.UTF-8
strip_ansi = true

#28 Updated by Marcin Kuzminski [staff] 4 months ago

after changes in supervisord.ini did you run rccontrol self-stop && rccontrol self-init ?

#29 Updated by Marcin Kuzminski [staff] 28 days ago

We've identified the problem for now. It looks like since glibc 2.27 the generated locale broke backward compatibility. We've yet not found a solution to this problem, the only thing right now is to potentially either re-generate the locale-archives using older glibc or downgrade glibc to pre 2.27

Also available in: Atom PDF