Page 1 of 1

Python utf-8 and apache

Posted: Fri Jan 03, 2014 11:02 pm
by YannickF
Hello

I have a python script that works very well on Archlinux, both from shell and a php page (using passthru() or shell_exec() )
This script prints some messages in french (with accented letters : é à è)
ex: print("Connexion à la base de données MySQL....")

I'm now using debian, everything works perfectly but this script.
no problem from shell : it works as usual.
But if I launch the script from a php page : error ! In Apache error.log I can see a message from Python that clearly indicates that Python can't encode "à" to ascii

So, why is Python using ascii encoding when launched from apache, and not when launched from shell ? Is there a way to force apache or Python to use UTF8 when the communicates ?

PS : Debian and Archlinux LOCALE are the same : UTF8
PS2: my script begin with
#! /usr/bin/python3
# -*- coding: Utf-8 -*-
PS3 : my text editor is configured for saving UTF-8 files
HTML page produced by php are UTF-8 (via meta tags)

I add a Default_charset= utf8 in php.ini (existed in Archlinux, not in Debian)

I try to add .encode('utf-8') to my text messages... No more errors, but it's not readable when displayed in HTML page (it displays something \x08)

I spent a lot of time in trying to solve this problem but I can't.... I need some help.

Re: Python utf-8 and apache

Posted: Fri Jan 03, 2014 11:43 pm
by paddyg
Probably stupid question but are you using python3 from terminal and on previous system? This kind of thing is the biggest 2v3 headache.

Re: Python utf-8 and apache

Posted: Fri Jan 03, 2014 11:48 pm
by YannickF
My script is working very well on both system when launched from terminal.
(evertything was initially made for python3, not for python2)

Re: Python utf-8 and apache

Posted: Fri Jan 03, 2014 11:57 pm
by jojopi
The difference is that from the command line Python inherits your locale settings (probably LANG=fr_FR.UTF-8), whereas from Apache it inherits LANG=C. It knows that your strings are Unicode, but it can not print them in an ASCII environment.

Changing locale within the script appears to be too late. You can set the encoding for standard output explicitly:

Code: Select all

import sys, io
sys.stdout = io.TextIOWrapper(sys.stdout.buffer, encoding="utf-8")
However, this is ugly and probably not the best solution.

You can export a more appropriate LANG setting in the PHP that calls the Python. Or you can set the locale for the whole of Apache by uncommenting this in /etc/apache2/envvars:

Code: Select all

## Uncomment the following line to use the system default locale instead:
#. /etc/default/locale

Re: Python utf-8 and apache

Posted: Sat Jan 04, 2014 12:20 am
by YannickF
Very very very big big thanks to jojopi !!! :D :D :D

Code: Select all

## Uncomment the following line to use the system default locale instead:
. /etc/default/locale
(uncommented now) is the solution....

I lost my day for this large problem (as large as a # :evil: )