YannickF
Posts: 5
Joined: Mon Dec 03, 2012 10:34 pm

Python utf-8 and apache

Fri Jan 03, 2014 11:02 pm

Hello

I have a python script that works very well on Archlinux, both from shell and a php page (using passthru() or shell_exec() )
This script prints some messages in french (with accented letters : é à è)
ex: print("Connexion à la base de données MySQL....")

I'm now using debian, everything works perfectly but this script.
no problem from shell : it works as usual.
But if I launch the script from a php page : error ! In Apache error.log I can see a message from Python that clearly indicates that Python can't encode "à" to ascii

So, why is Python using ascii encoding when launched from apache, and not when launched from shell ? Is there a way to force apache or Python to use UTF8 when the communicates ?

PS : Debian and Archlinux LOCALE are the same : UTF8
PS2: my script begin with
#! /usr/bin/python3
# -*- coding: Utf-8 -*-
PS3 : my text editor is configured for saving UTF-8 files
HTML page produced by php are UTF-8 (via meta tags)

I add a Default_charset= utf8 in php.ini (existed in Archlinux, not in Debian)

I try to add .encode('utf-8') to my text messages... No more errors, but it's not readable when displayed in HTML page (it displays something \x08)

I spent a lot of time in trying to solve this problem but I can't.... I need some help.

User avatar
paddyg
Posts: 2388
Joined: Sat Jan 28, 2012 11:57 am
Location: UK

Re: Python utf-8 and apache

Fri Jan 03, 2014 11:43 pm

Probably stupid question but are you using python3 from terminal and on previous system? This kind of thing is the biggest 2v3 headache.
also https://groups.google.com/forum/?hl=en-GB&fromgroups=#!forum/pi3d

YannickF
Posts: 5
Joined: Mon Dec 03, 2012 10:34 pm

Re: Python utf-8 and apache

Fri Jan 03, 2014 11:48 pm

My script is working very well on both system when launched from terminal.
(evertything was initially made for python3, not for python2)

User avatar
jojopi
Posts: 3085
Joined: Tue Oct 11, 2011 8:38 pm

Re: Python utf-8 and apache

Fri Jan 03, 2014 11:57 pm

The difference is that from the command line Python inherits your locale settings (probably LANG=fr_FR.UTF-8), whereas from Apache it inherits LANG=C. It knows that your strings are Unicode, but it can not print them in an ASCII environment.

Changing locale within the script appears to be too late. You can set the encoding for standard output explicitly:

Code: Select all

import sys, io
sys.stdout = io.TextIOWrapper(sys.stdout.buffer, encoding="utf-8")
However, this is ugly and probably not the best solution.

You can export a more appropriate LANG setting in the PHP that calls the Python. Or you can set the locale for the whole of Apache by uncommenting this in /etc/apache2/envvars:

Code: Select all

## Uncomment the following line to use the system default locale instead:
#. /etc/default/locale

YannickF
Posts: 5
Joined: Mon Dec 03, 2012 10:34 pm

Re: Python utf-8 and apache

Sat Jan 04, 2014 12:20 am

Very very very big big thanks to jojopi !!! :D :D :D

Code: Select all

## Uncomment the following line to use the system default locale instead:
. /etc/default/locale
(uncommented now) is the solution....

I lost my day for this large problem (as large as a # :evil: )

Return to “Python”