anddav87
Posts: 11
Joined: Tue Jun 26, 2012 7:50 pm
Location: Benfleet, Essex, UK
Contact: Website

Unicode Decode

Wed Mar 13, 2013 3:20 pm

Hi All

I am using this script to post something to twitter, but am getting this error. I have googled to my heart's content but still not a proper answer on what I need todo to resolve it. Am figuring this is pretty common, so any ideas on what I can do?

I am using Python 2.7

Error:
UnicodeDecodeError: 'ascii' codec can't decode byte 0xff in position 221: ordinal not in range(128)

Code:

Code: Select all

import RPi.GPIO as GPIO
import time
import sys
import os
from twython import Twython

os.system ("fswebcam /home/pi/pic.jpg")

consumer_key = ""
consumer_secret = ""
user_token = ""
user_token_secret = ""


twit = Twython(consumer_key, consumer_secret, user_token, user_token_secret)

twit.updateStatusWithMedia("pic.jpg", status = "New pic taken")
Thanks!

Andy

User avatar
davef21370
Posts: 897
Joined: Fri Sep 21, 2012 4:13 pm
Location: Earth But Not Grounded

Re: Unicode Decode

Wed Mar 13, 2013 5:21 pm

I'm not sure where the error lies exactly but the "ordinal not in range" is due to something in python or the OS only handling the first 128 ASCII characters. I had a similar error using the chr function to convert an ASCII code to a character.

Dave.

Edit..
It looks like the hex ff is in one or all of the strings and causing the error. Search your strings for characters above 127 and either delete them or make them 0 and see if that works.
Apple say... Monkey do !!

anddav87
Posts: 11
Joined: Tue Jun 26, 2012 7:50 pm
Location: Benfleet, Essex, UK
Contact: Website

Re: Unicode Decode

Wed Mar 13, 2013 11:34 pm

davef21370 wrote:I'm not sure where the error lies exactly but the "ordinal not in range" is due to something in python or the OS only handling the first 128 ASCII characters. I had a similar error using the chr function to convert an ASCII code to a character.

Dave.

Edit..
It looks like the hex ff is in one or all of the strings and causing the error. Search your strings for characters above 127 and either delete them or make them 0 and see if that works.
Thanks Dave. I have a feeling it's a character in the Twitter auth strings that I can't change (obviously I deleted those out from the code I posted) but will double check in the morning.

Thanks for your reply

Andy

User avatar
metachris
Posts: 96
Joined: Wed Feb 06, 2013 1:52 pm
Location: Vienna, Austria

Re: Unicode Decode

Thu Mar 14, 2013 1:17 pm

It seems that you are using an utf-8 encoded string which Python tries to decode as ascii (and fails because it includes non-ascii characters). If that is the case you just need to manually set the decoder to utf-8. Here is an example:

Code: Select all

>>> a = '(\xef\xbd\xa1\xef\xbd\xa5\xcf\x89\xef\xbd\xa5\xef\xbd\xa1)\xef\xbe\x89'
>>> print unicode(a)
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
UnicodeDecodeError: 'ascii' codec can't decode byte 0xef in position 1: ordinal not in range(128)
>>> print unicode(a.decode("utf-8"))
(。・ω・。)ノ
Edit: Two things that may help: Add this as the first line in your Python scripts, which tells the interpreter to handle all strings in your script as utf-8 unicode:

Code: Select all

# -*- coding: utf-8 -*-
Further you should double-check if your system language is set up with utf-8 support:

Code: Select all

$ echo $LANG
en_GB.UTF-8
pythonhosted.org/RPIO

anddav87
Posts: 11
Joined: Tue Jun 26, 2012 7:50 pm
Location: Benfleet, Essex, UK
Contact: Website

Re: Unicode Decode

Wed Mar 20, 2013 9:43 pm

metachris wrote:It seems that you are using an utf-8 encoded string which Python tries to decode as ascii (and fails because it includes non-ascii characters). If that is the case you just need to manually set the decoder to utf-8. Here is an example:

Code: Select all

>>> a = '(\xef\xbd\xa1\xef\xbd\xa5\xcf\x89\xef\xbd\xa5\xef\xbd\xa1)\xef\xbe\x89'
>>> print unicode(a)
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
UnicodeDecodeError: 'ascii' codec can't decode byte 0xef in position 1: ordinal not in range(128)
>>> print unicode(a.decode("utf-8"))
(。・ω・。)ノ
Edit: Two things that may help: Add this as the first line in your Python scripts, which tells the interpreter to handle all strings in your script as utf-8 unicode:

Code: Select all

# -*- coding: utf-8 -*-
Further you should double-check if your system language is set up with utf-8 support:

Code: Select all

$ echo $LANG
en_GB.UTF-8
Hi Chris

Sorry for the delay in my reply, I have been away with work.

Just got a chance to try this out, and it's still not working.

Could you maybe point out how I should incorporate the decode into my code? I tried this but to no avail...

Code: Select all

# -*- coding: utf-8 -*-
import RPi.GPIO as GPIO
import time
import sys
import os
from twython import Twython

os.system ("fswebcam /home/pi/catpoop.jpg")

consumer_key = "<deleted>"
consumer_secret = "<deleted>"
user_token = "<deleted>"
user_token_secret = "<deleted>"

consumer_key1 = unicode(consumer_key.decode("utf-8"))
consumer_secret1 = unicode(consumer_secret.decode("utf-8"))
user_token1 = unicode(user_token.decode("utf-8"))
user_token_secret1 = unicode(user_token_secret.decode("utf-8"))

timestamp = time.strftime("%d%m%Y - %H%M%S")

twit = Twython(consumer_key1, consumer_secret1, user_token1, user_token_secret1)

twit.updateStatusWithMedia("catpoop.jpg", status = "")

User avatar
jojopi
Posts: 3078
Joined: Tue Oct 11, 2011 8:38 pm

Re: Unicode Decode

Fri Mar 22, 2013 4:12 pm

anddav87 wrote:UnicodeDecodeError: 'ascii' codec can't decode byte 0xff in position 221: ordinal not in range(128)
Switching from ascii to utf-8 is not going to help here, because 0xff is still not part of any valid character.

I believe that the secret twitter strings should be base64 encoded (consisting only of the characters A-Za-z0-9+/=) anyway. And not as long as 221 bytes. In which case, the error may be caused by the image, and it is probably a bug in Twython.

anddav87
Posts: 11
Joined: Tue Jun 26, 2012 7:50 pm
Location: Benfleet, Essex, UK
Contact: Website

Re: Unicode Decode

Sun Mar 24, 2013 4:08 pm

Thanks jojopi...

I have almost given up, it's doing my head in! Have seen two other people use this code online and it works fine for them, updating a status with media.

Tried using Tweepy and another Twitter package and I can update my status ok with them, but they don't seem to support adding a picture as well.

My last resort is to include some code to FTP the file to my web server and include the link in the Twitter Status. Not what I want todo but it will have todo for now :(

Return to “Python”