JohnnyCyp
Posts: 63
Joined: Sun Mar 31, 2019 11:05 pm

UnicodeDecodeError: 'utf8' codec can't decode byte 0x92 in position 532: invalid start byte

Wed May 22, 2019 2:57 pm

When i run my python program for webserver and then try to entered using my RPi IP and 5000 port i got this error:


Traceback (most recent call last):
File "/usr/lib/python2.7/dist-packages/flask/app.py", line 1997, in __call__
return self.wsgi_app(environ, start_response)
File "/usr/lib/python2.7/dist-packages/werkzeug/contrib/fixers.py", line 152, in __call__
return self.app(environ, start_response)
File "/usr/lib/python2.7/dist-packages/flask/app.py", line 1985, in wsgi_app
response = self.handle_exception(e)
File "/usr/lib/python2.7/dist-packages/flask/app.py", line 1540, in handle_exception
reraise(exc_type, exc_value, tb)
File "/usr/lib/python2.7/dist-packages/flask/app.py", line 1982, in wsgi_app
response = self.full_dispatch_request()
File "/usr/lib/python2.7/dist-packages/flask/app.py", line 1614, in full_dispatch_request
rv = self.handle_user_exception(e)
File "/usr/lib/python2.7/dist-packages/flask/app.py", line 1517, in handle_user_exception
reraise(exc_type, exc_value, tb)
File "/usr/lib/python2.7/dist-packages/flask/app.py", line 1612, in full_dispatch_request
rv = self.dispatch_request()
File "/usr/lib/python2.7/dist-packages/flask/app.py", line 1598, in dispatch_request
return self.view_functions[rule.endpoint](**req.view_args)
File "/usr/lib/python2.7/dist-packages/flask/views.py", line 84, in view
return self.dispatch_request(*args, **kwargs)
File "/usr/lib/python2.7/dist-packages/flask/views.py", line 149, in dispatch_request
return meth(*args, **kwargs)
File "/home/pi/Project/app.py", line 18, in get
return flask.render_template('welcome.html')
File "/usr/lib/python2.7/dist-packages/flask/templating.py", line 133, in render_template
return _render(ctx.app.jinja_env.get_or_select_template(template_name_or_list),
File "/usr/lib/python2.7/dist-packages/jinja2/environment.py", line 851, in get_or_select_template
return self.get_template(template_name_or_list, parent, globals)
File "/usr/lib/python2.7/dist-packages/jinja2/environment.py", line 812, in get_template
return self._load_template(name, self.make_globals(globals))
File "/usr/lib/python2.7/dist-packages/jinja2/environment.py", line 774, in _load_template
cache_key = self.loader.get_source(self, name)[1]
File "/usr/lib/python2.7/dist-packages/flask/templating.py", line 57, in get_source
return self._get_source_fast(environment, template)
File "/usr/lib/python2.7/dist-packages/flask/templating.py", line 82, in _get_source_fast
return loader.get_source(environment, template)
File "/usr/lib/python2.7/dist-packages/jinja2/loaders.py", line 175, in get_source
contents = f.read().decode(self.encoding)
File "/usr/lib/python2.7/encodings/utf_8.py", line 16, in decode
return codecs.utf_8_decode(input, errors, True)
UnicodeDecodeError: 'utf8' codec can't decode byte 0x92 in position 532: invalid start byte

source code:

Code: Select all

import flask, flask.views
from flask import jsonify, request, Response
from werkzeug.contrib.fixers import ProxyFix
import functools
from subprocess import check_output
from functions import *
import urllib
import re
from time import sleep

app = flask.Flask(__name__)
app.secret_key = 'YourSecurityKey'
users = database().getUsers()


class Main(flask.views.MethodView):
    def get(self):
        return flask.render_template('welcome.html')

def login_required(method):
    @functools.wraps(method)
    def wrapper(*args, **kwargs):
        if 'username' in flask.session:
            return method(*args, **kwargs)
        else:
            flask.flash("A login is required to see the page!")
            return flask.redirect(flask.url_for('login'))
    return wrapper

class Tongles(flask.views.MethodView):
    @login_required
    def get(self):
        return flask.render_template('tongles.html', status=getStatus(), temp=GetTemp(), 
            inIP=GetInIP(), exIP=GetExIP(), pmode=status().pwr_mode())
    def post(self):
        formType = flask.request.form.get('formType')
        digitPass = ""
        if formType == "digitPassword" or "shutdown":
            required = ['firstdigit', 'seconddigit', 'thirddigit', 'fourthdigit']
            for r in required:
                    digitPass += flask.request.form.get(r)
            if str(digitPass) == str(database().getFourDigit()):
                state = database().getState()
                if formType == "shutdown":
                    if state == "True":
                        flask.flash("Disarm system before shutting down!")
                        return flask.redirect(flask.url_for('tongles'))
                    else:
                        print ("Shutting Down...")
                        database().writeLogs("System has been shutted down")
                        shutdown()
                elif formType == "digitPassword":
                    if state == "True":
                        status().disarm()
                    else:
                        status().arm()
                else:
                    flask.flash("Something went totally wrong here, please try again!")
                    return flask.redirect(flask.url_for('tongles'))
            else:
                flask.flash("Incorect Password, please try again!")
            return flask.redirect(flask.url_for('tongles'))

class Gallery(flask.views.MethodView):
    @login_required
    def get(self):
        return flask.render_template('gallery.html')

class Logs(flask.views.MethodView):
    @login_required
    def get(self):
        return flask.render_template('logs.html', records=database().getLogs())

class Stream(flask.views.MethodView):
    @login_required
    def get(self):
        return flask.render_template('stream.html')

class Admin(flask.views.MethodView):
    @login_required
    def get(self):
        return flask.render_template('admin.html')
    def post(self):
        global users
        formType = flask.request.form.get('formType')

        if formType == "changePassword":
            if 'password' not in flask.request.form:
                flask.flash("Something went totally wrong here. Please try to change your password again!")
                return flask.redirect(flask.url_for('admin'))
            newPassword = database().getHashed(flask.request.form.get('password'))
            currentPassword = database().getPassword()
            if newPassword == currentPassword:
                flask.flash("The password you entered is the same as the current one. Please use a different password and try again!")
                return flask.redirect(flask.url_for('admin'))
            else:
                database().changePass(newPassword)
        elif formType == "editUsername":
            if 'username' not in flask.request.form:
                flask.flash("Something went totally wrong here. Please try to change your password again!")
                return flask.redirect(flask.url_for('admin'))
            newUsername = flask.request.form.get('username')
            currentUsername = database().getUsername()
            if newUsername == currentUsername:
                flask.flash("The username you entered is the same as the current one. Please use a different username and try again!")
                return flask.redirect(flask.url_for('admin'))
            else:
                database().changeUsername(newUsername)
        elif formType == "editemail":
            if 'email' not in flask.request.form:
                flask.flash("Something went totally wrong here. Please try to change your e-Mail address again!")
                return flask.redirect(flask.url_for('admin'))
            newemail = flask.request.form.get('email')
            currentemail = database().getEmail()
            if newemail == currentemail:
                flask.flash("The e-Mail address you entered is the same as the current one. Please use a different one and try again!")
                return flask.redirect(flask.url_for('admin'))
            else:
                database().changeEmail(newemail)
        elif formType == "editphoneNumber":
            if 'phoneNumber' not in flask.request.form:
                flask.flash("Something went totally wrong here. Please try to change your phone number again!")
                return flask.redirect(flask.url_for('admin'))
            getPhonenumber = flask.request.form.get('phoneNumber')
            country = flask.request.form.get('countrySelectBox')
            if country == 'GB':
                newPhonenumber = '44{}'.format(getPhonenumber[1:] if getPhonenumber.startswith('0') else getPhonenumber)
            elif country == 'US':
                newPhonenumber = '1{}'.format(getPhonenumber)
            currentPhonenumber = database().getmNumber()
            if newPhonenumber == currentPhonenumber:
                flask.flash("The phone number you entered is the same as the current one. Please use a different one and try again!")
                return flask.redirect(flask.url_for('admin'))
            database().changePhoneNumber(newPhonenumber)
        elif formType == "editDigitPassword":
            if 'currentPassword' not in flask.request.form:
                flask.flash("Something went totally wrong here. Please try to change your four-digit password again!")
                return flask.redirect(flask.url_for('admin'))
            newPassword = flask.request.form.get('newPassword')
            currentPassword = database().getFourDigit()
            if currentPassword != flask.request.form.get('currentPassword'):
                flask.flash("incorrect current four-digit password, please try again")
                return flask.redirect(flask.url_for('admin'))
            elif newPassword == currentPassword:
                flask.flash("The password you entered is the same as the current one. Please use a different password and try again!")
                return flask.redirect(flask.url_for('admin'))
            else:
                database().changeFdigit(newPassword)
        else:
            flask.flash("Something went totally wrong here. Please try to change the preffered details again")
            return flask.render_template('admin.html')
        flask.flash("Your settings have been  modified succsessfuly. In order your changes to be effectful, you will be logged out in a few seconds!")
        users = database().getUsers()
        flask.session.pop('username', None)
        sleep (1)
        flask.flash("You've been succsessfuly logged out!")
        return flask.redirect(flask.url_for('index'))

class Login(flask.views.MethodView):
    def get(self):
        return flask.render_template('login.html')

    def post(self):
        global users
        if 'logout' in flask.request.form:
            flask.session.pop('username', None)
            database().writeLogs("User has been succsessfuly logged out!")
            flask.flash("You've been succsessfuly logged out!")
            return flask.redirect(flask.url_for('login'))
        if 'reset' in flask.request.form:
            Username = flask.request.form.get('username')
            if 'username' not in flask.request.form or (database().getUsername() != Username ): #Check That one
                flask.flash("Something went totally wrong here. Please to check your username and try again!")
                return flask.redirect(flask.url_for('login'))
            newPassword = database().resetPass()
            emailBody = "You recently asked to reset your password, your new password is: " + newPassword
            interact().sendEmail("Your new Password!", emailBody, "NO", database().getEmail())
            users = database().getUsers()
            flask.flash("Your password has been reset. You'll receive an email shortly with your new password")
            # users = database().getUsers()
            return flask.redirect(flask.url_for('login'))
        required = ['username', 'passwd']
        for r in required:
            if r not in flask.request.form:
                flask.flash("Error: {0} is required.".format(r))
                return flask.redirect(flask.url_for('login'))
        username = flask.request.form['username']
        passwd = flask.request.form['passwd']
        print (database().checkHashedPass(passwd, database().getPassword())) 
        if username in users and (database().checkHashedPass(passwd, database().getPassword())) == True :
            flask.session['username'] = username
            database().writeLogs("User has been succsessfuly logged in!")
            flask.flash("Login Succsessful!")
            flask.session['userEmail'] = database().getEmail()
            flask.session['mNumber'] = database().getmNumber()
            return flask.redirect(flask.url_for('admin'))
        else:
            flask.flash("Username doesn't exist or incorrect password")
        return flask.redirect(flask.url_for('login'))

app.add_url_rule('/',
                 view_func=Main.as_view('index'),
                 methods=["GET", "POST"])

app.add_url_rule('/tongles/',
                 view_func=Tongles.as_view('tongles'),
                 methods=['GET', 'POST'])

app.add_url_rule('/gallery/',
                 view_func=Gallery.as_view('gallery'),
                 methods=['GET', 'POST'])

app.add_url_rule('/login/',
                 view_func=Login.as_view('login'),
                 methods=['GET', 'POST'])

app.add_url_rule('/stream/',
                 view_func=Stream.as_view('stream'),
                 methods=['GET', 'POST'])

app.add_url_rule('/logs/',
                 view_func=Logs.as_view('logs'),
                 methods=['GET', 'POST'])

app.add_url_rule('/admin/',
                 view_func=Admin.as_view('admin'),
                 methods=['GET', 'POST'])
    
@app.route("/syStatus")
def getStatus():
    state = database().getState()
    if state == "True":
        status = "Armed"
    else:
        status = "Disarmed"
    return status

@app.route('/video_feed')
def video_feed():
    """Video streaming route. Put this in the src attribute of an img tag."""
    return Response(gen(Camera()),
                    mimetype='multipart/x-mixed-replace; boundary=frame')

@app.route("/picture")
def picture():
    link = interact().ftpSession(None, '/home/pi/Project/Pictures/' + interact().capture_image(), interact().capture_image())
    return  jsonify(pictureLink=link)

def shutdown():
    os.system("sudo shutdown -h now")
    return ""

def gen(camera):
    """Video streaming generator function."""
    while True:
        frame = camera.get_frame()
        yield (b'--frame\r\n'
               b'Content-Type: image/jpeg\r\n\r\n' + frame + b'\r\n')

#System details functions start here -->

def GetTemp():
    #from subprocess import check_output
    output = check_output(["/opt/vc/bin/vcgencmd","measure_temp"])
    temp = float(output.split('=')[1][:-3])
    return temp

def GetInIP():
    output = check_output(["ip","addr","show","eth0"])
    inIP = output.split('\n')[2].strip().split(' ')[1].split('/')[0]
    return inIP

def GetExIP():
    dyndnsRespond = urllib.urlopen("http://checkip.dyndns.org/").read()
    grabIP = re.findall('([0-9]+\.[0-9]+\.[0-9]+\.[0-9]+)', dyndnsRespond)
    exIP = grabIP[0]
    return exIP

#System details functions ends here <--

# run the webserver on port 68, requires sudo
app.wsgi_app = ProxyFix(app.wsgi_app)
try:
    if __name__ == '__main__':
        app.debug = True
        app.run(host='0.0.0.0', port=5000)
except KeyboardInterrupt:
    interact().clean()
    print ("Flask App ended safe!")

User avatar
PeterO
Posts: 4576
Joined: Sun Jul 22, 2012 4:14 pm

Re: UnicodeDecodeError: 'utf8' codec can't decode byte 0x92 in position 532: invalid start byte

Wed May 22, 2019 5:01 pm

What is in "welcome.html" ?
PeterO
Discoverer of the PI2 XENON DEATH FLASH!
Interests: C,Python,PIC,Electronics,Ham Radio (G0DZB),1960s British Computers.
"The primary requirement (as we've always seen in your examples) is that the code is readable. " Dougie Lawson

JohnnyCyp
Posts: 63
Joined: Sun Mar 31, 2019 11:05 pm

Re: UnicodeDecodeError: 'utf8' codec can't decode byte 0x92 in position 532: invalid start byte

Wed May 22, 2019 5:06 pm

PeterO wrote:
Wed May 22, 2019 5:01 pm
What is in "welcome.html" ?
PeterO

Code: Select all

{% extends "index.html" %}
{% block content %}

<!-- <h2>Home Security System Based on RaspberryPi</h2>
		<p>The aim of this project is to build a system that will allow the user to guard and monitor an area. The user will be able to control the system using a numeric keypad or the web control centre. When the system will detect motion, the system will warn and take a picture of the intruder, will allow him some time to type the correct passcode, if the passcode is not correct the picture will be sent to the system’s owner.</p>

		<p>Another feature will be a web streaming video, which will allow the user to monitor the area at anytime, or watch live what is happening to that area in case that an he informed about an unwelcome intruder.</p>

		<p>In addition, in order to prevent power and internet connection failures, a backup battery will be wired on the system, which will be able to charge and discharge at any time. The internet connection will be powered from a 3G portable stand-alone modem.</p>

    {% with messages = get_flashed_messages() %} {% for message in messages %}
        <div class="alert alert-info" role="alert">{{message}}</div>
    {% endfor %} {% endwith %} -->

      <div class="jumbotron">
        <h1>HomeSecPi Project</h1>
        <p class="lead">Get your room secure with this, DIY home security system, based on the Raspberry Pi.</p>
        <p><a class="btn btn-lg btn-success" href="#" role="button">Fork it at GitHub</a></p>
      </div>

      <div class="row marketing">
        <div class="col-lg-6">
          <h4><span class="glyphicon glyphicon-eye-open" aria-hidden="true"></span></span> 24/7 Monitoring</h4>
          <p>Give your self peace of mind with this 24/7 monitor system, and never worry again about leaving your place unprotected</p>

          <h4><span class="glyphicon glyphicon-flash" aria-hidden="true"></span> Power Failure Protection</h4>
          <p>Keep your place secure even if the home electricity fail. With the embeded back-up battery the system will stay alive up to 2 hours.</p>

          <h4><span class="glyphicon glyphicon-exclamation-sign" aria-hidden="true"></span> Live Notifications</h4>
          <p>Stay up-to-date about what is going on to your place. In case of an intrusion, you will be notified via SMS and e-Mail.</p>
        </div>

        <div class="col-lg-6">
          <h4><span class="glyphicon glyphicon-facetime-video" aria-hidden="true"></span> Live Streaming</h4>
          <p>With the Live Streaming feature, you can keep an eye on your room, at any time.</p>

          <h4><span class="glyphicon glyphicon-plane" aria-hidden="true"></span></span> Remote Access</h4>
          <p>With the remote access control panel you can keep have access to the system, even if you are away from home.</p>

          <h4><span class="glyphicon glyphicon-globe" aria-hidden="true"></span> Internet Failure Protection</h4>
          <p>HomeSecPi does not need your home internet connection, it has its own 3G internet connection. Even if your home internet connection fail, the protection it wont be interrupted.</p>
        </div>
      </div>
{% endblock %}

User avatar
jojopi
Posts: 3067
Joined: Tue Oct 11, 2011 8:38 pm

Re: UnicodeDecodeError: 'utf8' codec can't decode byte 0x92 in position 532: invalid start byte

Wed May 22, 2019 7:01 pm

The error corresponds to the fancy apostrophe (right single quotation mark) in "the system’s owner". That would be 0x92 in legacy single-byte Windows encodings, such as CP1252. In UTF-8 it should be the three byte sequence e2-80-99. Alternatively it could be replaced with a non-fancy ASCII apostrophe ("the system's owner"), which is 0x27 in both encodings.

It may be possible to tell Flask to expect the foreign encoding. However, other software on the Pi is likely to be similarly confused by legacy Windows code pages, so it would be better to find a way to save the file as UTF-8.

User avatar
PeterO
Posts: 4576
Joined: Sun Jul 22, 2012 4:14 pm

Re: UnicodeDecodeError: 'utf8' codec can't decode byte 0x92 in position 532: invalid start byte

Wed May 22, 2019 7:05 pm

jojopi wrote:
Wed May 22, 2019 7:01 pm
The error corresponds to the fancy apostrophe (right single quotation mark) in "the system’s owner". That would be 0x92 in legacy single-byte Windows encodings, such as CP1252. In UTF-8 it should be the three byte sequence e2-80-99. Alternatively it could be replaced with a non-fancy ASCII apostrophe ("the system's owner"), which is 0x27 in both encodings.
I wondered about that , but when I cut and pasted the text and looked at the hex I saw E2-80-99 which didn't have 92 in it so I assumed the problem was else where. And 0x92 didn't occur anywhere in the file I had !

PeterO
Discoverer of the PI2 XENON DEATH FLASH!
Interests: C,Python,PIC,Electronics,Ham Radio (G0DZB),1960s British Computers.
"The primary requirement (as we've always seen in your examples) is that the code is readable. " Dougie Lawson

JohnnyCyp
Posts: 63
Joined: Sun Mar 31, 2019 11:05 pm

Re: UnicodeDecodeError: 'utf8' codec can't decode byte 0x92 in position 532: invalid start byte

Wed May 22, 2019 9:07 pm

jojopi wrote:
Wed May 22, 2019 7:01 pm
The error corresponds to the fancy apostrophe (right single quotation mark) in "the system’s owner". That would be 0x92 in legacy single-byte Windows encodings, such as CP1252. In UTF-8 it should be the three byte sequence e2-80-99. Alternatively it could be replaced with a non-fancy ASCII apostrophe ("the system's owner"), which is 0x27 in both encodings.

It may be possible to tell Flask to expect the foreign encoding. However, other software on the Pi is likely to be similarly confused by legacy Windows code pages, so it would be better to find a way to save the file as UTF-8.
Thanks! i removed the apostrophe in "the system’s owner" and now it's working! :)

Return to “Networking and servers”