California
Posts: 160
Joined: Sat Jun 06, 2015 2:33 pm

Converting python 2.7 code to python 3.4

Mon Feb 04, 2019 3:58 pm

Dear all, below is part of the working code for RS485 communication between PC and Microchip PIC18F458 deweloped in python 2.7.
I woul like to move to python 3.4 with my codding, so your help would be very appreciatted. The general problem for me is part of the code with:

Code: Select all

readBuffer = array.array('c')

Code: Select all

import serial
import array
import struct
from time import sleep


maxReadCount=8
readBuffer = array.array('c')
#print type(readBuffer)

ser = 0

#-----------------------------------------------------------------------
def init_serial():
    COMNUM = 2
    global ser
    ser = serial.Serial()
    ser.baudrate = 19200
    ser.port = COMNUM - 1

    ser.timeout = 10
    ser.open()

    #if ser.isOpen():
    #    print 'Open'
#--------------------------------------------------------------------------
def crc_calc(start_, addr, datalen_, dat_0, dat_1, dat_2, stop_):
    crc = datalen_ ^ addr
    crc = crc ^ dat_0
    crc = crc ^ dat_1
    crc = crc ^ dat_2
    crc = 255 - crc
    if crc == start_ or crc == stop_:
        crc = crc + 1
    return crc
#------------- RESET PIC18F458---------------------------------------
def reset_18f458():
    start = 0x96
    address = 0xd2
    datalen = 0x83
    data_0 = 0xba
    data_1 = 0xde     # command
    data_2 = 0xba
    stop = 0xa9

    crc_ = crc_calc(start, address, datalen, data_0, data_1, data_2, stop)
    data = bytearray(
        [start, address, datalen, data_0, data_1, data_2, crc_, stop])
    ser.write(data)
    readBuffer = array.array('c')
    readCount = 0
    while readCount < maxReadCount:
        readBuffer.append(ser.read(1))
        readCount = readCount + 1
    i = 0
    while i < maxReadCount:
        val = ord(readBuffer[i])
        if i == 4:
            power_situation = int(val)
        i = i + 1
    #print power_situation
Thanks in advance ...

pcmanbob
Posts: 6180
Joined: Fri May 31, 2013 9:28 pm
Location: Mansfield UK

Re: Converting python 2.7 code to python 3.4

Mon Feb 04, 2019 4:05 pm

A simple google search " python array.array verses python3 array.array "

found this - https://stackoverflow.com/questions/477 ... n-python-3

guess google must be down were you are ;)
We want information… information… information........................no information no help
The use of crystal balls & mind reading are not supported

User avatar
MrYsLab
Posts: 331
Joined: Mon Dec 15, 2014 7:14 pm
Location: Noo Joysey, USA

Re: Converting python 2.7 code to python 3.4

Mon Feb 04, 2019 6:52 pm

If I look at the man page for python 2.7.15 for array.array, here is what I see:

class array.array(typecode[, initializer])

A new array whose items are restricted by typecode, and initialized from the optional initializer value, which must be a list, string, or iterable over elements of the appropriate type.

Changed in version 2.4: Formerly, only lists or strings were accepted.

If given a list or string, the initializer is passed to the new array’s fromlist(), fromstring(), or fromunicode() method (see below) to add initial items to the array. Otherwise, the iterable initializer is passed to the extend() method.
If then look the man page for python 3.7.2 I see:

class array.array(typecode[, initializer])

A new array whose items are restricted by typecode, and initialized from the optional initializer value, which must be a list, a bytes-like object, or iterable over elements of the appropriate type.

If given a list or string, the initializer is passed to the new array’s fromlist(), frombytes(), or fromunicode() method (see below) to add initial items to the array. Otherwise, the iterable initializer is passed to the extend() method.
Perhaps I am missing something, but they seem to be identical. Could you please state what exactly is your problem. If you are getting any exception trace, please provide that.

California
Posts: 160
Joined: Sat Jun 06, 2015 2:33 pm

Re: Converting python 2.7 code to python 3.4

Sat Feb 09, 2019 10:33 am

Thanks for the reply. Next step of convertion:

Code: Select all

    readBuffer = array.array('b')
    readCount = 0
    while readCount < maxReadCount:
        readBuffer.append(ser.read(1))
        readCount = readCount + 1
    i = 0
gives an error

Code: Select all

    readBuffer.append(ser.read(1))
TypeError: an integer is required (got type bytes)
Any help

Regards Vlado

User avatar
MrYsLab
Posts: 331
Joined: Mon Dec 15, 2014 7:14 pm
Location: Noo Joysey, USA

Re: Converting python 2.7 code to python 3.4

Sat Feb 09, 2019 12:10 pm

Ok, that helps quite a bit. Below is a simple example that may help. The program below simulates getting a byte as if that returned from a pyserial read().To append this to the array, this must be converted to an integer type value. The ord() method is probably what you are looking for. If you need to interpret the data on the readBuffer as characters, you may need to use the chr() method.

I hope this makes sense when you look at the code. Feel free to ask additional questions if it doesn't.

Code: Select all

import array

# converting a character to a byte
x = 'a'.encode()

readBuffer = array.array('b')

# use ordinal to convert byte to an integer.
# append this to the array.
readBuffer.append(ord(x))

# print the contents of the buffer.
# to convert the data back to a char use the chr() method.
for data in readBuffer:
    print(data)
    print(chr(data))

California
Posts: 160
Joined: Sat Jun 06, 2015 2:33 pm

Re: Converting python 2.7 code to python 3.4

Sat Feb 09, 2019 1:06 pm

Thanks, below is solutionfor that part of the code:

Code: Select all

    readBuffer = array.array('i')
    readCount = 0
    while readCount < maxReadCount:
        readBuffer.append(ord(ser.read(1)))
        readCount = readCount + 1
    i = 0
    while i < maxReadCount:
        val = readBuffer[i]
        if i == 4:
            power_situation = int(val)
        i = i + 1
but new isssue appeared in following part of the code:

Code: Select all

def convert_MICROCHIP_32bit_to_float(x):
    if x == 0:
        x = 1231459733
    x_hex = format(x, 'x')
    bits = bin(int(x_hex, 16))[2:].zfill(len(x_hex) * 4)
    i = 0
    val = list("                                ")
    b_s = bytearray(val)
    for char in bits:
        current_bit = int(char)
        if current_bit == 1:
            if i < 8:
                b_s[i + 1] = "1"
            elif i == 8:
                b_s[0] = "1"
            else:
                b_s[i] = "1"
        else:
            if i < 8:
                b_s[i + 1] = "0"
            elif i == 8:
                b_s[0] = "0"
            else:
                b_s[i] = "0"
        i = i + 1
    val = str(b_s)
    hex_val = hex(int(val, 2))[2:].rstrip("L")
    float_val = struct.unpack('!f', hex_val.decode('hex'))[0]
    return float_val
Error inside:

Code: Select all

    b_s = bytearray(val)
TypeError: an integer is required

User avatar
Paeryn
Posts: 2561
Joined: Wed Nov 23, 2011 1:10 am
Location: Sheffield, England

Re: Converting python 2.7 code to python 3.4

Sat Feb 09, 2019 1:14 pm

You appear to be trying to initialise a bytearray with a list of strings, when using a string to initialise bytearray() you have to give the encoding as well. Not sure if you can use a list of strings though.

Always a good idea to read the Python docs on the functions you are using.

Code: Select all

bytearray([source[, encoding[, errors]]])

The optional source parameter can be used to initialize the array in a few different ways:

If it is a string, you must also give the encoding (and optionally, errors) parameters; bytearray() then converts the string to bytes using str.encode().
She who travels light — forgot something.

User avatar
MrYsLab
Posts: 331
Joined: Mon Dec 15, 2014 7:14 pm
Location: Noo Joysey, USA

Re: Converting python 2.7 code to python 3.4

Sat Feb 09, 2019 2:18 pm

What you are dealing with is perhaps the most difficult part of going from Python 2 to Python 3 and that is the change in dealing with data representation - specifically, strings are Unicode in Python 3, and so we sometimes need to convert things to and from Unicode representations. It was a lot more intuitive (IMHO) in Python 2, but the changes in Python 3 actually are a good thing. Don't feel bad about getting caught up in this mess - I have been programming in Python 3 for several years and still get bitten by these problems. The only difference between you and me is that I have made those mistakes so many times, that I know how to fix them, or at least where to look for the solution.

Here is some more sample code that I hope will help:

Code: Select all

# encode the string
# to a bytestring
empty = "                                ".encode()

b_s = list(empty)
print(b_s)
And by the way, the documentation is great if you already fully understand how things work, not so good when you encounter issues that you have never seen before and are trying to solve them - just saying.

California
Posts: 160
Joined: Sat Jun 06, 2015 2:33 pm

Re: Converting python 2.7 code to python 3.4

Sat Feb 09, 2019 4:25 pm

Thanks, one step more:

Code: Select all

def convert_MICROCHIP_32bit_to_float(x):
    if x == 0:
        x = 1231459733
    x_hex = format(x, 'x')
    bits = bin(int(x_hex, 16))[2:].zfill(len(x_hex) * 4)
    i = 0
    val = "                                ".encode()
    b_s = list(val)
    for char in bits:
        current_bit = int(char)
        if current_bit == 1:
            if i < 8:
                b_s[i + 1] = "1"
            elif i == 8:
                b_s[0] = "1"
            else:
                b_s[i] = "1"
        else:
            if i < 8:
                b_s[i + 1] = "0"
            elif i == 8:
                b_s[0] = "0"
            else:
                b_s[i] = "0"
        i = i + 1
    val = str(b_s)
    hex_val = hex(int(val, 2))[2:].rstrip("L")
    float_val = struct.unpack('!f', hex_val.decode('hex'))[0]
    return float_val
gives another error:

Code: Select all

    hex_val = hex(int(val, 2))[2:].rstrip("L")
ValueError: invalid literal for int() with base 2: "['0', '1', '0', '0', '0', '0', '0', '1', '1', '0', '1', '1', '1', '0', '1', '0', '1', '0', '0', '0', '0', '0', '0', '0', '0', '0', '0', '0', '0', '0', '0', '0']"

User avatar
MrYsLab
Posts: 331
Joined: Mon Dec 15, 2014 7:14 pm
Location: Noo Joysey, USA

Re: Converting python 2.7 code to python 3.4

Sat Feb 09, 2019 8:11 pm

I don't fully appreciate what should be happening in convert_MICROCHIP_32bit_to_float().

What value would you expect to see returned when convert_MICROCHIP_32bit_to_float(0) is called? Or any other value passed in that you choose.

Some general questions first. In the line:

Code: Select all

    hex_val = hex(int(val, 2))[2:].rstrip("L")
Why does rstrip have an "L" parameter?

Also, why are you using struct.unpack? Where were things packed?

User avatar
Paeryn
Posts: 2561
Joined: Wed Nov 23, 2011 1:10 am
Location: Sheffield, England

Re: Converting python 2.7 code to python 3.4

Sat Feb 09, 2019 8:53 pm

California wrote:
Sat Feb 09, 2019 4:25 pm

Code: Select all

    val = str(b_s)
    hex_val = hex(int(val, 2))[2:].rstrip("L")
    float_val = struct.unpack('!f', hex_val.decode('hex'))[0]
    return float_val
gives another error:

Code: Select all

    hex_val = hex(int(val, 2))[2:].rstrip("L")
ValueError: invalid literal for int() with base 2: "['0', '1', '0', '0', '0', '0', '0', '1', '1', '0', '1', '1', '1', '0', '1', '0', '1', '0', '0', '0', '0', '0', '0', '0', '0', '0', '0', '0', '0', '0', '0', '0']"
You have b_s which is list of strings, val will therefore be the string representation of that list as shown in the error e.g. (elements substituted with ellipsis by me) "['0', '1', ... , '0']". That string cannot be converted into an integer as characters like [ and ' are not valid parts of a number, when you pass a string to int() it is expecting that string to contain valid digits (with a possible leading sign).

Again, the docs say for int()
class int(x, base=10)

If x is not a number or if base is given, then x must be a string, bytes, or bytearray instance representing an integer literal in radix base. Optionally, the literal can be preceded by + or - (with no space in between) and surrounded by whitespace. A base-n literal consists of the digits 0 to n-1, with a to z (or A to Z) having values 10 to 35. The default base is 10. The allowed values are 0 and 2–36. Base-2, -8, and -16 literals can be optionally prefixed with 0b/0B, 0o/0O, or 0x/0X, as with integer literals in code.
MrYsLab:
The .rstrip('L') was in Python 2 to remove a possible L suffix when the value was a long rather than an int. Python 3 doesn't have long any more, it became int and the old int (limited precision - usually 32 bit) was removed.
She who travels light — forgot something.

California
Posts: 160
Joined: Sat Jun 06, 2015 2:33 pm

Re: Converting python 2.7 code to python 3.4

Sat Feb 16, 2019 1:25 pm

Thanks, got solution in code below:

Code: Select all

def convert_MICROCHIP_32bit_to_float(x):
    if x == 0:
        x = 1231459733
    x_hex = format(x, 'x')
    bits = bin(int(x_hex, 16))[2:].zfill(len(x_hex) * 4)   
    i = 0
    val = "                                ".encode()
    b_s = list(val)
    for char in bits:
        current_bit = int(char)
        if current_bit == 1:
            if i < 8:
                b_s[i + 1] = "1"
            elif i == 8:
                b_s[0] = "1"
            else:
                b_s[i] = "1"
        else:
            if i < 8:
                b_s[i + 1] = "0"
            elif i == 8:
                b_s[0] = "0"
            else:
                b_s[i] = "0"
        i = i + 1   
    val = [int(i) for i in b_s]
    out = 0
    for bit in val:
        out = (out << 1) | bit   
    hex_val = hex(out) 
    def convert(s):
        i = int(s, 16)                   # convert from hex to a Python int
        cp = pointer(c_int(i))           # make this into a c integer
        fp = cast(cp, POINTER(c_float))  # cast the int pointer to a float pointer
        return fp.contents.value         # dereference the pointer, get the float    
    float_val = convert(hex_val)
    return float_val
Regards

Return to “Python”