Python help

[frying_fish]

Now I know this isn't a python help forum, but stay with me here…..

I have a very simple python related question but it seems either my google-fu is not great today or I'm trying to do something I'm not supposed to be able to do.

I create a 2 dimensional array and fill it with elements. this works absolutely fine

data = [[0 for a in range(2048)]  for b in range(numfiles)]

for fname in dirList: with open(fname,'rb') as file:
byte = file.read(32)
data.append([])
#Skip the bit we don't need, and output the data
for x in range(0,2048):
#We know there are 2048 channels.
data[y][x]=struct.unpack('L',file.read(4))

#Convert the binary data (32bit unsigned int)
#Whilst using the Tuple output to the file
y +=1  I then want to print an element say:
print(data[0][2])

However the output is of the form:
(0,)

I really want this to output as:
0

Am I missing something simple? 

[toxibunny]

Well, the documentation for struct.unpack says "The result is a tuple even if it contains exactly one item."

I don't know exactly what you're trying to do, and your python seems way better than mine, but there's your problem right there....

[frying_fish]

Thank you, I hadn't checked closely enough on Tuples. I have now just introduced an extra line to unpack the Tuple, and I get what I want in the array. Thanks again.

So I've also made some further improvements. The code was to write a quick parser for some data that I get from an old data acquisition program that we use in my experiments for my PhD. The code was just to take all the files produced, and parse out the data to a single text file that I can import into my graphing packages nicely. If people are interested in the code I'll host it somewhere, but be aware it probably would only work on this specific set of data, but can be manipulated to do what people need.

[jardiamj]

You cannot have a single element by itself inside a tuple in python, this is what the docs.python.org says:

A special problem is the construction of tuples containing 0 or 1 items: the syntax has some extra quirks to accommodate these. Empty tuples are constructed by an empty pair of parentheses; a tuple with one item is constructed by following a value with a comma (it is not sufficient to enclose a single value in parentheses)

So, might want a different representation or to accommodate to this ugly python feature.

[spurious]

how about trying:
data[y][x]=struct.unpack('L',file.read(4))[0]

I'm not a Python developer, but this seems to make sense as you only want the first element.
hope it works

[fromagique]

Here's another possible method:

Code: Select all

# Grab the struct module

import struct

# Initialize your data structure as an empty list

data = []

# Iterate through your list of filenames
# (no need to use 'numfiles')

for fname in dirList:

    # Open our current file and read 32 + (2048*4) bytes.
    # Because we didn't save the result of 'open', it goes
    # away automatically like the 'with' statement.
    
    raw = open(fname, 'rb').read(32+(2048*4))

    # Unpack the data all in one shot:
    #
    # '=2048L' means 2048  * 4 bytes of data (using "standard"
    # size and alignment, signified by the '=' at the start.)
    #
    # This returns a tuple of 2048 items, which you can
    # directly append to your data structure.
    # We pass everything after the first 32 bytes of 'raw'
    # to the unpack function with the slice '[32:]'.

  data.append(struct.unpack('=2048L',raw[32:]))
  
print data[0][2] # if that's the data point you want.

Hope this helps. You could probably do this as one huge list comprehension but I didn't want to go that far.

[fromagique]

I forgot to add to my above post that you don't need to do preinitialization of arrays like you did in your code sample. I understand your intent, but in Python what you're actually doing is building a list of lists, each list of which contains references to 2048 number objects, each with the value 0. And then you immediately throw these '0' objects away when you reassign their slot in the list to the value you read from the file. So instead of doing the preinit step, in my code I initialize the 'data' object as an empty list, and build the lists as we iterate through the files.

The unpack function will return tuples of your 2048 values per file instead of lists, but they function the same way except you can't change them. If you need mutable versions, use the list() builtin like so (changing the line from my previous post):

Code: Select all

data.append(list(struct.unpack('=2048L',raw[32:])))

This will make an extra copy of the 2048 items per iteration, but since the anonymous tuple returned from the unpack function will immediately be thrown out, it's not that big of a deal. If you were dealing with a lot more items per file, it might be better to do it in a stream-oriented fashion as in your original sample, in order to save memory. But ~8K per iteration is small enough not to worry about.

Happy coding!