Please clarify the following Python NumPy array initialization and splicing examples
multidimensional-array, numpy, python
Solution
Question 1
a) looks like a list of tuples from print, except without the comma between the tuples. If I print it with repr(a), it even has the commas. Even so, it should no longer be considered the same as b) correct?
Absolutely. `a` and `b` have different types: `type(a)` is `numpy.ndarray`, `type(b)` is `list`
Question 2
Is the below failing because the dtype does not match the list that I passed in?
No - the issue is that you're trying to fill it with a list of lists, rather than a list of tuples as you did with `a`. See here. I'm not totally sure what the deep reason is for this behaviour, but I suspect it has to do with tuples being immutable objects, whereas lists (and arrays) are mutable.
Question 3
Is the below failing because I am not allowed to specify a dtype that is a list?
Yes, and furthermore you would also fail to fill `d` with a list of lists (see previous answer).
Question 4
Why would e) below which is also a list of list work, but d) fail?
Look at the `dtype` of `e` - it is `|S1`, i.e. every element in the array is a string of length 1. If you don't specify a `dtype` for the array constructor, the type will be determined as the minimum type required to hold all of the objects in the sequence. In this case, since you handed it a sequence containing some strings, it will upcast the integers to strings.
Question 5
Same example as e), but this time initializing with list of tuples he print out of f) is identical as e), so initializing with list of tuples and list of list are really identical then?
Again, since you don't give the constructor a `dtype`, everything will get upcast to `|S1`.
Question 6
Same example as e and f, but this time initializing from file Minor difference in quoting for the print out. There should be no difference # between the array generated like this and e) and f) right?
No, now you're telling the constructor to create an array with `dtype=object`, whereas `e` and `f` will have `dtype=|S1`.
Question 7
Why would splicing e below work, but a fail above with Index error with the same syntax?
Look at `a.shape` - you'll see that it's `(3,)`, i.e. `a` is a 1d vector of length 3. Although it does have fields that you can index it by, it has no second dimension for you to index into. By contrast, `e.shape` is `(3,3)`, so you can index it by column.
Problem
I am using Python version 2.6 and am learning NumPy version 1.3. I have tried out several NumPy array initialization and column splicing examples below, and added some inline questions as comments and a list of findings in the end. Hopefully someone can explain to me what is behind the differences in behaviors. Lots of inter-related questions and a rather long post, but each example is small, feel free to just answer one or a couple. ``` import numpy as np print "Initializing a number of numpy arrays:\n" ``` a) Initialize from a list of tuples ``` a = np.zeros((3,),dtype=('i4,i4,a1')) a[:] = [(1, 2, 'A'), (3, 4, 'B'),(5, 6, 'A')] print "a: " print a # print => [(1, 2, 'A') (3, 4, 'B') (5, 6, 'A')] print repr(a) # print => array([(1, 2, 'A'), (3, 4, 'B'), (5, 6, 'A')], # dtype=[('f0', '<i4'), ('f1', '<i4'), ('f2', '|S1')] print '\n' ``` b) A normal list of tuples ``` b = []; b[:] = [(1, 2, 'A'), (3, 4, 'B'),(5, 6, 'A')] print "b: " print b # print => [(1, 2, 'A'), (3, 4, 'B'), (5, 6, 'A') print '\n' ``` Question 1: a) looks like a list of tuples from print, except without the comma between the tuples. If I print it with repr(a), it even has the commas. Even so, it should no longer be considered the same as b), correct? c) Fail: Try to initialize array returned from np.zeroes as a list of list Question 2: Is the below failing because the dtype does not match the list that I passed in? ``` c = np.zeros((3,),dtype=('i4,i4,a1')) #c[:] = [[1, 2, 'A'], [3, 4, 'B'],[5, 6, 'A']] # TypeError: expected a readable buffer object print '\n' ``` d) Fail: Same as c) but try to set the dtype as a list Question 3: Is the below failing, because I am not allowed to specify a dtype that is a list? ``` #d = np.zeros((3,),dtype=['i4,i4,a1']) # TypeError: data type not understood #d[:] = [[1, 2, 'A'], [3, 4, 'B'],[5, 6, 'A']] print '\n' ``` e) Try to initialize array using np.array from a list of a list Question 4: Why would e) below which is also a list of list work, but d) fail? ``` e = np.array( [[1, 2, 'A'], [3, 4, 'B'],[5, 6, 'A']] ) print "e: " print e # print => [['1' '2' 'A'] # ['3' '4' 'B'] # ['5' '6' 'A']] print '\n' ``` f) Try to initialize array using np.array from a list of a tuples Question 5: Same example as e), but this time initializing with list of tuples he print out of f) is identical as e), so initializing with list of tuples and list of list are really identical then? ``` f = np.array( [(1, 2, 'A'), (3, 4, 'B'),(5, 6, 'A')] ) print "f: " print f # print => [['1' '2' 'A'] # ['3' '4' 'B'] # ['5' '6' 'A']] print '\n' ``` g) Try to initialize array using np.array from a CSV file Question 6: Same example as e and f, but this time initializing from file Minor difference in quoting for the print out. There should be no difference # between the array generated like this and e) and f) right? ``` from StringIO import StringIO data = StringIO( """ 1, 2, A 3, 4, B 5, 6, A """.strip()) g = np.genfromtxt(data, dtype=object, delimiter=',') print "g: " print g # print => [[1 2 A] # [3 4 B] # [5 6 A]] print '\n' ``` h) Splicing the NumPy arrays by column ``` #print "a: " #print a[:,2] # IndexError: invalid index print "a: " print a['f2'] # This is ok though # Splicing a normal list of tuples if not expected to work #print "b: " #print b[:,2] # IndexError: invalid index ``` Question 7 Why would splicing e below work, but a fail above with Index error with the same syntax? ``` print "e: " print e[:,2] # print => ['A' 'B' 'A'] print "f: " print f[:,2] # print => ['A' 'B' 'A'] print "g: " print g[:,2] # print => [A B A] ``` Finding 1: Initializing numpy.ndarray by using nd.array and a list of tuples, list of list, or CSV file are identical. This is maybe contrary to what this other answer that I viewed that says np.array expects a list of a tuples, Stack Overflow question Define dtypes in NumPy using a list?. Finding 2: Initializing numpy.ndarray by using np.zeroes, I am unable to initialize the ndarray from a list of a list. Finding 3: For column splicing, initializing numpy.ndarray by using nd.array, I could do a column splice (that is, e[:,2], but the syntax of splicing, using the np.zeroes initialization method is different a['f2']. A normal list of tuples cannot be spliced.