Numpy snippets¶
Jérémie Decock (www.jdhp.org)
This notebook is inspired by the following page: https://docs.scipy.org/doc/numpy-dev/user/numpy-for-matlab-users.html
Import directives¶
In [2]:
import numpy as np
import math
Create arrays¶
In [3]:
np.array([1, 2, 3])
Out[3]:
In [4]:
np.array([[1, 2, 3],[4, 5, 6]])
Out[4]:
Special matrices¶
In [5]:
np.zeros(3)
Out[5]:
In [6]:
np.zeros((3, 4))
Out[6]:
In [7]:
np.ones(3)
Out[7]:
In [8]:
np.ones((3, 4))
Out[8]:
In [9]:
np.eye(3)
Out[9]:
Arange¶
In [10]:
np.arange(10)
Out[10]:
In [11]:
np.arange(10, 20)
Out[11]:
In [12]:
np.arange(10, 20, 2)
Out[12]:
Linspace¶
In [13]:
np.linspace(0., 2., 5)
Out[13]:
Meshgrid¶
In [14]:
xx, yy = np.meshgrid([1, 2, 3], [4, 5, 6])
print(xx)
print()
print(yy)
Random¶
Uniform distribution in [0, 1]¶
In [15]:
np.random.rand(3)
Out[15]:
In [16]:
np.random.rand(3, 4)
Out[16]:
Poisson distribution¶
In [17]:
np.random.poisson(10, size=[3, 4])
Out[17]:
Multivariate normal distribution¶
In [18]:
mu = np.array([0., 0.])
cov = np.array([[1., 0.3],
[0.3, 1.]])
num_points = 10
np.random.multivariate_normal(mu, cov, num_points)
Out[18]:
Print options¶
In [19]:
np.get_printoptions()
Out[19]:
Print large arrays¶
In [20]:
default_threshold = np.get_printoptions()["threshold"]
default_threshold
Out[20]:
Arrays with more than default_threshold elements are truncated.
In [21]:
max_size = math.ceil(math.sqrt(default_threshold))
max_size
Out[21]:
In [22]:
a = np.random.randint(1, size=[max_size + 1, max_size + 1])
a
Out[22]:
Print the full array (set threshold to infinity):
In [23]:
np.set_printoptions(threshold=np.inf)
a
Out[23]:
Go back to the default threshold:
In [24]:
np.set_printoptions(threshold=default_threshold)
a
Out[24]:
Dimension and shape¶
In [25]:
a = np.array([[1, 2, 3],[4, 5, 6]])
Number of dimensions:
In [26]:
a.ndim
Out[26]:
Number of elements:
In [27]:
a.size
Out[27]:
Number of elements per dimension:
In [28]:
a.shape
Out[28]:
Convert¶
In [29]:
l = [[1, 2, 3],[4, 5, 6]]
a = np.array([[1, 2, 3],[4, 5, 6]])
Python list to Numpy array¶
In [30]:
np.array(l)
Out[30]:
Numpy array to Python list¶
In [31]:
a.tolist()
Out[31]:
Copy¶
np.copy()¶
In [32]:
a = np.array([[1, 2, 3],[4, 5, 6]])
a
Out[32]:
In [33]:
b = a.copy()
b
Out[33]:
In [34]:
a[0,0] = 10
print(a)
print(b)
np.astype()¶
In [35]:
a = np.array([[1, 2, 3],[4, 5, 6]])
a
Out[35]:
In [36]:
b = a.astype('float64', copy=True)
b
Out[36]:
In [37]:
a[0,0] = 10
print(a)
print(b)
Access elements¶
In [38]:
a = np.arange(6)
In [39]:
a
Out[39]:
In [40]:
a[0]
Out[40]:
In [41]:
a[-1]
Out[41]:
Slices¶
In [42]:
a[1:4]
Out[42]:
In [43]:
a = np.array([[1, 2, 3, 4, 5, 6],
[10, 20, 30, 40, 50, 60],
[100, 200, 300, 400, 500, 600]])
a
Out[43]:
In [44]:
a[0,1]
Out[44]:
In [45]:
a[1, :]
Out[45]:
In [46]:
a[1, ::2]
Out[46]:
In [47]:
a[:, 1]
Out[47]:
In [48]:
a[0:2, 2:4]
Out[48]:
In [49]:
a[1:, 1:]
Out[49]:
In [50]:
a[:-1, :-1]
Out[50]:
Ellipsis¶
"The ellipsis is used to slice high-dimensional data structures.
It's designed to mean at this point, insert as many full slices (:) to extend the multi-dimensional slice to all dimensions."
https://stackoverflow.com/questions/118370/how-do-you-use-the-ellipsis-slicing-syntax-in-python
In [51]:
a = np.arange(2**3).reshape(2, 2, 2)
a
Out[51]:
To select all first elements in the last (3rd) dimension
In [52]:
a[..., 0]
Out[52]:
is equivalent to
In [53]:
a[:, :, 0]
Out[53]:
To select all first elements in the first (1st) dimension
In [54]:
a[0, ...]
Out[54]:
is equivalent to
In [55]:
a[0, :, :]
Out[55]:
Filter¶
In [56]:
a = np.array([[1, 2, 3, 4, 5, 6],
[10, 20, 30, 40, 50, 60],
[100, 200, 300, 400, 500, 600]])
a
Out[56]:
Boolean matrix whose i,jth element is (a_ij > 5)¶
In [57]:
(a>5)
Out[57]:
Find the indices where (a > 5)¶
In [58]:
np.nonzero(a>5)
Out[58]:
Set or copy a with elements greater than 5 zeroed out¶
In [59]:
a * (a<=5)
Out[59]:
In [60]:
a[a>5] = 0
a
Out[60]:
Select indices satisfying multiple conditions¶
Short version¶
In [61]:
a = np.array([[-1, 7, 3], [-11, -5, 20]])
a
Out[61]:
In [62]:
a[(a > -10) & (a < 10)] = 0
a
Out[62]:
In [63]:
a[(a < -10) | (a > 10)] = 1
a
Out[63]:
Detailed version¶
In [64]:
a = np.array([[-1, 7, 3], [-11, -5, 20]])
a
Out[64]:
In [65]:
m1 = (a > -10)
m2 = (a < 10)
print(m1)
print(m2)
print(m1 & m2)
a[m1 & m2] = 0
a
Out[65]:
Concatenate¶
Append 1D arrays¶
In [66]:
a = np.array([])
a = np.append(a, 3)
a
Out[66]:
Performance test¶
It's probably not a good idea to use np.append to often as it makes a copy of the array each time it is called...
In [67]:
%%timeit
a = np.array([])
for i in range(10000):
a = np.append(a, i)
Lists use a different data structure that makes them more efficient for repeated additions...
In [68]:
%%timeit
l = []
for i in range(10000):
l.append(i)
a = np.array(l)
In this case, the better option is probably the following:
In [69]:
%%timeit
a = np.array([i for i in range(10000)])
Concatenate 1D arrays¶
In [70]:
a = np.zeros(3)
b = np.ones(3)
print("a:", a)
print("b:", b)
In [71]:
np.concatenate([a, b])
Out[71]:
In [72]:
np.hstack([a, b])
Out[72]:
Concatenate 2D arrays¶
In [73]:
a = np.zeros([2, 3])
b = np.ones([2, 3])
In [74]:
a
Out[74]:
In [75]:
b
Out[75]:
On the first dimension¶
Using vstack:
In [76]:
np.vstack([a, b])
Out[76]:
Using concatenate:
In [78]:
np.concatenate([a, b], axis=0)
Out[78]:
On the second dimension¶
Using hstack:
In [80]:
np.hstack([a, b])
Out[80]:
Using concatenate:
In [82]:
np.concatenate([a, b], axis=1)
Out[82]:
Join a sequence of arrays along a new axis¶
The axis parameter specifies the index of the new axis in the dimensions
of the result.
In [84]:
a = np.zeros([2, 3])
b = np.ones([2, 3])
In [85]:
a
Out[85]:
In [86]:
b
Out[86]:
Along axis 0¶
In [87]:
np.stack([a, b], axis=0)
Out[87]:
In [88]:
np.stack([a, b], axis=0).shape
Out[88]:
Along axis 1¶
In [89]:
np.stack([a, b], axis=1)
Out[89]:
In [90]:
np.stack([a, b], axis=1).shape
Out[90]:
Along axis 2¶
In [91]:
np.stack([a, b], axis=2)
Out[91]:
In [92]:
np.stack([a, b], axis=2).shape
Out[92]:
Tile¶
In [93]:
a = np.array([[1, 2, 3], [4, 5, 6]])
np.tile(a, (2, 3))
Out[93]:
Reshape or transpose¶
In [94]:
a = np.array([[1, 2, 3], [4, 5, 6]])
Transpose¶
In [95]:
a.T
Out[95]:
Flatten¶
In [96]:
a.flatten()
Out[96]:
Reshape¶
In [97]:
a = np.arange(6)
a
Out[97]:
Row vector to column vector¶
In [98]:
a.reshape([-1, 1])
Out[98]:
Vector to matrix¶
In [99]:
a.reshape([2, 3])
Out[99]:
In [100]:
a.reshape([3, 2])
Out[100]:
Repeat¶
In [101]:
a = np.arange(3)
a
Out[101]:
In [102]:
np.repeat(a, 5)
Out[102]:
In [103]:
a = np.arange(3).reshape([-1, 1])
a
Out[103]:
In [104]:
np.repeat(a, 5, axis=0)
Out[104]:
In [105]:
a = np.array([[1, 3, 5],[2, 4, 6]])
a
Out[105]:
In [106]:
np.repeat(a, 5, axis=0)
Out[106]:
Sort¶
Return the indices that would sort an array¶
In [107]:
a = np.array([8, 5, 1])
a
Out[107]:
In [108]:
a.argsort()
Out[108]:
Sort an array by the $n^{\text{th}}$ column¶
In [109]:
a = np.array([[4, 4, 2],
[8, 5, 1],
[7, 0, 0],
[3, 1, 1],
[3, 0, 5]])
a
Out[109]:
In [110]:
n = 0 # the column sorted by
In [111]:
a[a[:,n].argsort()]
Out[111]:
In [112]:
n = 1 # the column sorted by
In [113]:
a[a[:,n].argsort()]
Out[113]:
In [114]:
n = 2 # the column sorted by
In [115]:
a[a[:,n].argsort()]
Out[115]:
Aggregation / reduction¶
In [116]:
a = np.array([[1, 2, 3], [4, 5, 6]])
Change the axis value in the following functions to aggregate along a given axis.
In [117]:
np.sum(a, axis=None)
Out[117]:
In [118]:
np.cumsum(a, axis=None)
Out[118]:
In [119]:
np.diff(a.ravel())
Out[119]:
In [120]:
np.mean(a, axis=None)
Out[120]:
In [121]:
np.var(a, axis=None)
Out[121]:
In [122]:
np.std(a, axis=None)
Out[122]:
In [123]:
np.median(a, axis=None)
Out[123]:
In [124]:
np.min(a, axis=None)
Out[124]:
In [125]:
np.max(a, axis=None)
Out[125]:
In [126]:
np.prod(a, axis=None)
Out[126]:
In [127]:
np.cumprod(a, axis=None)
Out[127]:
Compute the histogram of a set of data (with a specific binning)¶
In [128]:
a = np.array([1, 1, 3, 2, 2, 2])
a
Out[128]:
All but the last (righthand-most) bin is half-open. In other words,
if bins is:
[1, 2, 3, 4]
then the first bin is [1, 2) (including 1, but excluding 2) and
the second [2, 3). The last bin, however, is [3, 4], which
includes 4.
In [129]:
bins = np.array([1, 2, 3, 4])
bins
Out[129]:
In [130]:
hist, bins_ = np.histogram(a, bins=bins)
hist
Out[130]:
Linear algebra¶
Dot product of two arrays¶
In [131]:
a = np.array([1, 2, 3])
b = np.array([10, 20, 30])
In [132]:
np.dot(a, b)
Out[132]:
In [133]:
a.dot(b)
Out[133]:
Compute the (multiplicative) inverse of a matrix¶
In [134]:
a = np.random.normal(size=(3, 3))
a
Out[134]:
In [135]:
np.linalg.inv(a)
Out[135]:
Compute the eigenvalues and right eigenvectors of a square array¶
In [136]:
a = np.random.normal(size=(3, 3))
a
Out[136]:
In [137]:
np.linalg.eig(a)
Out[137]:
Singular Value Decomposition¶
In [138]:
a = np.random.normal(size=(3, 3))
a
Out[138]:
In [139]:
U, s, V = np.linalg.svd(a)
print(U, s, V)
Solve a linear matrix equation, or system of linear scalar equations¶
In [140]:
a = np.array([[3, 1], [1, 2]])
b = np.array([9, 8])
np.linalg.solve(a, b)
Out[140]:
Diagonals¶
Extract the diagonal:
In [141]:
a = np.array([[1, 2, 3], [4, 5, 6], [7, 8, 9]])
np.diag(a)
Out[141]:
Make a diagonal matrix:
In [142]:
d = np.array([1, 2, 3])
np.diag(d)
Out[142]:
Trace¶
In [143]:
a = np.array([[1, 2, 3], [4, 5, 6], [7, 8, 9]])
np.trace(a)
Out[143]:
Upper and lower triangles of an array¶
In [144]:
a = np.array([[1, 2, 3], [4, 5, 6], [7, 8, 9]])
In [145]:
np.triu(a)
Out[145]:
In [146]:
np.tril(a)
Out[146]:
Data types¶
Get type¶
In [147]:
a = np.arange(-2., 2., 0.5)
a.dtype
Out[147]:
Size in memory (in bytes)¶
In [148]:
a = np.arange(-2., 2., 0.5)
Per item:
In [149]:
a.itemsize
Out[149]:
Full array:
In [150]:
a.nbytes
Out[150]:
Init¶
In [151]:
a = np.zeros(3)
a.dtype
Out[151]:
In [152]:
a = np.zeros(3, dtype=np.bool)
a.dtype
Out[152]:
In [153]:
a = np.zeros(3, dtype=np.int)
a.dtype
Out[153]:
In [154]:
a = np.zeros(3, dtype=np.int8)
a.dtype
Out[154]:
In [155]:
a = np.zeros(3, dtype=np.uint8)
a.dtype
Out[155]:
Conversions¶
In [156]:
a = np.arange(-2., 2., 0.5)
a
Out[156]:
In [157]:
a.astype(np.bool)
Out[157]:
In [158]:
a.astype(np.int)
Out[158]:
In [159]:
a.astype(np.int8)
Out[159]:
In [160]:
a.astype(np.uint8)
Out[160]:
Masked arrays¶
Without masked array¶
In [161]:
a = np.array([[np.nan, 2, 3], [1, np.nan, 6]])
a
Out[161]:
In [162]:
a.min()
Out[162]:
In [163]:
np.nanmin(a)
Out[163]:
In [164]:
a.max()
Out[164]:
In [165]:
np.nanmax(a)
Out[165]:
In [166]:
a.mean()
Out[166]:
In [167]:
np.nanmean(a)
Out[167]:
In [168]:
a.shape
Out[168]:
With masked array¶
In [169]:
ma = np.ma.masked_where(np.isnan(a), a)
ma
Out[169]:
In [170]:
ma.min()
Out[170]:
In [171]:
ma.max()
Out[171]:
In [172]:
ma.mean()
Out[172]:
In [173]:
ma.shape
Out[173]: