NumPy - What is broadcasting?

Question

I don't understand broadcasting. The documentation explains the rules of broadcasting but doesn't seem to define it in English. My guess is that broadcasting is when NumPy fills a smaller dimensional array with dummy data in order to perform an operation. But this doesn't work:

>>> x = np.array([1,3,5])
>>> y = np.array([2,4])
>>> x+y
*** ValueError: operands could not be broadcast together with shapes (3,) (2,) 

The error message hints that I'm on the right track, though. Can someone define broadcasting and then provide some simple examples of when it works and when it doesn't?

Answer 1

1.What is Broadcasting? Broadcasting is a Tensor operation. Helpful in Neural Network (ML, AI)

2.What is the use of Broadcasting?

• Without Broadcasting addition of only identical Dimension(shape) Tensors is supported.

• Broadcasting Provide us the Flexibility to add two Tensors of Different Dimension.

  for Example: adding a 2D Tensor with a 1D Tensor is not possible without broadcasting see the image explaining Broadcasting pictorially

Run the Python example code understand the concept

x = np.array([1,3,5,6,7,8]) 
y = np.array([2,4,5])
X=x.reshape(2,3)

x is reshaped to get a 2D Tensor X of shape (2,3), and adding this 2D Tensor X with 1D Tensor y of shape(1,3) to get a 2D Tensor z of shape(2,3)

print("X =",X)
print("\n y =",y)
z=X+y
print("X + y =",z)

You are almost correct about smaller Tensor, no ambiguity, the smaller tensor will be broadcasted to match the shape of the larger tensor.(Small vector is repeated but not filled with Dummy Data or Zeros to Match the Shape of larger).

3. How broadcasting happens? Broadcasting consists of two steps:

1 Broadcast axes are added to the smaller tensor to match the ndim of the larger tensor.

2 The smaller tensor is repeated alongside these new axes to match the full shape of the larger tensor.

4. Why Broadcasting not happening in your code? your code is working but Broadcasting can not happen here because both Tensors are different in shape but Identical in Dimensional(1D). Broadcasting occurs when dimensions are nonidentical. what you need to do is change Dimension of one of the Tensor, you will experience Broadcasting.

5. Going in Depth. Broadcasting(repetition of smaller Tensor) occurs along broadcast axes but since both the Tensors are 1 Dimensional there is no broadcast Axis. Don't Confuse Tensor Dimension with the shape of tensor, Tensor Dimensions are not same as Matrices Dimension.

Answer 2

The term broadcasting describes how numpy treats arrays with different shapes during arithmetic operations.

It's basically a way numpy can expand the domain of operations over arrays.

The only requirement for broadcasting is a way aligning array dimensions such that either:

• Aligned dimensions are equal.
• One of the aligned dimensions is 1.

So, for example if:

x = np.ndarray(shape=(4,1,3))
y = np.ndarray(shape=(3,3))

You could not align x and y like so:

4 x 1 x 3
3 x 3

But you could like so:

4 x 1 x 3
    3 x 3

How would an operation like this result?

Suppose we have:

x = np.ndarray(shape=(1,3), buffer=np.array([1,2,3]),dtype='int')
array([[1, 2, 3]])

y = np.ndarray(shape=(3,3), buffer=np.array([1,1,1,1,1,1,1,1,1]),dtype='int')
array([[1, 1, 1],
       [1, 1, 1],
       [1, 1, 1]])

The operation x + y would result in:

array([[2, 3, 4],
       [2, 3, 4],
       [2, 3, 4]])

I hope you caught the drift. If you did not, you can always check the official documentation here.

Cheers!

Answer 3

Broadcasting is numpy trying to be smart when you tell it to perform an operation on arrays that aren't the same dimension. For example:

2 + np.array([1,3,5]) == np.array([3, 5, 7])

Here it decided you wanted to apply the operation using the lower dimensional array (0-D) on each item in the higher-dimensional array (1-D).

You can also add a 0-D array (scalar) or 1-D array to a 2-D array. In the first case, you just add the scalar to all items in the 2-D array, as before. In the second case, numpy will add row-wise:

In [34]: np.array([1,2]) + np.array([[3,4],[5,6]])
Out[34]: 
array([[4, 6],
       [6, 8]])

There are ways to tell numpy to apply the operation along a different axis as well. This can be taken even further with applying an operation between a 3-D array and a 1-D, 2-D, or 0-D array.

Answer 4

>>> x = np.array([1,3,5])
>>> y = np.array([2,4])
>>> x+y
*** ValueError: operands could not be broadcast together with shapes (3,) (2,)

Broadcasting is how numpy do math operations with array of different shapes. Shapes are the format the array has, for example the array you used, x , has 3 elements of 1 dimension; y has 2 elements and 1 dimension.

To perform broadcasting there are 2 rules: 1) Array have the same dimensions(shape) or 2)The dimension that doesn't match equals one.

for example x has shape(2,3) [or 2 lines and 3 columns]; y has shape(2,1) [or 2 lines and 1 column]

Can you add them? x + y? Answer: Yes, because the mismatched dimension is equal to 1 (the column in y). If y had shape(2,4) broadcasting would not be possible, because the mismatched dimension is not 1.

In the case you posted: operands could not be broadcast together with shapes (3,) (2,); it is because 3 and 2 mismatched altough both have 1 line.