# Why is the Weight Matrix the transposed of NN's?

**URL:** <https://community.deeplearning.ai/t/why-is-the-weight-matrix-the-transposed-of-nns/12316>\
**Category:** Neural Networks and Deep Learning\
**Tags:** coursera-platform\
**Created:** [June 6, 2021, 5:53pm UTC](https://community.deeplearning.ai/t/why-is-the-weight-matrix-the-transposed-of-nns/12316 "2021-06-06T17:53:08Z")\
**Posts on this page:** 3\
**Page:** 1

<div class="post-metadata">

**Author:** ![crackingthecode](https://sea1.discourse-cdn.com/dlai/user_avatar/community.deeplearning.ai/crackingthecode/32/2653_2.png) [@crackingthecode](https://community.deeplearning.ai/u/crackingthecode)\
**Post date:** [June 6, 2021, 5:53pm UTC](https://community.deeplearning.ai/t/why-is-the-weight-matrix-the-transposed-of-nns/12316/1 "2021-06-06T17:53:08Z")

</div>

Z calculation for neural network is:  
 ![Screen Shot 2021-06-06 at 12.39.43 PM](https://global.discourse-cdn.com/dlai/original/2X/3/325f1d710a84b9d3dc1e6f0cb3e392747db02946.png)

Please note that here W is NOT transposed

 ![Screen Shot 2021-06-06 at 12.49.31 PM](https://global.discourse-cdn.com/dlai/original/2X/3/3ca5c46513508d96d77739eea9d09325ec6f2a87.png)

However, for the logistic regression it is transposed

What’s the reason for flipping the weight matrix?

---

<div class="post-metadata">

**Author:** ![vjmalkoti](https://avatars.discourse-cdn.com/v4/letter/v/e0b2c6/32.png) [@vjmalkoti](https://community.deeplearning.ai/u/vjmalkoti)\
**Post date:** [June 9, 2021, 8:49pm UTC](https://community.deeplearning.ai/t/why-is-the-weight-matrix-the-transposed-of-nns/12316/2 "2021-06-09T20:49:53Z")

</div>

We apply the dot product between W and X to form the equation `w1x1 + w2x2 + ... + wnxn` for each x example. The dot product requires that number of columns in first matrix should match the number of rows in second, `A (p, q) . B (q, r)`.

In logistic regression initializations, W and X have same orientation as there’s a one-to-one relation. Therefore, we apply transpose operation on the matrix to make them dot-product compatible such that the dot-product results in a single value for each x example. This value for each example is its y value.

In neural network, the matrix orientations are different. Dimensions of first layer’s W is  
`number of units in first layer X number of input features`. That means, each row corresponds to one activation which has weights for each of the input features - a many-to-many relation. The dimensions of the X is number of `features X number of examples`. Since the matrices are already dot-product compatible, there’s no need to transpose either matrix. Similarly in second, third and deeper layers, the dimensions of W are `number of units in the layer X number of units in previous layer` which makes them compatible with previous layer’s A matrices for dot-product. The final layer has units `number of output classes X number of units in previous layer` which maps the calculated value to output classes.

---

<div class="post-metadata">

**Author:** ![crackingthecode](https://sea1.discourse-cdn.com/dlai/user_avatar/community.deeplearning.ai/crackingthecode/32/2653_2.png) [@crackingthecode](https://community.deeplearning.ai/u/crackingthecode)\
**Post date:** [June 16, 2021, 5:31am UTC](https://community.deeplearning.ai/t/why-is-the-weight-matrix-the-transposed-of-nns/12316/3 "2021-06-16T05:31:05Z")

</div>

Thanks for your answer. My question is why are the orientations different? Seems like a strange convention.

> [@vjmalkoti](#):
>
> In neural network, the matrix orientations are different.