Hey Guys,

I have a couple of doubts in the markdown/implementation of `our_hash_vectors`

.

### Step 1 (Markdown)

Each vector will be hashed into a hash table and into `rot_size//2`

buckets. We use `rot_size//2`

to reduce computation.

- I am a little confused about the highlighted markdown, since in step 3, it is mentioned that n_buckets = rotsize. Moreover, in the function’s implementation too, we have used
`rot_size = n_buckets`

. Now, the below statement is also mentioned in step 1.
- So, is the above markdown incorrect, or like, is it only temporarily true, till Step 3 to be more precise.

Later in the routine we will form the negative rotations with a simple negation and concatenate to get a full `rot_size`

number of rotations.

### Step 2 (Code)

rotated_vecs = np.einsum(“tf,fhb->htb”, vecs, random_rotations)

Last, I am unable to find the meaning of the subscripts used in the above statement. Can someone provide the link which describes these subscripts, or elaborate these subscripts? I was unable to find the link, that’s why the doubt.

Cheers,

Elemento

Hi @Elemento

or like, is it only temporarily true, till Step 3 to be more precise.

Yes, this is the case.

For example, (in case of `n_hashes=3`

, `n_buckets = 4`

, `seq_len = 16`

) at the end of Step 2, `rotated_vecs`

could be `rot_size//2`

for each hash, like :

and after Step 3 would concatenate each:

If I remember correctly, jax uses the same subscripts as numpy (numpy.einsum — NumPy v1.26 Manual). It is just matrix multiplication with different shapes 2d and 3d.

Cheers

Hey @arvyzukai,

Thanks a lot for your quick response. I am still a little confused in my second query, since even in numpy, I am unable to find an exhaustive list of subscripts, which can tell me, the meaning of each character. Can you please help me with finding that list?

Cheers,

Elemento

No problem @Elemento

I cannot explain all the variants the `np.einsum`

could be used (there are a lot of examples in the documentation I posted previously), but I can try to explain the one used in the Lab.

`rotated_vecs = np.einsum(“tf,fhb->htb”, vecs, random_rotations)`

The best way for me is to explain with an example:

For example, if the `vecs`

array is of shape (16 x 5, here `t=16`

, `f=5`

) and `random_rotations`

is of shape (5 x 3 x 2, here `f=5`

, `h=3`

, `b=2`

), then the result is matrix multiplication (or dot product) of shape (3 x 16 x 2, or `htb`

). The matrix multiplication is done on the `f`

(5) dimension. So the `rotated_vecs`

has the shape of (3 x 16 x 2).

I hope that makes it somewhat clearer? If not, feel free to ask.

Cheers

Hey @arvyzukai,

Thanks a lot for the explanation. Just one last small query. Do these `tf`

and `fhb`

subscripts are specific to denote the dimensions, or can we replace them with any alphabet. For instance, can we use `ab, bcd -> cad`

, and will it produce the same results?

Cheers

Hey @Elemento

Yes, that would be equivalent to `tf,fhb->htb`

Cheers