Math clarification for learning NLP

Many times I come across something that looks familiar but then it turns out to be something entirely different.

For example,

Double Summation

At first glance, it appears there’s multiplication involved. But no, it’s just a sum. The double summations are like nested loops in computer programming. From left to right, you have the outer loop and inner loop(s).

Another mathematical symbol that looks familiar but is something else entirely is the double vertical lines with the matrix norm.

||x|| is not the same as |x|

It looks similar to the absolute value|x| but how different is it?

Looks similar but is very different.

Simply put, think of the norm as a function that deals with length. In NLP translations (example: English to French and vice versa), you may come across the Frobenius norm in the context of the Loss function.

Given X and Y, the objective is to learn R

Here is an example, where A = XR-Y

Frobenius norm example based on Transforming Word Vectors lecture from Coursera

and A is a matrix with m rows, n columns, and the triple bar represents “identical to”.

Putting it all together, the equation looks more manageable.

To get more context for this article, I highly recommend the lecture and notes from week 4 of Coursera’s Natural Language Processing with Classification and Vector Spaces — “Transforming word vectors” by deeplearning.ai