The one hot representation of a word is a vector the length of the vocabulary, with all 0’s except a 1 to represent that particular word.
This encoding has problems:
- There is no notion of similarity.
- Large number of dimensions.
The one hot representation is an example of a local representation.