Memory-Efficient Embeddings
Creating smaller models with a new kind of embedding layer
Published in
13 min readJan 1, 2024
Whenever dealing with categorical data, beginners resort to one-hot encoding. This is often okay, but if you are dealing with thousands or even millions of categories, this approach becomes infeasible. This has the following reasons:
- Increased dimensionality: For each category, you get an…