URL details: schwartz-lab-huji.github.io/publication/abc/
URL title:
ABC: Attention with Bounded-memory Control | Home
URL description:
Transformer architectures have achieved stateof-the-art results on a variety of natural language processing (NLP) tasks. However, their attention mechanism comes with a quadratic complexity in sequence lengths, making the computational overhead prohibitive, especially for long sequences. Attention context can be seen as a random-access memory with each token taking a slot. Under this perspective, the memory size grows linearly with the sequence length, and so does the overhead of reading from it. One way to
URL last crawled:
2022-07-06
URL speed:
0.770 MB/s,
downloaded in 0.030 seconds
We found no external links pointing to this url.