Microsoft’s Differential Transformer cancels consideration noise in LLMs

Bettering the capabilities of huge language fashions (LLMs) in retrieving in-prompt data stays an space of energetic analysis that may influence necessary functions corresponding to retrieval-augmented technology (RAG) and in-context studying (ICL).

Microsoft Analysis and Tsinghua College researchers have launched Differential Transformer (Diff Transformer), a brand new LLM structure that improves efficiency by amplifying consideration to related context whereas filtering out noise. Their findings, revealed in a analysis paper, present that Diff Transformer outperforms the basic Transformer structure in varied settings.

Transformers and the “lost-in-the-middle” phenomenon

The Transformer structure is the muse of most fashionable LLMs. It makes use of an consideration mechanism to weigh the significance of various components of the enter sequence when producing output. The eye mechanism employs the softmax operate, which normalizes a vector of values right into a chance distribution. In Transformers, the softmax operate assigns consideration scores to completely different tokens within the enter sequence.

Nevertheless, research have proven that Transformers battle to retrieve key data from lengthy contexts.

“We began by investigating the so-called ‘lost-in-the-middle’ phenomenon,” Furu Wei, Companion Analysis Supervisor at Microsoft Analysis, advised VentureBeat, referring to earlier analysis findings that confirmed that LLMs “do not robustly make use of information in long input contexts” and that “performance significantly degrades when models must access relevant information in the middle of long contexts.”

Wei and his colleagues additionally noticed that some LLM hallucinations, the place the mannequin produces incorrect outputs regardless of having related context data, correlate with spurious consideration patterns.

“For example, large language models are easily distracted by context,” Wei stated. “We analyzed the attention patterns and found that the Transformer attention tends to over-attend irrelevant context because of the softmax bottleneck.”

The softmax operate utilized in Transformer’s consideration mechanism tends to distribute consideration scores throughout all tokens, even these that aren’t related to the duty. This will trigger the mannequin to lose give attention to a very powerful components of the enter, particularly in lengthy contexts.

“Previous studies indicate that the softmax attention has a bias to learn low-frequency signals because the softmax attention scores are restricted to positive values and have to be summed to 1,” Wei stated. “The theoretical bottleneck renders [it] such that the classic Transformer cannot learn sparse attention distributions. In other words, the attention scores tend to flatten rather than focusing on relevant context.”

Differential Transformer

To handle this limitation, the researchers developed the Diff Transformer, a brand new basis structure for LLMs. The core concept is to make use of a “differential attention” mechanism that cancels out noise and amplifies the eye given to essentially the most related components of the enter.

The Transformer makes use of three vectors to compute consideration: question, key, and worth. The basic consideration mechanism performs the softmax operate on the complete question and key vectors.

The proposed differential consideration works by partitioning the question and key vectors into two teams and computing two separate softmax consideration maps. The distinction between these two maps is then used as the eye rating. This course of eliminates frequent noise, encouraging the mannequin to give attention to data that’s pertinent to the enter.

The researchers examine their strategy to noise-canceling headphones or differential amplifiers in electrical engineering, the place the distinction between two indicators cancels out common-mode noise.

Whereas Diff Transformer includes a further subtraction operation in comparison with the basic Transformer, it maintains effectivity due to parallelization and optimization strategies.

“In the experimental setup, we matched the number of parameters and FLOPs with Transformers,” Wei stated. “Because the basic operator is still softmax, it can also benefit from the widely used FlashAttention cuda kernels for acceleration.”

On reflection, the tactic utilized in Diff Transformer looks like a easy and intuitive answer. Wei compares it to ResNet, a preferred deep studying structure that launched “residual connections” to enhance the coaching of very deep neural networks. Residual connections made a quite simple change to the normal structure but had a profound influence.

“In research, the key is to figure out ‘what is the right problem?’” Wei stated. “Once we can ask the right question, the solution is often intuitive. Similar to ResNet, the residual connection is an addition, compared with the subtraction in Diff Transformer, so it wasn’t immediately apparent for researchers to propose the idea.”

Diff Transformer in motion

The researchers evaluated Diff Transformer on varied language modeling duties, scaling it up by way of mannequin dimension (from 3 billion to 13 billion parameters), coaching tokens, and context size (as much as 64,000 tokens).

Their experiments confirmed that Diff Transformer constantly outperforms the basic Transformer structure throughout completely different benchmarks. A 3-billion-parameter Diff Transformer educated on 1 trillion tokens confirmed constant enhancements of a number of proportion factors in comparison with equally sized Transformer fashions.

Additional experiments with completely different mannequin sizes and coaching dataset sizes confirmed the scalability of Diff Transformer. Their findings counsel that usually, Diff Transformer requires solely round 65% of the mannequin dimension or coaching tokens wanted by a basic Transformer to attain comparable efficiency.

The researchers additionally discovered that Diff Transformer is especially efficient in utilizing growing context lengths. It confirmed important enhancements in key data retrieval, hallucination mitigation, and in-context studying.

Whereas the preliminary outcomes are promising, there’s nonetheless room for enchancment. The analysis staff is engaged on scaling Diff Transformer to bigger mannequin sizes and coaching datasets. Additionally they plan to increase it to different modalities, together with picture, audio, video, and multimodal information.

The researchers have launched the code for Diff Transformer, carried out with completely different consideration and optimization mechanisms. They imagine the structure might help enhance efficiency throughout varied LLM functions.

“As the model can attend to relevant context more accurately, it is expected that these language models can better understand the context information with less in-context hallucinations,” Wei stated. “For example, for the retrieval-augmented generation settings (such as Bing Chat, Perplexity, and customized models for specific domains or industries), the models can generate more accurate responses by conditioning on the retrieved documents.”