Which term describes the architecture that processes the entire input sentence at once, enabling attention across the full sequence?

Get ready for the GARP Risk and AI Exam with flashcards and multiple choice questions. Each question comes with hints and explanations. Prepare for success!

Multiple Choice

Which term describes the architecture that processes the entire input sentence at once, enabling attention across the full sequence?

Transformers are designed to process the entire input sequence in parallel and allocate attention across all positions through self-attention. In practice, each token can directly attend to every other token in the same layer, enabling full-sequence context without processing tokens one after another. This parallelizable, all-at-once computation is why it's distinct from other architectures. Recurrent neural networks read tokens sequentially, which limits parallelism and can struggle with long-range dependencies. Convolutional networks focus on local windows and need many layers to expand their reach, so they don’t inherently provide attention across the full sequence in a single pass. While attention mechanisms describe weighing across positions, Transformers embody an architecture that applies this idea across the entire sequence, making them the correct term.

Transformers are designed to process the entire input sequence in parallel and allocate attention across all positions through self-attention. In practice, each token can directly attend to every other token in the same layer, enabling full-sequence context without processing tokens one after another. This parallelizable, all-at-once computation is why it's distinct from other architectures. Recurrent neural networks read tokens sequentially, which limits parallelism and can struggle with long-range dependencies. Convolutional networks focus on local windows and need many layers to expand their reach, so they don’t inherently provide attention across the full sequence in a single pass. While attention mechanisms describe weighing across positions, Transformers embody an architecture that applies this idea across the entire sequence, making them the correct term.

More practice questions