What term describes a long stretch where nothing seems to change despite continued effort, indicating stagnation in learning?

Get ready for the GARP Risk and AI Exam with flashcards and multiple choice questions. Each question comes with hints and explanations. Prepare for success!

Multiple Choice

What term describes a long stretch where nothing seems to change despite continued effort, indicating stagnation in learning?

Plateaus describe a long stretch where progress stalls even though you keep training. In optimization terms, you can encounter flat regions of the loss or objective surface where the gradient is near zero, so parameter updates produce little to no improvement. This shows up in practice as a learning curve that barely moves for many iterations, even though effort continues. To break through, you can try adjusting the learning rate or its schedule, using momentum or adaptive optimizers, adding regularization or architectural tweaks, or introducing methods to encourage exploration so the model can move into a region where learning resumes. Local optima would mean getting stuck at a peak or trough in the landscape, not necessarily a broad flat region of little change. Vanishing gradients involve gradients becoming extremely small as they backpropagate, which slows learning but isn’t specifically described as a long flat stretch of no change. Exploding gradients cause instability from gradients becoming too large, which is the opposite problem of stagnation.

Plateaus describe a long stretch where progress stalls even though you keep training. In optimization terms, you can encounter flat regions of the loss or objective surface where the gradient is near zero, so parameter updates produce little to no improvement. This shows up in practice as a learning curve that barely moves for many iterations, even though effort continues. To break through, you can try adjusting the learning rate or its schedule, using momentum or adaptive optimizers, adding regularization or architectural tweaks, or introducing methods to encourage exploration so the model can move into a region where learning resumes.

Local optima would mean getting stuck at a peak or trough in the landscape, not necessarily a broad flat region of little change. Vanishing gradients involve gradients becoming extremely small as they backpropagate, which slows learning but isn’t specifically described as a long flat stretch of no change. Exploding gradients cause instability from gradients becoming too large, which is the opposite problem of stagnation.

More practice questions