1. Gradient Descent

    Wed 10 April 2013

    Gradient Descent is perhaps the most intuitive of all optimization algorithms. Imagine you're standing on the side of a mountain and want to reach the bottom. You'd probably do something like this,

    1. Look around you and see which way points the most downwards
    2. Take a step in that direction, then …