What does “_” mean in lambda function and why is i

Question

This question already has an answer here:

• What is the purpose of the single underscore “_” variable in Python? 4 answers

I have an anonymous function with "_" as parameters, I don't know what it means and why it is used here.

and function is:

f = lambda _: model.loss(X, y)[0]

grad_num = eval_numerical_gradient(f, model.params[name], verbose=False, h=1e-5)

model.loss:

def loss(self, X, y=None):


    # Unpack variables from the params dictionary
    W1, b1 = self.params['W1'], self.params['b1']
    W2, b2 = self.params['W2'], self.params['b2']

    h1, h1_cache = affine_relu_forward(X, W1, b1)
    scores, h2_cache = affine_forward(h1, W2, b2)


    # If y is None then we are in test mode so just return scores
    if y is None:
        return scores

    loss, grads = 0, {}


    loss, dscores = softmax_loss(scores, y)
    loss = loss + 0.5*self.reg*(np.sum(W2**2) + np.sum(W1**2))
    dh1, grads['W2'], grads['b2'] = affine_backward(dscores,h2_cache)
    dX, grads['W1'], grads['b1'] = affine_relu_backward(dh1,h1_cache)
    grads['W1'] += self.reg*W1
    grads['W2'] += self.reg*W2

    return loss, grads

and the function eval_numerical_gradient:

def eval_numerical_gradient(f, x, verbose=True, h=0.00001):

    fx = f(x) # evaluate function value at original point
    grad = np.zeros_like(x)
    # iterate over all indexes in x
    it = np.nditer(x, flags=['multi_index'], op_flags=['readwrite'])
    while not it.finished:

        # evaluate function at x+h
        ix = it.multi_index
        oldval = x[ix]
        x[ix] = oldval + h # increment by h
        fxph = f(x) # evalute f(x + h)
        x[ix] = oldval - h
        fxmh = f(x) # evaluate f(x - h)
        x[ix] = oldval # restore

        # compute the partial derivative with centered formula
        grad[ix] = (fxph - fxmh) / (2 * h) # the slope
        if verbose:
            print(ix, grad[ix])
        it.iternext() # step to next dimension

    return grad

Loss function isn't complex, I want to know what the "_" represented and function in there.

Answer 1

It's a convention in Python to use _ for variables that are not going to be used later. There is no black magic involved and it is an ordinary variable name that behaves exactly as you'd expect.

In this case it is used because f is passed as a callback which will be passed an argument when it is called (fxph = f(x)).

If f would have been implemented as

f = lambda: model.loss(X, y)[0]

then a TypeError: <lambda>() takes 0 positional arguments but 1 was given error will be raised.

Answer 2

In your case, it's a convention, telling that the lambda parameter is not used (the answer from DeepSpace explain why).

General use:

You can use _ when you have to get a value but you do not use it. It is a python convention, developers use it to make their code more readable to other developers. With _, you say that you are aware that the variable is not used. Some IDE like PyCharm warn you if you don't:

def test(s):
    print("foobar")


if __name__ == '__main__':
    test("barfoo")

Will result of a warning in Pycharm for example:

But not with _:

def test(_):
    print("foobar")


if __name__ == '__main__':
    test("barfoo")

Result no warning: