How error derivative becomes zero in gradient descent

Previous questions this & this does not answer my question import matplotlib.pyplot as plt inputs = [(0.0000, 0.0000), (0.1600, 0.1556), (0.2400, 0.3543), (0.2800, 0.3709)] targets = [230, 555, 815, 860] weights = [0.1, 0.2] b = 0.3 learning_rate = 0.1 epochs = 4 # prediction def predict(inputs): return sum([(w * i) for w, i in zip(weights, inputs)]) + b # train the network for epoch in range(epochs): # Feed forward--------- pred = [predict(inp) for inp in inputs] print("Pred:", pred) # Back propagation------ # error derivative errors_d = [(p - t) for p, t in zip(pred, targets)] # error partial derivative weight_d = [[(e * i) for i in (inp)] for e, inp in zip(errors_d, inputs)] bias_d = [(e * 1) for e in errors_d] weight_d_T = list(zip(*weight_d)) # Update weights and bias for j in range(len(weights)): weights[j] -= learning_rate * (sum(weight_d_T[j]) / len(weight_d)) b = b - (learning_rate * (sum(bias_d) / len(bias_d))) From theory, In order to minimize error, we need to take the derivative with respect to the weights and bias. In the above code, I did partial derivation of the error function. And use it with learning rate and update weights and bias. After doing some tests I figured out that if I write the weight updating equation as weights[j] -= learning_rate * (sum(weight_d_T[j]) / len(weight_d)) it will move towards down of the slope. If I write the weight updating equation as weights[j] += learning_rate * (sum(weight_d_T[j]) / len(weight_d)) I mean add partial derivat…