Handson-ml: Chapter 4 - Logistic Regression

[nitml]

can you please explain me how this code is making decision boundary in iris dataset by logistic Regression
cell 60 in jupyter notebook chapter 4

log_reg = LogisticRegression(C=10**10, random_state=42)
left_right = np.array([2.9, 7])
boundary = -(log_reg.coef_[0][0] * left_right + log_reg.intercept_[0]) / log_reg.coef_[0][1]

[ageron]

Hi @nitml ,

In logistic regression, the estimated probability for the positive class is:

p = σ(w1x1 + w2x2 + w3x3 + ... + wnxn + b)

where σ(.) is the sigmoid function, x1, x2, ..., xn are the input features (e.g., petal width, petal length, etc.), and w1, w2, ..., wn are the corresponding weights, and b is the bias term.

If we decide that the decision boundary is at p=50% (in other words we choose the class with the highest estimated probability), then the decision boundary is the set of input features x1, x2, ..., xn such that:

p = σ(w1x1 + w2x2 + w3x3 + ... + wnxn + b) = 50%

The sigmoid function is such that if σ(0)=0.5, so if p=0.5, then:

w1x1 + w2x2 + w3x3 + ... + wnxn + b = 0

In the iris dataset there are four input features, but we only used two in this example, so there are just two weights w1 and w2. So:

w1x1 + w2x2 + b = 0

Therefore:

w2x2 = -(w1x1 + b)

So:

x2 = -(w1x1 + b)/w2

If we want to plot the decision boundary (which is a straight line), we just need two points: one on the left of the figure, and one on the right. In this figure, the left is at x1=2.9, and the right is at x1=7. Next, we just need to compute x2 = -(w1x1 + b)/w2 for these two values of x1, which is what the code does.

Hope this helps!

[nitml]

Thanks so much for your time....The book is really great for beginner.