Keras image binary classification which class is assigned probability 0 and 1 Using Functional API

Question

When implementing binary classification, the sigmoid function is used as activation for the last layer by majority of the people. As I understand the sigmoid function gives a probability between 0 and 1 and we set a threshold value (mostly 0.5) to determine the class label.

However I'm confused as to which class does is that probability for ? Say I have two classes A and B and I get prediction result p or p% . Is that the probability of being class A or the probability of not being class A (i.e B).

I apologize if this has been asked here before but I couldn't find it. Since I'm using Keras Functional API i cannot use the predict_classes() function. Also I mostly use generators for loading my dataset mostly flow_from_dataframe() from ImageDatagenerator in which you can just provide the class labels or class list. In my case it's two strings in my dataframe "REAL" or "FAKE" .

Also is there a way to set which class I want the probability for ?

Answer 1

Not exactly.  If your sigmoid output was 0.8, that doesn't mean your network produced probabilities for each class as sigmoid outputs don't add up to 1. In other words, an output of 0.8 does not imply that it is likely to belong to other classes with a probability of 0.2.

Also, with sigmoid network outputs p(y=1) in Binary Classification. Then p(y=0) = 1 - p(y=1) by definition of probability. Only for basic binary categorization, they add up to one.

Softmax activation should be used to see the probability of each class because its output will total to 1. Softmax outputs can be interpreted as probabilities.

These models, on the other hand, are deterministic rather than probabilistic. As a result, it is customary to interpret softmax results as probabilities, but there is no mathematical connection between them.

Is there a way to specify which class the probability applies to?

You can choose from a variety of thresholds, the most common of which is 0.5, but this is dependent on your data and situation. You can adjust the threshold to observe how it affects the AUC-ROC, and then interpret the results to find the ideal threshold for you.

If you wish to decide the classes, you can use the following formula, keeping in mind that the threshold is 0.5 and that you can adjust it:

predicted_classes = [1 * (x[0]>=0.5) for x in preds_sigmoid]

If the output is greater than 0.5, it is classified as second class.