Distance Between Ideal and Reality: Loss Function

A loss function is a function that measures the difference between the predicted value and the actual value in a model. It takes the predicted and actual values (ground truth) as input and returns loss (error) as output.

The smaller the value of the loss function, the closer the model's predictions are to the actual values. Conversely, a larger value indicates that predictions are farther from the actual values. The goal of AI model training is to minimize the value of the loss function.

Consider a scenario where you are playing a game of archery; the loss function can be thought of as follows:

• Actual Value: The precise target point where the arrow should hit (bullseye)

• Predicted Value: The point where the arrow actually lands

• Calculating the Distance from Bullseye: Measure how far the arrow is from the center (loss function)

• Loss Function Input: The bullseye (actual value) and where the arrow landed (predicted value)

• Loss Function Output: The calculated distance of the arrow from the bullseye

Scoring how close the arrow is to the bullseye is the role of the loss function. The farther the arrow is from the center, the higher the score (loss), and the closer it is, the lower the score (loss).

The goal of a loss function is always to keep the score low, i.e., shoot the arrow as close to the center as possible.

MSE (Mean Squared Error)

This method computes the average of the squares of the differences between predicted and actual values. It's similar to squaring the distance between the arrow and bullseye and averaging those distances.

• Formula: (predicted value - actual value)^2

MAE (Mean Absolute Error)

This approach calculates the average of the absolute differences between predicted and actual values. It's akin to taking the absolute distance between the arrow and bullseye and averaging those distances.

• Formula: |predicted value - actual value|

Cross-Entropy Loss

Used in classification problems, this loss function measures the difference between the probability distribution predicted by the model and the actual labels. It's like measuring how accurately the arrow hits the bullseye.

• Binary Classification Formula: -(actual value * log(predicted value) + (1 - actual value) * log(1 - predicted value))