CS229 Machine Learning - 3 (Linear regression, Probabilistic interpretation, Logistic regression, Newton's method)

Outline:
- Linear regression
- Probabilistic interpretation
- Logistic regression
- Newton's method

CS229 Machine Learning - 2 (Linear regression, Gradient descent, Normal equation)

Outline:
- Linear regression
- Gradient descent
- Normal equation

Bayes' theorem

• Bayes' theorem was used to convert a prior probability into a posterior probability by incorporating the evidence provided by the observed data.
• P(W | D) = P(D | W)P(W)/P(D)     ==> posterior = likelihood x prior
• P(W | D): posterior probability
• P(D | W): likelihood function
• P(W): prior probability
• P(D): normalization constant, ensures that the posterior distribution on the left-hand side is a valid probability density and integrates to one.
• maximum likelihood: w is set to the value that maximizes the likelihood function p(D | W). This corresponds to choosing the value of W for which the probability of the observed data set is maximized.

Machine learning applications

Machine learning:

• Supervised learning
• finite number of discrete categories --> classification problem
• continuous variables --> regression problem
• Unsupervised learning
• discover groups of similar examples --> clustering problem
• determine the distribution of data within the input space --> density estimation problem
• project the data form a high-dimensional space down to two or three dimensions --> visualization purpose

Bias vs Variance in machine learning

High bias -> underfitting

High variance -> overfitting

Reference: https://www.coursera.org/learn/machine-learning/home/welcome

[ML] Precision / Recall

For classification problem

Precision:
    True positives                            True positives
------------------------   =   ---------------------------------------
#predicted positives         True positives + False positives

Recall:
    True positives                            True positives
------------------------   =   ---------------------------------------
  #actual positives            True positives + False negative


There is a tradeoff between precision and recall.
One way to compute the precision and recall is using F_1 score
                           precision * recall
F_1 score = 2 * -----------------------
                           precision + recall

Reference: https://www.coursera.org/learn/machine-learning/lecture/tKMWX/error-metrics-for-skewed-classes