Statistical Models

  1. Generative Models and Discrimative Models

Generative models try to maximum the joint probability of the token sequence A and label sequence B, while discrimitive models try to maximum the conditional probability of label sequence B conditional on token sequence A.
NB and HMM are generative models; CRF is a discrimitive model.

  1. Posterior probability
    Posterior probability is the conditional probability itself: P(B/A), in contrast to the prior probability P(B), posterior indicates that this probability is the likelihood of event B happens given that the occuring of event A is already known.

  2. Assumptions
    HMM: a. n-order markov assumption b. conditional independent assumption (apply chain rule in conditional paradigm)
    Maximum Likelihood Estimation.

Naive Bayes: a. naive bayes assumption b. conditional independent assumption (apply chain rule in conditional paradigm)
Maximum a posterior estimation.

Generative model can be identified when the model needs to calculate the marginal distribution of the predicted label P(Y)