- Generative Models and Discrimative Models

Generative models try to maximum the joint probability of the token sequence A and label sequence B, while discrimitive models try to maximum the conditional probability of label sequence B conditional on token sequence A.

NB and HMM are generative models; CRF is a discrimitive model.

- Posterior probability

Posterior probability is the conditional probability itself: P(B/A), in contrast to the prior probability P(B), posterior indicates that this probability is the likelihood of event B happens given that the occuring of event A is already known. -
Assumptions

HMM： a. n-order markov assumption b. conditional independent assumption (apply chain rule in conditional paradigm)

Maximum Likelihood Estimation.

Naive Bayes: a. naive bayes assumption b. conditional independent assumption (apply chain rule in conditional paradigm)

Maximum a posterior estimation.

Generative model can be identified when the model needs to calculate the marginal distribution of the predicted label P(Y)