01 Introduction to Natural Language Processing & Machine Learning Basics

Applications of natural language processing, examples; Basics of machine learning and deep learning

02 Static Word Representation & Text Classification

Sparse representation: one-hot, ppmi, SVD.; Dense representation: word2vec; Generative classifier: generative classifier; Naive Bayes: Naive Bayes; Discriminative classifier: discriminative classifier; Sparse features BoW/N-gram; Dense features

Exercises: Word2vec, text classification

03 Language Model Language Modeling (I)

n-gram language model; RNN recurrent neural network and language model; RNN, RNN-LM - LSTM / GRU -

Exercises: RNN language model

04 Language Model Language Modeling & Attention Mechanism (II)

Attention mechanism Attention; RNN LM with Attention

Exercises: Attention mechanism

05 Self-attention mechanism Self-attention & Transformer

Transformer Encoder; Transformer Decoder; Sequence-to-sequence Seq2Seq, concepts of encoder-decoder

Exercises: huggingface tutorial

06 Pretrained Language Model Pretrained LM

Encoder LM (BERT); Decoder-only LM (GPT); Seq2seq LM (T5) -

Exercises: huggingface tutorial

07 Large Language Model (Large Language Model) -

Prompting and prompt learning; Contextual learning In-context Learning; Evolutionary trajectory of GPT series; Basic introduction (GPT-3, GPT3.5, ChatGPT, GPT4...); Instruction learning (Super-Inst, Self-Inst, Flan …); Retrieval enhancement; Introduction to applications of large language models

Exercises: Using large language models

08 Advances

Efficient language models, linear attention mechanism; Integration of knowledge in natural language processing; Multimodal; Introduction to research in natural language processing