NLP 엔지니어의 취업을 위해 끊임없는 학습을 위한 논문리뷰를 진행하고자 합니다.
가장 기본이 되는 논문부터 천천히 스스로의 언어로 재정의하며 논문 리뷰를 할 예정이니 추후 올라오는 포스팅에 대한 따끔한 지적과 충고 및 조언은 환영입니다 😊
논문등재 연도에 따라 아래와 같이 리스트를 구성하였으며 포스팅 시기는 공부가 완전히 끝나는대로 올라올 예정입니다.
즉, 정해진 날짜에 올리는 것이 아니기에 자주 올라올수도, 아닐수도 있음을 공지합니다.
※ 임베딩은 파란색, 모델은 빨간색, 알고리즘/기법은 보라색, Survey는 초록색으로 표기하였습니다.
필수내용은 *로 표시하였으니 참고하세요.
- RNN* : Recurrent neural network based language model (2010) (처음 등장은 1986)
- Word2Vec* : Efficient Estimation of Word Representations in Vector Space (2013)
- LSTM* : Long Short Term Memory Recurrent Neural Network Architectures for Large Scale Acoustic Modeling (2014)
- GloVe* : Global Vectors for Word Representation (2014)
- GRU : Learning Phrase Representation using RNN Encoder-Decoder for Stistical Machine Translation (2014)
- Seq2Seq* : Sequence to Sequence Learning with Neural Networks (2014)
- Attention* : Neural Machine Translation by Jointly Learning to Align and Translate (2015)
- FastText* : Enriching Word Vectors with Subword Information (2016)
- Transformer* : Attention is All You Need (2017) ✅
- ELMo* : Deep contextualized word representations (2018)
- GPT-1* : Improving Language Understanding by Generative Pre-Training (2018)
- BERT* : Pre-training of Deep Bidirectional Transformers for Language Understanding (2018)
- GPT-2 : Language Models are Unsupervised Multitask Learners (2018)
- RoBERTa : A Robustly Optimized BERT Pretraining Approach (2019)
- ALBERT: A Lite BERT for Self-supervised Learning of Language Representations (2019)
- Transformer-XL : Attentive Language Models Beyond a Fixed-Length Context (2019)
- XLNet : Generalized Autoregressive Pretraining for Language Understanding (2019)
- ELECTRA : Pre-training Text Encoders as Discriminators Rather Than Generators (2020)
- GPT-3* : Language Models are Few-Shot Learners (2020)
- RAG* : Retrieval-Augemented Generation for Knowledge Intensive NLP Tasks (2020)
- LoRA* : Low-Rank Adaptation of Large Language Models (2021) ✅
- QLoRA* : Efficient Finetuning of Quantized LLMs (2023)
- Survey on Large Language Models - Upstage (2023) ✅
- Self-Discover : Large Language Models Self-Compose Reasoning Structures (2024)
- More Agent Is All You Need (2024)
- Large Language Models : A Survey (2024)
유용한 링크도 첨부합니다 😆
Aman's AI Journal • Papers List
GPO is designed to perform group alignment by learning a few-shot preference model that augments the base LLM. Once learned, the preference module can be used to update the LLM via any standard preference optimization or reweighting algorithm (e.g., PPO, D
aman.ai
'개발 > 논문' 카테고리의 다른 글
| LoRA : Low-Rank Adaptation of Large Language Models (0) | 2024.02.18 |
|---|---|
| Upstage - Survey on Large Language Models (0) | 2023.12.22 |
| Transformer - Attention is All You Needs (0) | 2023.11.24 |
| 무식한 개발자의 논문리뷰 : (Basic) (1) | 2023.05.08 |
