#Pre-training
Showing 27 of 27 repositories tagged #pre-training, ranked by stars
The official GitHub page for the survey paper "A Survey of Large Language Models".
Data processing for and with foundation models! ๐ ๐ ๐ฝ โก๏ธ โก๏ธ๐ธ ๐น ๐ท
Awesome resources for in-context learning and prompt engineering: Mastery of the LLMs such as ChatGPT, GPT-3, and FlanT5, with up-to-date and cutting-edge updates.
Papers about pretraining and self-supervised learning on Graph Neural Networks (GNN).
Unified Training of Universal Time Series Forecasting Transformers
Code for TKDE paper "Self-supervised learning on graphs: Contrastive, generative, or predictive"
An Open-sourced Knowledgable Large Language Model Framework.
Pre-training of Deep Bidirectional Transformers for Language Understanding: pre-train TextCNN
Large Language Model-enhanced Recommender System Papers
Official repository for "Craw4LLM: Efficient Web Crawling for LLM Pretraining"
[ICLR 2024] Sheared LLaMA: Accelerating Language Model Pre-training via Structured Pruning
Multi-modality pre-training
The official repo for [TPAMI'25] "HyperSIGMA: Hyperspectral Intelligence Comprehension Foundation Model"
One-click training of your own GPT. Training a GPT has never been easier for beginners. / ไธ้ฎ้ข่ฎญ็ป+SFTไธไธชๅฑไบ่ชๅทฑ็LLM๏ผ0ๅบ็ก่ฎญ็ปGPTๅๆฅๅฏไปฅ่ฟไน็ฎๅ๏ผ
Paper List of Pre-trained Foundation Recommender Models
[Survey] Masked Modeling for Self-supervised Representation Learning on Vision and Beyond (https://arxiv.org/abs/2401.00897)
A simple and working implementation of Electra, the fastest way to pretrain language models from scratch, in Pytorch
Foundation models based medical image analysis
A framework for state-of-the-art pre-trained bio foundation models on genomics and transcriptomics modalities.
A collection of Audio and Speech pre-trained models.
An implementation of masked language modeling for Pytorch, made as concise and simple as possible
Official implementation of pre-training via denoising for TorchMD-NET
Team Kakao&Brain's Grammatical Error Correction System for the ACL 2019 BEA Shared Task
A paper list of pre-trained language models (PLMs).
Implementation of Marge, Pre-training via Paraphrasing, in Pytorch
Salesforce AI Research's open diffusion language model
Implementation of COCO-LM, Correcting and Contrasting Text Sequences for Language Model Pretraining, in Pytorch