2018年是NLP丰收的一年,这一年中比较大火的又是BERT,而BERT又和2017年大火的Transformer相关,本文就网上阐释比较好的几篇关于这两个模型的文章进行简单收集,方便想了解这两个模型的研究者阅读。
当然最权威的是官方论文和源码:
网上比较好的解读:
BERT运用实例:
- 奇点智能BERT实战教程
- AI赋能法律 | NLP最强之谷歌BERT模型在智能司法领域的实践浅谈
- bert-Chinese-classification-task
bert中文分类实践 - bert-chinese-ner: https://github.com/ProHiryu/bert-chinese-ner
使用预训练语言模型BERT做中文NER - BERT-BiLSTM-CRF-NER
Tensorflow solution of NER task Using BiLSTM-CRF model with Google BERT Fine-tuning - bert-sequence-tagging: https://github.com/zhpmatrix/bert-sequence-tagging
基于BERT的中文序列标注 - Google BERT 中文应用之春节对对联
- Google BERT 中文应用之《红楼梦》对话人物提取
- bert-as-service: https://github.com/hanxiao/bert-as-service
将不同长度的句子用BERT预训练模型编码,映射到一个固定长度的向量上:Mapping a variable-length sentence to a fixed-length vector using pretrained BERT model - bert_language_understanding: https://github.com/brightmart/bert_language_understanding
BERT实战:Pre-training of Deep Bidirectional Transformers for Language Understanding: pre-train TextCNN - sentiment_analysis_fine_grain: https://github.com/brightmart/sentiment_analysis_fine_grain
BERT实战,多标签文本分类,在 AI Challenger 2018 细粒度情感分析任务上的尝试:Multi-label Classification with BERT; Fine Grained Sentiment Analysis from AI challenger - BERT-NER: https://github.com/kyzhouhzau/BERT-NER
BERT实战,命名实体识别: Use google BERT to do CoNLL-2003 NER ! - BERT、GPT-2这些顶尖工具到底该怎么用到我的模型里
第三方BERT实现代码:
- pytorch-pretrained-BERT: https://github.com/huggingface/pytorch-pretrained-BERT
Google官方推荐的PyTorch BERB版本实现,可加载Google预训练的模型:PyTorch version of Google AI's BERT model with script to load Google's pre-trained models - BERT-pytorch: https://github.com/codertimo/BERT-pytorch
另一个Pytorch版本实现:Google AI 2018 BERT pytorch implementation - BERT-tensorflow: https://github.com/guotong1988/BERT-tensorflow
Tensorflow版本:BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding - bert-chainer: https://github.com/soskek/bert-chainer
Chanier版本: Chainer implementation of "BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding" - BERT-keras: https://github.com/Separius/BERT-keras
Keras版: Keras implementation of BERT with pre-trained weights - tbert: https://github.com/innodatalabs/tbert
PyTorch port of BERT ML model
本文资源收集中参考了网上很多博客、知乎帖子以及AINLP微信公众号,在此对这些文章的作者一并感谢,后面会持续更新看到较好资源。