Python Transformers By Huggingface Hands On

Python Transformers By Huggingface Hands On

101 practical implementation hands-on of ALBERT/ViT/BigBird and other latest models with huggingface transformers

Have you ever heard of the word "Transformer"?

It's not a movie title, but it's a latest technologies introduced by Google Researchers.

Even if you are not a data scientist or programmer, it is good to know this word.

Think of electricity. Even if you don't know how electricity works, we depend on it in every aspect of our lives.

It is no exaggeration to say that Transformer is gradually becoming as pervasive in our lives as electricity.

In this book, we will look at the Transformer as a black box like electricity, and focus on what it can do for us.

We will focus on the question, "What can we do with Transformer?" and let you experience the cutting edge of artificial intelligence
by coding in python.

Just as you would use an electric air conditioner, electric kettle, microwave oven, or refrigerator, you can use Transformer for translation, summarization,
speech recognition, image recognition and so on.

You may be surprised to learn that speech recognition, machine translation, and image recognition can be done in a few lines of code.

There is a mechanism behind it.

A company called huggingface is still small as of 2021/8, but is growing rapidly.

The company provides a library called transformers, and has been very successful in open sourcing transformers and building an ecosystem.

As of the end of 2020, transformers has been downloaded more than 5 million times, has more than 40,000 Github stars,
and is used by more than 5,000 companies.

It is also very popular among researchers at Google/Microsoft/Facebook, and Microsoft uses the library in bing.

Let's have fun solving 101 examples on huggingface transformers and catch up with the state-of-the-art implementation of artificial intelligence.


–-Table of Contents–-

* Latest Trend in Deep Learning

* Chapter 1 pipeline

* Chapter 2 Fine-tuning and Evaluation of DistilBERT using real data

* Chapter 3 Model Performance Evaluation

* Chapter4 Composition using GPT series

* Chapter 5 MLM(Masked Language Model)

* Chapter6 CLIP～Bridging Image Recognition and Natural Language Processing～

* Chapter7 Wave2Vec2 Automatic Speech Recognition

* Chapter 8 Multi-class classification in BERT

* Chapter9 Automatic Summarization by BART

* Chapter10 Ensemble learning with two BERTs

* Chapter11 BigBird

* Chapter12 PEGASUS

* Chapter 13 M2M100

* Chapter14 Mobile BERT

* Chapter15 GPT, DialoGPT, DistilGPT2

* Chapter16 Practical exercise Moderna v.s. Pfizer (compare with BERT and tSNE)