C5_W4_A1_Transformer_Subclass

Document Content and Description Below

C5_W4_A1_Transformer_Subclass_v1 May 12, 2021 1 Transformer Network Welcome to Week 4’s assignment, the last assignment of Course 5 of the Deep Learning Specialization! And congratulations on mak... ing it to the last assignment of the entire Deep Learning Specialization - you’re almost done! Ealier in the course, you’ve implemented sequential neural networks such as RNNs, GRUs, and LSTMs. In this notebook you’ll explore the Transformer architecture, a neural network that takes advantage of parallel processing and allows you to substantially speed up the training process. After this assignment you’ll be able to: • Create positional encodings to capture sequential relationships in data • Calculate scaled dot-product self-attention with word embeddings • Implement masked multi-head attention • Build and train a Transformer model For the last time, let’s get started! 1.1 Table of Contents • Section ?? • Section ?? – Section ?? ∗ Section ?? – Section ?? ∗ Section ?? • Section ?? – Section ?? – Section ?? • Section ?? – Section ?? • Section ?? – Section ?? ∗ Section ?? – Section ?? ∗ Section ?? • Section ?? – Section ?? [Show More]

Last updated: 1 year ago

Preview 1 out of 31 pages