Attention heat map

Attention heat map preview
remixableManimCE2Dtransformersloop

manim/attention_heat_map.py

"""Attention heat map: token-by-token weights for one short sentence.

Render (the driver does this for every palette; these are for eyeballing):
    P=/opt/homebrew/opt/python@3.11/bin/python3.11
    ASSET_MODE=still  $P -m manim -s -r 1600,1600 manim/attention_heat_map.py AttentionHeatMap
    ASSET_MODE=motion $P -m manim -qh   -r 1080,1080 manim/attention_heat_map.py AttentionHeatMapLoop
"""

import numpy as np
from manim import (
    DOWN,
    LEFT,
    PI,
    RIGHT,
    UP,
    Line,

The full Manim source and the rendered video come with a plan.