This series of tutorial is based onBlog post: https://sustcsonglin.github.io/blog/2024/deltanet-1/Blog post: https://sustcsonglin.github.io/blog/2024/deltanet-2/Paper: https://arxiv.org/a

Generate a realistic drawing of a cube To draw a cube, we ask Modeling: How do we describe the cube Rendering: How do we visualize the model Modeling Suppose we construct a cube that centered at th

1 From an Example Let’s observe the following example 123456789101112131415161718>>> import torch>>> a = torch.tensor([1, 2, 3, 4], requires_grad=True, dtype=torch.float32)>>&g

位置编码–广泛使用的RoPE 1 RoPE位置编码 RoPE的理论及由来：https://kexue.fm/archives/8265 假设位置为m的query向量q的维度为ddd，则RoPE会将q变换为 [q0q1q2q3⋮qd−2qd−1]⊗[cos⁡mθ0cos⁡mθ0cos⁡mθ1cos⁡mθ1⋮cos⁡mθd/2−1cos⁡mθd/2−1]+[−q1q0−q3q2⋮−qd−1qd−2]⊗

1 Logic Operations There are 4 operators, they are ¬\neg¬, ∧\land∧, ∨\lor∨, ⇒\Rightarrow⇒, the outcome of the operation is the Truth Table. For proposition ¬A\neg A¬A A 0 1 ¬\neg¬A 1 0 For c

This is a lecture note of the course CSE 234 - Data Systems for ML - LE [A00].From UC SanDiegoProf. Zhang HaoWinter, 2025Link: https://podcast.ucsd.edu/watch/wi25/cse234_a00/1

1 Preliminaries Mixture-of-Experts is an essential architecture choice when building LLMs. Since the prevalence of DeepSeekV3, companies will consider whether to use MoE structure before LLM pretraini

This is a lecture note of the course CSE 234 - Data Systems for ML - LE [A00].From UC SanDiegoProf. Zhang HaoWinter, 2025Link: https://podcast.ucsd.edu/watch/wi25/cse234_a00/1

This is a lecture note of the course CSE 234 - Data Systems for ML - LE [A00].From UC SanDiegoProf. Zhang HaoWinter, 2025Link: https://podcast.ucsd.edu/watch/wi25/cse234_a00/1

This is a lecture note of the course CSE 234 - Data Systems for ML - LE [A00].From UC SanDiegoProf. Zhang HaoWinter, 2025Link: https://podcast.ucsd.edu/watch/wi25/cse234_a00/1