Looped Transformers: 2x Deeper Without More Weights
Share

Post Content

 

 [[{“value”:”Inkling 2 Mini is a looped transformer. Looped transformers explained: how a language model can run part of itself more than once for every token, getting more steps of reasoning without storing more weights. Using Thinking Machines’ new open model Inkling 2 Mini as the example (24 layers, with the middle 12 running three times for 48 layer steps), we look at what the extra passes improve, the one thing they don’t improve, what they cost, and why moving reasoning inside the loop worried people.

Inkling 2 Mini release post: https://thinkingmachines.ai/news/inkling-2-mini/
Inkling 2 Mini model card: https://huggingface.co/thinkingmachines/Inkling-2-Mini
Research model with 1 to 5 passes (charts by pass count): https://thinkingmachines.ai/model-card/tml-d24-varloop-research/
Goodfire, Interpreting Inkling 2 Mini: https://www.goodfire.com/research/interpreting-inkling-2-mini/
Universal Transformers (2018 paper): https://arxiv.org/abs/1807.03819
Run it with Ollama: https://ollama.com/library/inkling-2-mini
NVIDIA Dory: https://huggingface.co/nvidia/NVIDIA-Nemotron-Labs-Dory1.0-4B-BF16
ByteDance Ouro: https://huggingface.co/ByteDance/Ouro-2.6B

What happens with a single pass and no thinking: https://youtu.be/S4efgdWLvIc
Speculative Decoding Explained: https://youtu.be/eFgknPFK-g0
Next-Concept Models (NCM): https://youtu.be/_87vrjxx1EE

All benchmark numbers in this video are Thinking Machines’ own, measured on their research version of the model.

#AI #LLM #LoopedTransformers

My voice to text App: whryte.com
Website: https://engineerprompt.ai/
RAG Beyond Basics Course:
https://prompt-s-site.thinkific.com/courses/rag
Signup for Newsletter, localgpt:
https://tally.so/r/3y9bb0
🦾 Discord: https://discord.com/invite/t4eYQRUcXB
☕ Buy me a Coffee: https://ko-fi.com/promptengineering
🔴 Patreon: https://www.patreon.com/PromptEngineering
💼Consulting: https://calendly.com/engineerprompt/consulting-call
📧 Business Contact: engineerprompt@gmail.com
Become Member: http://tinyurl.com/y5h28s6h
💻 Pre-configured localGPT VM: https://bit.ly/localGPT (use Code: PromptEngineering for 50% off).

Chapters
0:00 What Inkling 2 Mini is
0:33 What is a looped transformer?
1:08 How a transformer takes a fixed number of steps
2:29 Thinking tokens (chain of thought)
3:20 Two obvious fixes: more layers or more thinking
4:16 The looped transformer idea
4:50 Universal Transformers, Ouro and Dory
5:13 Inside Inkling 2 Mini: 4 blocks, middle loops 3 times
6:09 Why loop the middle layers
6:56 Mixture of experts: how it fits on a laptop
8:13 The pass dial: 1 to 5 passes
9:02 Shorter thinking with more passes
9:33 What looping does not improve
10:21 What looping costs: memory vs compute
11:36 Why this worried people: chain-of-thought monitoring
12:59 Run it with Ollama on a Mac or DGX Spark
13:31 Limitations
14:16 When to use a looped model”}]] Read More Prompt Engineering 

#Promptengineering #AI

By ali

Leave a Reply