[{"@context":"https:\/\/schema.org\/","@type":"BlogPosting","@id":"https:\/\/blog.terabox.com\/insights\/mamba-modelo-secuencia-lineal-espacios-estados-selectivos#BlogPosting","mainEntityOfPage":"https:\/\/blog.terabox.com\/insights\/mamba-modelo-secuencia-lineal-espacios-estados-selectivos","headline":"Mamba: El modelo de tiempo lineal que desaf\u00eda a Transformers","name":"Mamba: El modelo de tiempo lineal que desaf\u00eda a Transformers","description":"\ud83d\udcfa V\u00eddeo de estudio recomendado hoy: https:\/\/www.youtube.com\/watch?v=9dSkvxS2EB0 Mamba: El fin del reinado de los Transformers y el ascenso del tiempo linealEl dilema entre la atenci\u00f3n y la recurrenciaLa magia de la selectividad en los SSMHardware-Aware: Maximizando la potencia de la... ","datePublished":"2026-08-18","dateModified":"2026-08-18","author":{"@type":"Person","@id":"https:\/\/blog.terabox.com\/author\/flextech-admin\/#Person","name":"flextech-admin","url":"https:\/\/blog.terabox.com\/author\/flextech-admin\/","image":{"@type":"ImageObject","@id":"https:\/\/secure.gravatar.com\/avatar\/ad516503a11cd5ca435acc9bb6523536?s=150&d=mm&r=gforcedefault=1","url":"https:\/\/secure.gravatar.com\/avatar\/ad516503a11cd5ca435acc9bb6523536?s=150&d=mm&r=gforcedefault=1","height":96,"width":96}},"publisher":{"@type":"Organization","name":"terabox","logo":{"@type":"ImageObject","@id":"http:\/\/blog.terabox.com\/wp-content\/uploads\/2021\/11\/logo\u4ea7\u54c1\u540d-\u7ad6\u7248.png","url":"http:\/\/blog.terabox.com\/wp-content\/uploads\/2021\/11\/logo\u4ea7\u54c1\u540d-\u7ad6\u7248.png","width":900,"height":900}},"image":{"@type":"ImageObject","@id":"https:\/\/img.youtube.com\/vi\/9dSkvxS2EB0\/maxresdefault.jpg","url":"https:\/\/img.youtube.com\/vi\/9dSkvxS2EB0\/maxresdefault.jpg","height":"","width":""},"url":"https:\/\/blog.terabox.com\/insights\/mamba-modelo-secuencia-lineal-espacios-estados-selectivos","video":{"@context":"http:\/\/schema.org\/","@type":"VideoObject","@id":"https:\/\/www.youtube.com\/watch?v=9dSkvxS2EB0#VideoObject","contentUrl":"https:\/\/www.youtube.com\/watch?v=9dSkvxS2EB0","name":"Mamba: Linear-Time Sequence Modeling with Selective State Spaces (Paper Explained)","description":"#mamba #s4 #ssm \n\nOUTLINE:\n0:00 - Introduction\n0:45 - Transformers vs RNNs vs S4\n6:10 - What are state space models?\n12:30 - Selective State Space Models\n17:55 - The Mamba architecture\n22:20 - The SSM layer and forward propagation\n31:15 - Utilizing GPU memory hierarchy\n34:05 - Efficient computation via prefix sums \/ parallel scans\n36:01 - Experimental results and comments\n38:00 - A brief look at the code\n\n\nPaper: https:\/\/arxiv.org\/abs\/2312.00752\n\nAbstract:\nFoundation models, now powering most of the exciting applications in deep learning, are almost universally based on the Transformer architecture and its core attention module. Many subquadratic-time architectures such as linear attention, gated convolution and recurrent models, and structured state space models (SSMs) have been developed to address Transformers' computational inefficiency on long sequences, but they have not performed as well as attention on important modalities such as language. We identify that a key weakness of such models is their inability to perform content-based reasoning, and make several improvements. First, simply letting the SSM parameters be functions of the input addresses their weakness with discrete modalities, allowing the model to selectively propagate or forget information along the sequence length dimension depending on the current token. Second, even though this change prevents the use of efficient convolutions, we design a hardware-aware parallel algorithm in recurrent mode. We integrate these selective SSMs into a simplified end-to-end neural network architecture without attention or even MLP blocks (Mamba). Mamba enjoys fast inference (5\u00d7 higher throughput than Transformers) and linear scaling in sequence length, and its performance improves on real data up to million-length sequences. As a general sequence model backbone, Mamba achieves state-of-the-art performance across several modalities such as language, audio, and genomics. On language modeling, our Mamba-3B model outperforms Transformers of the same size and matches Transformers twice its size, both in pretraining and downstream evaluation.\n\nAuthors: Albert Gu, Tri Dao\n\nLinks:\nHomepage: https:\/\/ykilcher.com\nMerch: https:\/\/ykilcher.com\/merch\nYouTube: https:\/\/www.youtube.com\/c\/yannickilcher\nTwitter: https:\/\/twitter.com\/ykilcher\nDiscord: https:\/\/ykilcher.com\/discord\nLinkedIn: https:\/\/www.linkedin.com\/in\/ykilcher\n\nIf you want to support me, the best thing to do is to share out the content :)\n\nIf you want to support me financially (completely optional and voluntary, but a lot of people have asked for this):\nSubscribeStar: https:\/\/www.subscribestar.com\/yannickilcher\nPatreon: https:\/\/www.patreon.com\/yannickilcher\nBitcoin (BTC): bc1q49lsw3q325tr58ygf8sudx2dqfguclvngvy2cq\nEthereum (ETH): 0x7ad3513E3B8f66799f507Aa7874b1B0eBC7F85e2\nLitecoin (LTC): LQW2TRyKYetVC8WjFkhpPhtpbDM4Vw7r9m\nMonero (XMR): 4ACL8AGrEo5hAir8A9CeVrW8pEauWvnp1WnSDZxW7tziCDLhZAGsgzhRQABDnFy8yuM9fWJDviJPHKRjV4FWt19CJZN9D4n","thumbnailUrl":["https:\/\/i.ytimg.com\/vi\/9dSkvxS2EB0\/default.jpg","https:\/\/i.ytimg.com\/vi\/9dSkvxS2EB0\/mqdefault.jpg","https:\/\/i.ytimg.com\/vi\/9dSkvxS2EB0\/hqdefault.jpg","https:\/\/i.ytimg.com\/vi\/9dSkvxS2EB0\/sddefault.jpg","https:\/\/i.ytimg.com\/vi\/9dSkvxS2EB0\/maxresdefault.jpg"],"uploadDate":"2023-12-24T15:47:57+00:00","duration":"PT40M40S","embedUrl":"https:\/\/www.youtube.com\/embed\/9dSkvxS2EB0","publisher":{"@type":"Organization","@id":"https:\/\/www.youtube.com\/channel\/UCZHmQk67mSJgfCCTn7xBfew#Organization","url":"https:\/\/www.youtube.com\/channel\/UCZHmQk67mSJgfCCTn7xBfew","name":"Yannic Kilcher","description":"I make videos about machine learning research papers, programming, and issues of the AI community, and the broader impact of AI in society.\n\nTwitter: https:\/\/twitter.com\/ykilcher\nDiscord: https:\/\/ykilcher.com\/discord\nBitChute: https:\/\/www.bitchute.com\/channel\/yannic-kilcher\nLinkedIn: https:\/\/www.linkedin.com\/in\/ykilcher\nBiliBili: https:\/\/space.bilibili.com\/2017636191\n\nIf you want to support me, the best thing to do is to share out the content :)\n\nIf you want to support me financially (completely optional and voluntary, but a lot of people have asked for this):\nSubscribeStar: https:\/\/www.subscribestar.com\/yannickilcher\nPatreon: https:\/\/www.patreon.com\/yannickilcher\nBitcoin (BTC): bc1q49lsw3q325tr58ygf8sudx2dqfguclvngvy2cq\nEthereum (ETH): 0x7ad3513E3B8f66799f507Aa7874b1B0eBC7F85e2\nLitecoin (LTC): LQW2TRyKYetVC8WjFkhpPhtpbDM4Vw7r9m\nMonero (XMR): 4ACL8AGrEo5hAir8A9CeVrW8pEauWvnp1WnSDZxW7tziCDLhZAGsgzhRQABDnFy8yuM9fWJDviJPHKRjV4FWt19CJZN9D4n\n\n\n","logo":{"url":"https:\/\/yt3.ggpht.com\/ytc\/AIdro_nqmmpWC-iPIeVF4grbJGcGmoWyYX0E6_PFGITlKv7jTMrh=s800-c-k-c0x00ffffff-no-rj","width":800,"height":800,"@type":"ImageObject","@id":"https:\/\/www.youtube.com\/watch?v=9dSkvxS2EB0#VideoObject_publisher_logo_ImageObject"}},"potentialAction":{"@type":"SeekToAction","@id":"https:\/\/www.youtube.com\/watch?v=9dSkvxS2EB0#VideoObject_potentialAction","target":"https:\/\/www.youtube.com\/watch?v=9dSkvxS2EB0&t={seek_to_second_number}","startOffset-input":"required name=seek_to_second_number"},"interactionStatistic":[[{"@type":"InteractionCounter","@id":"https:\/\/www.youtube.com\/watch?v=9dSkvxS2EB0#VideoObject_interactionStatistic_WatchAction","interactionType":{"@type":"WatchAction"},"userInteractionCount":175729}],{"@type":"InteractionCounter","@id":"https:\/\/www.youtube.com\/watch?v=9dSkvxS2EB0#VideoObject_interactionStatistic_LikeAction","interactionType":{"@type":"LikeAction"},"userInteractionCount":3676}]},"about":["Insights","\u300eSpanish\u300f"],"wordCount":1432},{"@context":"https:\/\/schema.org\/","@type":"BreadcrumbList","itemListElement":[{"@type":"ListItem","position":1,"name":"Insights","item":"https:\/\/blog.terabox.com\/insights\/#breadcrumbitem"},{"@type":"ListItem","position":2,"name":"Mamba: El modelo de tiempo lineal que desaf\u00eda a Transformers","item":"https:\/\/blog.terabox.com\/insights\/mamba-modelo-secuencia-lineal-espacios-estados-selectivos#breadcrumbitem"}]}]