Generative modeling with sparse transformers

OpenAI Blog2019-04-23 15:000 views

We’ve developed the Sparse Transformer, a deep neural network which sets new records at predicting what comes next in a sequence—whether text, images, or sound. It uses an algorithmic improvement of the attention mechanism to extract patterns from sequences 30x longer than possible previously.

Source：https://openai.com/index/sparse-transformer

My favorites

PreviousMuseNet NextOpenAI Five defeats Dota 2 world champions

A massive new simulation reveals how the universe emerged from darkness

A groundbreaking new supercomputer simulation is revealing how the universe escaped its cosmic dark age and transformed

0Science Daily Tech

YubiKey PIV security depends on slot policy, not model alone

YubiKey PIV security depends on slot policy, not model alone A YubiKey PIV deployment can look consistent at the hardwar

0dev.to

Check for editable text in a reconstructed PowerPoint slide

I rebuilt a complex teaching diagram from a PNG into a PowerPoint slide. The first-slide XML had nine text-bearing shape

0dev.to

ISC Stormcast For Thursday, November 12th, 2026 https://isc.sans.edu/podcastdetail/10176, (Thu, Nov 12th)

0SANS ISC

Tác động chiến thuật từ bàn thắng ra mắt của Olaha Mạnh Đức tại tuyển Việt Nam

Chiến thắng 2-0 trước Đài Loan trên sân Thiên Trường mang lại những tín hiệu tích cực về mặt vận hành chiến thuật cho độ

0dev.to Data

Google Launches Gemini Decision 1, Its First Decision-Focused Model

Google on November 12 2026 introduced Gemini Decision 1 and the Gemini Decisions API, a new class of model that does not

0dev.to News