Security through data

CONTENT

  • Home
  • Blog
  • Data
  • Directory
  • Events
  • Tutorials

FEATURED

  • CyberAlerts
  • CyberDecoded
  • CyberWeekly
  • CyberStory
  • CyberTips

COMPANY

  • About us
  • Advertise
  • Legal & Policy
Cybermaterial
  • CATEGORIES
    • Alerts
    • APIs
    • Apps
    • Blog
    • Cyber101
    • Documents
    • Entertainment
    • Learning
    • Quotes
    • Stats
    • Tools
No Result
View All Result
Contact Us
Newsletter
Cybermaterial
  • CATEGORIES
    • Alerts
    • APIs
    • Apps
    • Blog
    • Cyber101
    • Documents
    • Entertainment
    • Learning
    • Quotes
    • Stats
    • Tools
No Result
View All Result
Contact Us
Newsletter
Cybermaterial
No Result
View All Result

Sparse Transformers

in Software, Tools
2 min read

Generative Modeling with Sparse Transformers

We’ve developed the Sparse Transformer, a deep neural network which sets new records at predicting what comes next in a sequence—whether text, images, or sound. It uses an algorithmic improvement of the attention mechanism to extract patterns from sequences 30x longer than possible previously.

One existing challenge in AI research is modeling long-range, subtle interdependencies in complex data like images, videos, or sounds. The Sparse Transformer incorporates an O(N \sqrt{N})O(NN​) reformulation of the O(N^2)O(N2) Transformer self-attention mechanism, along with several other improvements, to apply it directly to these rich data types. Previously, models used on these data were specifically crafted for one domain or difficult to scale to sequences more than a few thousand elements long. In contrast, our model can model sequences with tens of thousands of elements using hundreds of layers, achieving state-of-the-art performance across multiple domains. At OpenAI, we’re using it to help us build AI systems that possess a greater ability to understand the world.

LEARN MORE

Tags: AIArtificial IntelligenceDeepfakeDeepfake-toolsSparse Transformers
11
VIEWS

Related Posts

Parrot OS
Software

Parrot OS

Engage Packet Builder
Software

Tool: WPScan

10 PenTesting Tools V.1
Software

Tool: Wfuzz

MORE

Memes

Wrong, wrong, right

IP Address: Internet Protocol address

Quotes

“What’s different today vs. 2016, from a hacking perspective, is higher use of automated credential stuffing — to hijack accounts.”

Definitions

Exploit

ADVERTISEMENT

Tags

Books Cyber Definition Cybersecurity Hackers Malware Memes Movies Quantum Computing Software Word of the day

© 2021 | CyberMaterial | All rights reserved.

SECURITY THROUGH DATA

No Result
View All Result
  • Home
  • Blog
  • Data
  • Directory
  • Events
  • Tutorials
  • CyberDecoded
  • Stats
  • CyberStory
  • CyberTips
  • Cyber Weekly

© 2020 CyberMaterial - Cyber Decoded.

This website uses cookies. By continuing to use this website you are giving consent to cookies being used. Visit our Privacy and Cookie Policy.