AMD fully enters AI with its first LLM

AMD has introduced its first series of large language models (LLM) with 1 billion parameters and open source, called AMD OLMo, aimed at various applications and pre-trained on the company’s Instinct MI250 GPUs. AMD’s open-source LLMs aim to improve the company’s position in the AI industry and enable its customers (and everyone) to deploy these open-source models with AMD hardware. By making the data, weights, training recipes, and code public, AMD intends to enable developers not only […]

AMD has introduced its first series of large language models (LLM) with 1 billion parameters and open source, called AMD OLMo, aimed at various applications and pre-trained on the company’s Instinct MI250 GPUs.

Subscribe to the Softonic newsletter and get the latest in tech, gaming, entertainment and deals right in your inbox.

Subscribe (it's FREE) ►

AMD’s open-source LLMs aim to improve the company’s position in the AI industry and enable its customers (and everyone) to deploy these open-source models with AMD hardware.

By making the data, weights, training recipes, and code public, AMD aims to enable developers not only to reproduce the models but also to build upon them to continue innovating.

Beyond use in data centers, AMD has enabled the local deployment of OLMo models on AMD Ryzen AI PCs equipped with neural processing units (NPUs), allowing developers to leverage AI models on personal devices.

Everything we know about AMD’s LLM

The AMD OLMo models were trained on a broad dataset of 1.3 trillion tokens across 16 nodes, each with four AMD Instinct MI250 GPUs (64 processors in total). The AMD OLMo model line was trained in three steps.

In AMD’s own tests, AMD’s OLMo models showed impressive performance against similarly sized open-source models, such as TinyLlama-1.1B, MobiLlama-1B, and OpenELM-1_1B in standard benchmark tests for general reasoning capabilities and multitasking comprehension.

The two-phase SFT model experienced significant improvements in accuracy, with a 5.09% increase in MMLU scores and a 15.32% increase in GSM8k, demonstrating the impact of AMD’s training approach.

The final model AMD OLMo 1B SFT DPO outperformed other open-source chat models by at least 2.60% on average in benchmark tests.

ChatGPT DOWNLOAD

Additionally, AMD tested responsible AI tests, such as ToxiGen (which measures toxic language, where a lower score is better), crows_pairs (which evaluates bias), and TruthfulQA-mc2 (which assesses truthfulness in responses). And it was found that AMD’s OLMo models were on par with similar models in handling ethical and responsible AI tasks.

Author: Chema Carvajal Sarabia

{ "de-DE": "Journalist, spezialisiert auf Technologie, Unterhaltung und Videospiele. Über das zu schreiben, was mich begeistert (Gadgets, Spiele und Filme), ermöglicht es mir, bei Verstand zu bleiben und mit einem Lächeln im Gesicht aufzuwachen, wenn der Wecker klingelt. PS: Das stimmt nicht 100% der Zeit.", "en-US": "Journalist specialized in technology, entertainment and video games. Writing about what I'm passionate about (gadgets, games and movies) allows me to stay sane and wake up with a smile on my face when the alarm clock goes off. PS: this is not true 100% of the time.", "es-ES": "Content Manager - Periodista especializado en tecnología, entretenimiento y videojuegos. Escribir sobre lo que me apasiona (cacharros, juegos y cine) me permite seguir cuerdo y despertarme con una sonrisa cuando suena el despertador. PD: esto no es cierto el 100 % de las veces.", "fr-FR": "Journaliste spécialisé dans la technologie, le divertissement et les jeux vidéo. Écrire sur ce qui me passionne (gadgets, jeux et films) me permet de rester sain d'esprit et de me réveiller avec le sourire aux lèvres quand le réveil sonne. PS : cela n'est pas vrai 100 % du temps.", "it-IT": "Giornalista specializzato in tecnologia, intrattenimento e videogiochi. Scrivere di ciò che mi appassiona (gadget, giochi e film) mi permette di mantenere la sanità mentale e di svegliarmi con un sorriso sul viso quando suona la sveglia. PS: questo non è vero al 100% del tempo.", "ja-JP": "", "nl-NL": "", "pl-PL": "", "pt-BR": "Jornalista especializado em tecnologia, entretenimento e videogames. Escrever sobre o que me apaixona (gadgets, jogos e filmes) me permite manter a sanidade e acordar com um sorriso no rosto quando o despertador toca. PS: isso não é verdade 100% do tempo.", "social": { "email": "chemacs91@gmail.com", "facebook": "", "twitter": "https://twitter.com/chematopetazo", "linkedin": "" } }