EPFL and ETH Zurich are developing a fully open-source, multilingual large language model (LLM) that will be released later this summer under the Apache 2.0 license, emphasizing transparency and broad accessibility.

The model, announced after the International Open-Source LLM Builders Summit in Geneva, will be available in two versions—8 billion and 70 billion parameters—with the larger one ranking among the most powerful fully open LLMs globally.

This initiative was highlighted at a major international summit in Geneva, hosted by EPFL and ETH Zurich, which brought together around 50 organizations to promote open-source LLMs and trustworthy AI as credible alternatives to proprietary systems.

Training was conducted on Switzerland’s Alps supercomputer at CSCS in Lugano, equipped with over 10,000 NVIDIA Grace Hopper chips, enabling efficient, carbon-neutral large-scale AI training through a long-term collaboration with NVIDIA and HPE/Cray.

The Alps supercomputer, powered entirely by renewable energy, leverages Switzerland’s sovereign AI infrastructure to facilitate large-scale, sustainable AI development.

The multilingual model is trained on texts in over 1,500 languages, with about 40% of the data in non-English languages, enhancing its usability across diverse cultural and linguistic contexts.

Designed for global applicability, the model can operate fluently in over 1,000 languages, trained on a dataset that includes both code and mathematics to support diverse applications.

Development adheres to Swiss data protection and copyright laws, aligning with the EU AI Act’s transparency requirements, and respects web crawling opt-outs with minimal impact on performance.

Responsible data practices are incorporated into the development process, ensuring compliance with legal standards and emphasizing transparency.

Supported by the Swiss AI Initiative, the project aims to promote trustworthy, inclusive AI and foster global collaboration through open science and extensive computational resources.

The model emphasizes transparency by providing open access to its source code, weights, and training data, supporting reproducibility, trust, and regulatory compliance across sectors.

Open models like this promote innovation, attract top talent, and foster international collaboration, contrasting with the closed development of commercial models and aiming to advance AI for societal benefit.

This project is part of the Swiss AI Initiative launched in December 2023, involving over 800 researchers and supported by 20 million GPU hours annually, making it the largest open science effort in AI foundation models.