IIT Madras sets up Centre of AI Research to solve compute accessibility challenges

Kompact AI supports India’s AI Mission, “AI for All”, and has been developed by Indian scientists and engineers. Ziroh Labs has optimised 17 AI models — including DeepSeek, Qwen, and Llama — to run efficiently on CPUs. These models have been benchmarked in collaboration with IIT Madras, assessing both their quantitative performance and qualitative accuracy. This marks the first instance of high-performance AI being efficiently built and deployed on CPUs, enhancing accessibility, scalability, and cost-effectiveness.

Ziroh Labs team showcased Kompact AI’s unique capabilities during the event and demonstrated how it redefines AI development on top of CPUs by eliminating dependency on GPUs.

The demonstration took place in the presence of Prof. V. Kamakoti, Director, IIT Madras, Dr Whitfield Diffie, Fellow of the Royal Society and Turing Awardee, Mr Scott McNealy, Founder of Sun Microsystems, Dr William J Raduchel, Former CTO of AOL Time Warner and Chief Strategy Officer at Sun Microsystems, Prof. S. Sadagopan, Former Director, IIIT-Bangalore and Mr Raj Singh, Founder, APIGee, Cerrent, Fiberlanem, alongside other students, faculty and researchers.

Key features of Kompact AI includes efficient AI model development and deployment on CPUs — eliminating the need for costly GPUs, optimizing models like DeepSeek, Qwen, and Llama for low-cost performance, and operating without constant internet, making it ideal for remote areas.

Elaborating on the need for such developments, Prof. V. Kamakoti, Director, IIT Madras, said, “Nature has taught us that one can effectively acquire knowledge and subsequently infer in only a limited set of domains. Attempts to acquire everything under the universe are not sustainable and bound to fail over a period of time.”

Prof. V. Kamakoti added, “This effort by Ziroh Labs and IITM Pravartak is a nature-inspired one wherein they provide a platform that uses customized and trained domain-specific models to provide accurate inferences on affordable conventional compute machines. This effort is certainly a major step in arresting the possible AI divide between one who can afford the modern hyper scalar systems and one who cannot.”

Kompact AI can play a transformative role as it leverages multiple optimisations algorithmically and implementation-wise to execute models on the low end without any support required for external communication, such as the Internet. As a part of this upcoming CoAIR, IIT Madras will work on smaller AI models for specific use cases, leveraging Kompact AI to optimise execution, making high-performance AI more efficient, accessible, and relevant to real-world applications.

This AI platform aims to democratise AI by allowing developers across the globe to build, train and infer AI using CPUs commonly found in Cloud Data Centers and Edge Devices. Kompact AI enables AI development without violating any data privacy and data residency regulations across the globe, an IIT Madras statement issued in this regard said.