AI parameters

The AI landscape is spearheaded by AI21 Labs and Databricks. They’re flipping the script on what we’ve come to expect from AI powerhouses. Let’s dive in.

AI21 Labs’ Jamba: The Lightweight Contender

Imagine an AI model that’s not just smart but also incredibly efficient. That’s Jamba for you. With just 12 billion parameters, Jamba performs on par with Llama-2’s 70 billion parameters. But here’s the kicker: it only needs 4GB of memory. Compare that to Llama-2’s 128GB. Impressive, right?

But let’s ask the question: How? It’s all about combining a Transformer neural network with something called a “state space model”. This combo is a game-changer, making Jamba not just another AI model, but a beacon of efficiency.

Databricks’ DBRX: The Smart Giant

On the other side, we have DBRX. This model is a beast with 132 billion parameters. But wait, it gets better. Thanks to a “mixture of experts” approach, it actively uses only 36 billion parameters. This not only makes it more efficient but also enables it to outshine GPT-3.5 in benchmarks, and it’s even faster than Llama-2.

Now, one might wonder, why go through all this trouble? The answer is simple: flexibility and customization. By making DBRX open-source, Databricks is handing over the keys to enterprises, allowing them to make this technology truly their own.

The Bigger Picture

Both Jamba and DBRX aren’t just models; they’re statements. They challenge the norm that bigger always means better. By focusing on efficiency and customization, they’re setting a new standard for what AI can and should be.

But here’s a thought: what does this mean for the closed-source giants? There’s a space for everyone, but the open-source approach is definitely turning heads. It’s about democratizing AI, making it accessible and customizable.

In a world where resources are finite, maybe the question we should be asking isn’t how big your model is, but how smartly you can use what you have. Jamba and DBRX are leading the charge, showing that in the race for AI supremacy, efficiency might just be the ultimate superpower.