Jonathan Adashek’s Post

Transcript

When it comes to picking a large language model with support for choice, last time I checked there was something like 700,000 different Ellms or large language models on Hugging Face. Now I'd like to cover just a couple of those, specifically the IBM Granite Foundation models. But first let's consider how to pick an enterprise grade foundation model, meaning an LLM suitable for deployment in an enterprise setting. Something you'd be happy to run your business with O Let's consider that through three different metrics. O, the foundation model, it needs to be performant. That's an important metric, but it also needs to be. Cost. Effective, and it needs to be. Trusted. Those are the three metrics we're going to consider and trusted of course, because you can't scale generative AI with models that you cannot trust O take these one by one now by. Performant we're talking about measurements like latency and throughput. Is the foundation model able to keep U with the seed and enterprise requires it to operate at? Then related to that. Is cost effectiveness. Now, according to the scientific journal Nature, a search that's driven by generative AI will use something like four to five times the amount of energy that's needed to run a conventional web search. So we need a foundation model that can deliver the necessary performance with low inferencing costs, and we need the foundation model. To be trusted and we can gauge that through metrics like hallucination scores, but also a model that offers transparency. So we know what data the model was trained on. And I think in many instances, models are kind of skewed a bit like this. They're highly performant, but they're expensive to run at inference time. And there's a lack of transparency on the training data the model was built with now. With the Granite models, IBM set out to create enterprise grade foundation models that alley an equal weight to all three of these metrics. So it looks more like this. So what should you know about the IBM Granite Foundation models? Well, many of the models are. Open source. You can find them on Hugging Face under the Apache 2.0 license. That enables broad commercial usage. Now these models also have transparency in training data, meaning we actually know the data sources that we use to train the models, and that's quite atypical. Most MLMS are notoriously vague on how their models were trained, so that's a nice change. Now Granite language models are trained on trusted enterprise. Data spanning academic code, legal and finance data sources as such as well the first 13 billion parameter Granite LLM was trained on about 6.5 terabytes of data and that includes 1.8 million scientific papers that were posted on archive. It also includes all US utility patterns granted by the US. DTO, and that's from 1975 all the way through to 2023 and it includes the public domain free law. Which are legal opinions from U.S. Federal and state courts. Essentially, the models have been governed and filtered to only use enterprise safe data sources. The granite models have also been designed to be performant as well. Especially in areas of coding and language tasks, outperforming some models that are actually twice their size and smaller models means also. Then more efficient with less compute requirements and at lower cost of inferencing. Now I keep mentioning the granite models, plural. So which models are we talking about? So granite is actually a family of LLM Foundation models spanning multiple modalities. And you can find many of these on Hugging Face. So let's take a look at some of them and we'll start with granite for language. Now these are decoder models of different parameter sizes O that includes A7B open source model and the B here refers to billions of parameters, so 7 billion parameters. There's also an. 8B model that's designed specifically for Japanese text. There's a couple of 13B models and there is a 20 billion parameter multilingual model that supports English, German, Spanish, French, and Portuguese. Now there's also granite for. Code. And that again comes in different parameter sizes, from 3 billion all the way through to 34 billion parameters. And granite for code is trained on 116 programming languages. Now there's also Granite for. Time series. That's a family of retrain models for time series forecasting. These models are trained on a collection of data sets spanning a range of business and industrial application domains. And these models are optimized to run on pretty much anything, even a laptop. And then finally, there is Granite for Geospatial, which is a partnership between NASA and IBM to create a foundation model for Earth observations using large scale. Satellite and remote sensing data O that's the IBM Granite models, models that are trusted performance and efficient and that can be applied to a wide variety of enterprise use cases.