Mozilla.ai’s Post

View organization page for Mozilla.ai, graphic

3,205 followers

When contributing to #opensource: 5. The LLM ecosystem is currently bifurcated between Hugging Face and OpenAI compatibility. In this new space of developer tooling around transformer-style language models at an industrial scale, you are generally conforming to be downstream of one of two interfaces: a) models that are trained and hosted using HuggingFace libraries and particularly the HuggingFace hub as infrastructure - in practicality, this means dependence on PyTorch’s programming paradigm, which HuggingFace tools wrap (although they now also provide interop between Tensorflow and JAX) b) models available via API endpoints, particularly as hosted by OpenAI. Given that OpenAI was a first mover in the product LLM space, they currently have the API advantage, and many tools that have developed have done so with OpenAI-compatible endpoints which don’t always mean using OpenAI, but conform to the same set of patterns that the Chat Completions API `v1/chat/completions` offers. For example, adding OpenAI-style interop chat completions allowed us to stand up our own vLLM OpenAI-compatible server that works against models we’ve started with on HuggingFace and fine-tuned locally. If you want to be successful in this space today, you as a library or service provider have to be able to interface with both of these.

Open Source in the Age of LLMs

Open Source in the Age of LLMs

blog.mozilla.ai

To view or add a comment, sign in

Explore topics