Clearly explained guide for running quantized open-source LLM applications on CPUs using Llama 2, C Transformers, GGML, and LangChain — Third-party commercial large language model (LLM) providers like OpenAI’s GPT4 have democratized LLM use via simple API calls. However, teams may still require self-managed or private deployment for model inference within enterprise perimeters due to various reasons around data privacy and compliance.

This blog is the first in a series of posts explaining why the mainstream pipeline for deploying domain-specialized chatbots with large language models (LLMs) is too costly and inefficient. In this first post, we discuss why vector databases, despite the recent surge in popularity, are fundamentally limited when deployed in…

A step-by-step guide using the unofficial HuggingChat API (no APIs required) — Introduction The remarkable progress made in natural language processing and large language models (LLMs) over the past few months has led to the widespread adoption of generative AI and chatbots. Chatbots that are powered by large language models, like ChatGPT, have become particularly popular. These chatbots are capable of understanding and…

Why DataFrame libraries need to understand vector embeddings — This post is an expanded version of the upcoming joint talk Quokka + LanceDB talk at the Data+AI Summit this June. Vector embeddings are here to stay Vector embeddings are here to stay. It is hard to conceive of constructing recommendation system, search engine, or LLM-based app in this day and age without using vector embeddings…

How LLMs are changing the way we build AI-powered products and the landscape of MLOps — This article was originally published on the Weights & Biases’ blog “Fully Connected” on April 21st, 2023. It feels like the release of OpenAI’s ChatGPT has opened Pandora’s box of large language models (LLMs) in production. Not only does your neighbor now bother you with small talk about Artificial Intelligence…