Open Source AI
Local LLMs, open models, free AI infrastructure
7 min readMPT-7B: MosaicML's Commercial-Ready LLM With ALiBi Attention
MPT-7B introduced ALiBi positional encoding for length generalization and shipped with an Apache 2.0 license, making it one of the first truly commercial-ready open LLMs.
7 min readGemma 2 27B: Google's Open Model With Novel Architecture Choices
Gemma 2 27B beats Llama 3 70B on MMLU (75.2% vs 73.1%) using knowledge distillation from Gemini and a novel sliding window attention design.
7 min readLlama 3.2 Vision: Meta's First Multimodal Open-Source Model
Llama 3.2 introduces vision capability to the Llama family with 11B and 90B vision models, plus 1B and 3B text-only variants for on-device deployment.
7 min readOLMo 2: Allen AI's Fully Open LLM (Weights + Data + Code)
OLMo 2 is the only major LLM where you can reproduce the entire training run: weights, 3T-token Dolma dataset, training code, and evaluation suite are all public.
7 min readLLaVA 1.6: Open-Source Visual Instruction Tuning That Rivals GPT-4V
LLaVA 1.6 (LLaVA-Next) improves on its predecessor with dynamic high-resolution processing and 4x more instruction tuning data, achieving MMBench scores competitive with GPT-4V on several benchmarks.
8 min readLlama 3.3 70B: Why Meta's November 2024 Release Rivals 405B
Llama 3.3 70B closes most of the gap with the 405B model through better instruction following data and RLHF improvements - delivering 405B-class performance at a fraction of the serving cost.
7 min readLlama 3.1 405B: Meta's Open-Source Answer to GPT-4
Llama 3.1 405B achieves 88.6% on MMLU and matches GPT-4 on multiple benchmarks, with a commercial license for up to 700M MAU. Here's how to run it.
7 min readOpenHermes 2.5: Mistral Fine-Tuned on 1M Synthetic GPT-4 Conversations
Nous Research's OpenHermes 2.5 demonstrates that one million carefully curated synthetic conversations can produce an instruction model that rivals much larger open weights.
7 min readZephyr 7B: Fine-Tuning Mistral for Alignment With Synthetic Data
HuggingFace H4 aligned a 7B model to beat Llama 2 70B Chat using only synthetic GPT-4 data and DPO - no reinforcement learning required.
7 min readMistral 7B Instruct v0.3: The Best 7B Open Model for Production
Mistral 7B Instruct v0.3 delivers 32K context, function calling, and inference efficiency that rivals much larger models - here is how to deploy it.