LOOM-Eval Documentation

Welcome to LOOM-Eval, a comprehensive framework for evaluating long-context language models.

Overview

LOOM-Eval provides:

• Comprehensive Benchmarks: 26 standard long-context benchmarks

• Efficient Inference: Support for vLLM and 17 acceleration methods

• Model Compatibility: Works with Transformers, Mamba, and RWKV

• Easy to Use: Simple command-line interface and WebUI

Getting Started

• Installation
  • Basic Installation
  • Acceleration Methods
  • RAG Installation
  • Next Steps
• Quick Start
  • Prerequisites
  • Basic Usage
  • Key Parameters
  • WebUI Usage
  • Example Scenarios
  • Running LOOMBench Suite
  • Output Structure
  • Next Steps

User Guide

• Acceleration Methods
  • KV Cache Optimization
  • Sparse Attention
  • Usage
  • Performance (128K Context)
  • Model Compatibility
  • Installation Notes
  • Hardware Requirements
• API Reference
  • Command Line Interface
  • Custom Templates
  • Examples
• RAG (Retrieval-Augmented Generation)
  • Supported Methods
  • Installation
  • Quick Start
  • Configuration
  • Command-line Options
  • Task Compatibility
  • Best Practices

Quick Links

• Index

• Search Page

• Project Page

• Overview Page