aiagent.club
中文

Observability & evals

Tracing, evaluation, monitoring, cost control, and agent quality systems.

Rule-inferred category · a project may appear in more than one

Observability & evals · 3

Ranked projects have at least two current score dimensions across two source types. Score = observed adoption (45%) + momentum (30%) + attention (25%); stale or missing signals receive no weight.

  1. 2
    litellm GHPyPI

    The fastest, litest AI Gateway. Rust core with Python SDK. Call 100+ LLM APIs in OpenAI (or native) format with cost tracking, guardrails, load balancing, and logging [Bedrock, Azure, OpenAI, Anthropic, OpenAI, VertexAI, vLLM, Nvidia NIM]

    92.2M observed/mo 62.3k ★ ▲ 496 ★/wk 3/3 current signals

    90
  2. 40
    agentops GHPyPI

    Python SDK for AI agent monitoring, LLM cost tracking, benchmarking, and more. Integrates with most LLMs and agent frameworks including CrewAI, Agno, OpenAI Agents SDK, Langchain, Autogen, AG2, and CamelAI

    94.7k observed/mo 5.9k ★ ▲ 19 ★/wk 3/3 current signals

    22
  3. 45

    Symbol-level code retrieval for AI agents: fetch the function you need, not the whole file. Runs locally. 96.2% fewer tokens per lookup than grep-and-read on our 15-lookup benchmark. Works with Claude Code, Cursor, Codex and any MCP client.

    29.4k observed/mo 2.8k ★ ▲ 22 ★/wk 3/3 current signals

    14