Pharia Studio SDK

Welcome to the Pharia Studio SDK 👋

The Pharia Studio SDK provides a comprehensive set of tools for evaluating and benchmarking Large Language Models (LLMs). Formerly the intelligence_layer/evaluation package.

Key Features

  • 🔗 Connectors: Integration with Studio and Argilla platforms

  • 📊 Evaluation: Comprehensive evaluation framework with aggregation and benchmarking

  • 🎯 Benchmarking: Tools for creating and running LLM benchmarks

  • 📈 Dataset Management: Repository patterns for managing evaluation datasets

Getting Started

  • New to the Pharia Studio SDK? Start with the Core Concepts to understand our architecture and key concepts.

  • Ready to build? Check out the Quick Start to get up and running in minutes.

  • Need detailed API information? Explore our comprehensive SDK Reference for all classes and methods.

  • Want to contribute? See our Contributing Guide for details on how to set up the development environment and submit changes.