Quickstart

Learn the common workflow for running any evaluator: choose content, run the evaluator, and review results.

What you’ll do

Run our Literacy evaluators on some text to evaluate it for a given grade level.

Method	When to use
Evaluators Playground	For a quick demo of how evaluators work
SDK	To integrate into your TypeScript or Python project
Python notebooks	For quick prototyping

What you’ll need

• Text you want to evaluate
• Grade level of the intended audience
• API keys required by the evaluators you want to use Not required for Evaluators Playground

Required API keys

By default, each evaluator uses a provider and model that we have tested for reliability for that task.

As a result, different evaluators require different API keys.

Evaluator	Required API key(s)
Literacy evaluators (except for Vocabulary and Sentence Structure)	Google
Vocabulary	Google and OpenAI
Sentence Structure	OpenAI
Feedback evaluators	OpenAI
Standards evaluators	Anthropic

You can override our evaluators’ defaults with any Google, OpenAI, or Anthropic model. When overriding with an Anthropic model, an Anthropic API key is required.

Steps

Evaluators Playground

The Evaluators Playground ↗ is the easiest way to see our evaluators in action.

1. Select target grade level

   Choose the grade level of your intended audience.

2. Provide your text

   Copy and paste the content you want to evaluate.

   
3. Run the evaluators

   Click Evaluate to run our Literacy evaluators on your text.

4. View results

   Expand each evaluator’s results to see its:

   • Text complexity score (e.g., “Exceedingly complex”)

   • Reasoning (e.g., “conceptually heavy Tier 3 words”)

   • Performance relative to a minimal baseline prompt (e.g., “13% more accurate”)

     

The Evaluators Playground is primarily a demo. Check out the SDK or Python notebooks when implementing evaluators into your automated processes.

SDK

1. Install the SDK

   Run evaluators from your project by installing the SDK of your choice.

   Language	Installation	Latest version
   TypeScript	GitHub ↗
   Python	GitHub ↗

   Contact us ↗ to request additional SDK language support. You can also sign up on the Learning Commons Platform ↗ for updates on availability.

2. Import and run evaluators

   Import evaluators to start evaluating text in your project:

   TypeScript

   import { GradeLevelAppropriatenessEvaluator } from "@learning-commons/evaluators";
   
   const evaluator = new GradeLevelAppropriatenessEvaluator({
     googleApiKey: process.env.GOOGLE_API_KEY,
   });
   
   const text = "The cat's out of the bag now.";
   
   const result = await evaluator.evaluate(text);
   console.log(result.score); // 4-5
   

See the SDK API Reference for more implementation details.

Python notebooks

The Python interpreter powers our evaluators. All downloadable examples ↗ and tutorials are provided as Python snippets.

Mac/Linux

Install Python 3.10 or newer. To verify your version of Python:

python3 --version

1. Create a virtual environment

   Creating an isolated environment prevents conflicts between Python packages used in this project and others on your system:

   python3 -m venv .venv
   
   source .venv/bin/activate # Activates environment
   

   Remember to activate the virtual environment for each new shell session.

2. Install dependencies

   Install all required packages listed in requirements.txt ↗:

   pip install -r evals/requirements.txt
   
3. Set your API keys

   Set your API keys as environment variables in your shell session:

   export OPENAI_API_KEY="sk-your-key-here"
   export GOOGLE_API_KEY="your-key-here"
   
4. Run an evaluator

   Start a Jupyter Notebook in your web browser (usually at http://localhost:8888):

   jupyter lab
   

   Create a new notebook by clicking the Notebook: Python 3 (ipykernel) tile in the Jupyter Lab launcher. Alternatively, select File > New > Notebook > Python 3 (ipykernel) in the menu.

   Run an evaluator by copying the code from our examples ↗ into the cells of your notebook.

   You can also use an IDE with Python and Jupyter Notebook support (e.g. VS Code with Microsoft’s Python and Jupyter extensions).

Windows

Install Python 3.10 or newer. To verify your version of Python:

python3 --version

1. Create a virtual environment

   In Command Prompt:

   python -m venv .venv
   .venv\Scripts\activate
   

   Or in PowerShell:

   python -m venv .venv
   
   .venv\Scripts\Activate.ps1
   

   Remember to activate the virtual environment for each new shell session.

2. Install dependencies

   Install all required packages listed in requirements.txt ↗:

   pip install -r evals/requirements.txt
   
3. Set your API keys

   Set your API keys as environment variables in Command Prompt:

   export OPENAI_API_KEY="sk-your-key-here"
   export GOOGLE_API_KEY="your-key-here"
   

   Or in PowerShell:

   $env:OPENAI_API_KEY="sk-your-key-here"
   $env:GOOGLE_API_KEY="your-key-here"
   
4. Run an evaluator

   Start a Jupyter Notebook in your web browser (usually at http://localhost:8888):

   jupyter lab
   

   Create a new notebook by clicking the Notebook: Python 3 (ipykernel) tile in the Jupyter Lab launcher. Alternatively, select File > New > Notebook > Python 3 (ipykernel) in the menu.

   Run an evaluator by copying the code from our examples ↗ into the cells of your notebook.

   You can also use an IDE with Python and Jupyter Notebook support (e.g. VS Code with Microsoft’s Python and Jupyter extensions).