> ## Documentation Index
> Fetch the complete documentation index at: https://mintlify.com/NVIDIA/Isaac-GR00T/llms.txt
> Use this file to discover all available pages before exploring further.

# BEHAVIOR benchmark

> 50 household loco-manipulation tasks with Galaxea R1 Pro robot

BEHAVIOR is a benchmark testing loco-manipulation capabilities across 50 diverse household tasks. The benchmark uses the Galaxea R1 Pro robot in simulated home environments powered by OmniGibson.

For more information, see the [BEHAVIOR website](https://behavior.stanford.edu/).

## Benchmark results

Checkpoint: [nvidia/GR00T-N1.6-BEHAVIOR1k](https://huggingface.co/nvidia/GR00T-N1.6-BEHAVIOR1k)

GR00T N1.6 achieves **26.30% average task progress** across all 50 tasks, compared to Pi0.5's 11.30%.

### Top performing tasks

| Task                                      | Task progress (N1.6) |
| ----------------------------------------- | -------------------- |
| clean\_a\_trumpet                         | 60.00%               |
| getting\_organized\_for\_work             | 53.57%               |
| boxing\_books\_up\_for\_storage           | 51.54%               |
| attach\_a\_camera\_to\_a\_tripod          | 46.00%               |
| make\_microwave\_popcorn                  | 45.00%               |
| picking\_up\_trash                        | 44.87%               |
| turning\_on\_radio                        | 43.33%               |
| clearing\_food\_from\_table\_into\_fridge | 42.31%               |

<Note>
  Task Progress is a denser metric than Q Score, measuring the proportion of subtasks completed within each household activity.
</Note>

### Individual task post-training

Starting from the base checkpoint, post-training on individual tasks shows significant improvement:

| Task                            | Task progress | Q score |
| ------------------------------- | ------------- | ------- |
| turning\_on\_radio              | 80.56%        | 0.70    |
| chopping\_wood                  | 20.00%        | 0.125   |
| cleaning\_up\_plates\_and\_food | 22.00%        | 0.11    |
| setting\_mousetraps             | 19.17%        | 0.10    |

## Fine-tuning

<Steps>
  <Step title="Download dataset">
    Download the BEHAVIOR dataset from HuggingFace (all 50 tasks):

    ```bash theme={null}
    huggingface-cli download nvidia/PhysicalAI-Robotics-GR00T-X-Embodiment-Sim \
        --repo-type dataset \
        --include "sim_behavior_r1_pro.*" \
        --local-dir $HOME/gr00t_dataset
    ```

    To download a specific task, replace `sim_behavior_r1_pro.*` with the task name.
  </Step>

  <Step title="Run fine-tuning">
    ```bash theme={null}
    uv run bash examples/BEHAVIOR/finetune_BEHAVIOR.sh
    ```

    Note the use of `BEHAVIOR_R1_PRO` embodiment tag.
  </Step>
</Steps>

## Evaluation

### Setup environment

<Warning>
  BEHAVIOR simulation is built on Omniverse and Isaac Sim. GPUs without RT cores (A100, H100) are not supported. Tested on L40 and L40s. See [Isaac Sim requirements](https://docs.isaacsim.omniverse.nvidia.com/latest/installation/requirements.html) for details.
</Warning>

<Steps>
  <Step title="Clone and setup BEHAVIOR-1K">
    ```bash theme={null}
    git clone https://github.com/StanfordVL/BEHAVIOR-1K.git
    cd BEHAVIOR-1K

    # Checkout branch with task progress metric
    git checkout feat/task-progress

    # Activate GR00T uv environment
    source PATH_TO_GR00T/.venv/bin/activate

    # Headless installation (auto-accepts EULA and license)
    bash ./setup_uv.sh
    ```
  </Step>

  <Step title="Download test instances">
    Download test cases from the [BEHAVIOR Challenge](https://behavior.stanford.edu/challenge/index.html):

    ```bash theme={null}
    python gr00t/eval/sim/BEHAVIOR/prepare_test_instances.py
    ```
  </Step>
</Steps>

### Run evaluation

<Steps>
  <Step title="Start policy server">
    In Terminal 1:

    ```bash theme={null}
    uv sync --python 3.10
    uv pip install -e .

    uv run gr00t/eval/run_gr00t_server.py \
        --model-path nvidia/GR00T-N1.6-BEHAVIOR1k \
        --embodiment-tag BEHAVIOR_R1_PRO \
        --use-sim-policy-wrapper
    ```
  </Step>

  <Step title="Start evaluation client">
    In Terminal 2:

    ```bash theme={null}
    uv run python gr00t/eval/rollout_policy.py \
        --n_episodes 10 \
        --policy_client_host 127.0.0.1 \
        --policy_client_port 5555 \
        --max_episode_steps=999999999 \
        --env_name sim_behavior_r1_pro/turning_on_radio \
        --n_action_steps 8 \
        --n_envs 1
    ```

    <Note>
      We set `max_episode_steps` to a large value because BEHAVIOR uses 2x human steps as the horizon. Set a smaller value for faster debugging. Video recording is disabled to prevent simulation crashes with `decord`.
    </Note>
  </Step>
</Steps>

## Task list

All 50 BEHAVIOR tasks are available with the `sim_behavior_r1_pro/` prefix:

<AccordionGroup>
  <Accordion title="View all 50 tasks">
    * sim\_behavior\_r1\_pro/turning\_on\_radio
    * sim\_behavior\_r1\_pro/hanging\_pictures
    * sim\_behavior\_r1\_pro/make\_microwave\_popcorn
    * sim\_behavior\_r1\_pro/attach\_a\_camera\_to\_a\_tripod
    * sim\_behavior\_r1\_pro/picking\_up\_trash
    * sim\_behavior\_r1\_pro/clean\_a\_trumpet
    * sim\_behavior\_r1\_pro/set\_up\_a\_coffee\_station\_in\_your\_kitchen
    * sim\_behavior\_r1\_pro/chop\_an\_onion
    * sim\_behavior\_r1\_pro/spraying\_for\_bugs
    * sim\_behavior\_r1\_pro/hiding\_Easter\_eggs
    * sim\_behavior\_r1\_pro/cook\_bacon
    * sim\_behavior\_r1\_pro/putting\_shoes\_on\_rack
    * sim\_behavior\_r1\_pro/clean\_boxing\_gloves
    * sim\_behavior\_r1\_pro/preparing\_lunch\_box
    * sim\_behavior\_r1\_pro/spraying\_fruit\_trees
    * sim\_behavior\_r1\_pro/wash\_a\_baseball\_cap
    * sim\_behavior\_r1\_pro/rearranging\_kitchen\_furniture
    * sim\_behavior\_r1\_pro/setting\_the\_fire
    * sim\_behavior\_r1\_pro/bringing\_water
    * sim\_behavior\_r1\_pro/cook\_hot\_dogs
    * sim\_behavior\_r1\_pro/setting\_mousetraps
    * sim\_behavior\_r1\_pro/outfit\_a\_basic\_toolbox
    * sim\_behavior\_r1\_pro/chopping\_wood
    * sim\_behavior\_r1\_pro/putting\_dishes\_away\_after\_cleaning
    * sim\_behavior\_r1\_pro/tidying\_bedroom
    * sim\_behavior\_r1\_pro/wash\_dog\_toys
    * sim\_behavior\_r1\_pro/can\_meat
    * sim\_behavior\_r1\_pro/sorting\_vegetables
    * sim\_behavior\_r1\_pro/clean\_a\_patio
    * sim\_behavior\_r1\_pro/freeze\_pies
    * sim\_behavior\_r1\_pro/clearing\_food\_from\_table\_into\_fridge
    * sim\_behavior\_r1\_pro/bringing\_in\_wood
    * sim\_behavior\_r1\_pro/cleaning\_up\_plates\_and\_food
    * sim\_behavior\_r1\_pro/putting\_up\_Christmas\_decorations\_inside
    * sim\_behavior\_r1\_pro/putting\_away\_Halloween\_decorations
    * sim\_behavior\_r1\_pro/cook\_cabbage
    * sim\_behavior\_r1\_pro/carrying\_in\_groceries
    * sim\_behavior\_r1\_pro/moving\_boxes\_to\_storage
    * sim\_behavior\_r1\_pro/getting\_organized\_for\_work
    * sim\_behavior\_r1\_pro/sorting\_household\_items
    * sim\_behavior\_r1\_pro/picking\_up\_toys
    * sim\_behavior\_r1\_pro/collecting\_childrens\_toys
    * sim\_behavior\_r1\_pro/make\_pizza
    * sim\_behavior\_r1\_pro/loading\_the\_car
    * sim\_behavior\_r1\_pro/storing\_food
    * sim\_behavior\_r1\_pro/clean\_up\_your\_desk
    * sim\_behavior\_r1\_pro/canning\_food
    * sim\_behavior\_r1\_pro/boxing\_books\_up\_for\_storage
    * sim\_behavior\_r1\_pro/assembling\_gift\_baskets
    * sim\_behavior\_r1\_pro/slicing\_vegetables
  </Accordion>
</AccordionGroup>