Grounded-SAM-2

Grounded SAM 2: Ground and Track Anything with Grounding DINO and SAM 2

Contents

Installation

Since we need the CUDA compilation environment to compile the Deformable Attention operator used in Grounding DINO, we need to check whether the CUDA environment variables have been set correctly (which you can refer to Grounding DINO Installation for more details). You can set the environment variable manually as follows if you want to build a local GPU environment for Grounding DINO to run Grounded SAM 2:

export CUDA_HOME=/path/to/cuda-12.1/

Install segment-anything-2:

pip install -e .

Install grounding dino:

pip install --no-build-isolation -e grounding_dino

Download the pretrained grounding dino and sam 2 checkpoints:

cd checkpoints
bash download_ckpts.sh

cd gdino_checkpoints
wget https://github.com/IDEA-Research/GroundingDINO/releases/download/v0.1.0-alpha/groundingdino_swint_ogc.pth
wget https://github.com/IDEA-Research/GroundingDINO/releases/download/v0.1.0-alpha2/groundingdino_swinb_cogcoor.pth

Run demo

Grounded-SAM-2 Image Demo

Note that Grounding DINO has already been supported in Huggingface, so we provide two choices for running Grounded-SAM-2 model:

• Use huggingface API to inference Grounding DINO (which is simple and clear)

python grounded_sam2_hf_model_demo.py

• Load local pretrained Grounding DINO checkpoint and inference with Grounding DINO original API (make sure you've already downloaded the pretrained checkpoint)

python grounded_sam2_local_demo.py