CoralNet-Toolbox

CoralNet-Toolbox 🪸🧰

[](https://pypi.org/project/CoralNet-Toolbox) [](https://pypi.python.org/pypi/CoralNet-Toolbox) [](https://pypi.org/project/CoralNet-Toolbox) [](https://pypi.org/project/CoralNet-Toolbox) [](https://pypi.org/project/CoralNet-Toolbox) [](https://pypi.org/project/CoralNet-Toolbox)

🔍 Annotation Create patches, rectangles, and polygons with AI assistance

🧠 AI-Powered Leverage SAM, YOLOE, and various foundation models

🚀 Complete Workflow From data collection to model training and deployment

🚦 Quick Start

Running the following command will install the coralnet-toolbox, which you can then run from the command line:

# cmd

# Install
pip install coralnet-toolbox

# Run
coralnet-toolbox

📚 Guides

For further instructions please see the following guides:

• Installation
• Usage
• Patch-based Image Classifier

🎥 Watch the Video Demos

⏩ TL;Dr

The CoralNet-Toolbox is an unofficial codebase that can be used to augment processes associated with those on CoralNet.

It uses✨Ultralytics🚀 as a base, which is an open-source library for computer vision and deep learning built in PyTorch. For more information on their AGPL-3.0 license, see here.

🚀 Supported Models

The toolbox integrates a variety of state-of-the-art models to help you create rectangle and polygon annotations efficiently. Below is a categorized overview of the supported models and frameworks:

| Category | Models | |-------------------------|---------------------------------------------------------------------------------------------------------| | **Trainable** | - 🦾 [YOLOv3](https://docs.ultralytics.com/models/)
- 🦈 [YOLOv4](https://docs.ultralytics.com/models/)
- 🦅 [YOLOv5](https://docs.ultralytics.com/models/)
- 🐬 [YOLOv6](https://docs.ultralytics.com/models/)
- 🐢 [YOLOv7](https://docs.ultralytics.com/models/)
- 🐙 [YOLOv8](https://docs.ultralytics.com/models/)
- 🐠 [YOLOv9](https://docs.ultralytics.com/models/)
- 🦑 [YOLOv10](https://docs.ultralytics.com/models/)
- 🚀 [YOLO11](https://docs.ultralytics.com/models/)
- 🐳 [YOLO12](https://docs.ultralytics.com/models/) | | **Segment Anything** | - 🪸 [SAM](https://github.com/facebookresearch/segment-anything)
- 🌊 [CoralSCOP](https://github.com/zhengziqiang/CoralSCOP)
- ⚡ [FastSAM](https://github.com/CASIA-IVA-Lab/FastSAM)
- 🔁 [RepViT-SAM](https://github.com/THU-MIG/RepViT)
- ✂️ [EdgeSAM](https://github.com/chongzhou96/EdgeSAM)
- 📱 [MobileSAM](https://github.com/ChaoningZhang/MobileSAM) | | **Visual Prompting** | - 👁️ [YOLOE](https://github.com/THU-MIG/yoloe)
- 🤖 [AutoDistill](https://github.com/autodistill):
   • 🦒 Grounding DINO
   • 🦉 OWLViT
   • ⚡ OmDetTurbo |

These models enable fast, accurate, and flexible annotation workflows for a wide range of use cases for patch-based image classification, object detection, instance segmentation.

🛠️ Toolbox Features

| 
_{**Patch Annotation**} | 
_{**Rectangle Annotation**} | 
_{**(Multi) Polygon Annotation**} | |:--:|:--:|:--:| | 
_{**Image Classification**} | 
_{**Object Detection**} | 
_{**Instance Segmentation**} | | 
_{**Segment Anything (SAM)**} | 
_{**Polygon Classification**} | 
_{**Region-based Detection**} | | 
_**Cut** | 
_**Combine** | 
_**Simplify** | | 
_{**See Anything (YOLOE)**} | 
_{**Patch-based LAI Classification**} | 
_{**Video Inference**} |

</div>

Enhance your CoralNet experience with these tools:

• 📥 Download: Retrieve Source data (images and annotations) from CoralNet
• 🎬 Rasters: Import images, or extract frames directly from video files
• ✏️ Annotate: Create annotations freely
• 👁️ Visualize: See CoralNet and CPCe annotations superimposed on images
• 🔬 Sample: Sample patches using various methods (Uniform, Random, Stratified)
• 🧩 Patches: Create patches (points)
• 🔳 Rectangles: Create rectangles (bounding boxes)
• 🟣 Polygons: Create polygons (instance masks)
  • 👨‍👩‍👧‍👦 MultiPolygons: Combine multiple, non-overlapping polygons (i.e, genets)
• ✍️ Edit: Cut and Combine polygons and rectangles
• 🦾 SAM: Use FastSAM, CoralSCOP, RepViT-SAM, EdgeSAM, MobileSAM, and SAM to create polygons
  • Uses xSAM
• 👷‍♂️ Work areas: Perform region-specific detections / segmentations with any model
• 👀 YOLOE (See Anything): Detect similar appearing objects using visual prompts automatically
• 🧪 AutoDistill: Use AutoDistill to access the following for creating rectangles and polygons:
  • Uses Grounding DINO, OWLViT, OmDetTurbo
• 📻 Tune: Tune hyperparameters to identify ideal training conditions
• 🧠 Train: Build local patch-based classifiers, object detection, and instance segmentation models
• 🔮 Deploy: Use trained models for predictions
• 📊 Evaluation: Evaluate model performance
• 🚀 Optimize: Productionize models for faster inferencing
• ⚙️ Batch Inference: Perform predictions on multiple images, automatically
• 🎞️ Video Inference: Perform predictions on a video in real-time, record the output and analytics
• ↔️ I/O: Import and Export annotations from / to CoralNet, Viscore, and TagLab
  • Export annotations as GeoJSONs, segmentation masks
• 📸 YOLO: Import and Export YOLO datasets for machine learning
• 🧱 Tile Dataset: Tile existing Detection / Segmentation datasets
  • Uses yolo-tiling
• 🏗️ Tile Inference: Pre-compute multiple work areas for an entire image

📝 TODO

• 🤗 Model Zoo: Download Ultralytics models from HuggingFace for use in toolbox
• 🦊 BioCLIP, MobileCLIP (AutoDistill): Automatically classify annotations
• 📦 Toolshed: Access tools from the old repository

This site is open source. Improve this page.