AutoE2E - End-to-End AI for Self Driving

DataModelConsole dashboard

The read-only DataModelConsole production dashboard brings AutoE2E datasets, model results and pipeline state into one workspace. Use it to:

• inspect published dataset versions, shards, samples and geographic coverage;
• play synchronized seven-camera scenes with ego-state and map context;
• compare ground-truth and model-predicted trajectories in camera and bird's-eye views;
• explore reasoning labels, MLflow models and Flyte executions.

Free and fully open-source End-to-End AI model

AutoE2E outputs can be fused with Physics-based sensors such as LIDAR/RADAR to power fully driverless Robotaxi applications, and the basline camera-only model can be used to enable L2++ automotive ADAS applications for point-to-point hands-free navigation.

To learn more about how to participate in this project, please read the onboarding guide

Getting started

Requires Python 3.12 (the pinned PyTorch build has no wheels for 3.13+).

Using make tool ###

git clone https://github.com/autowarefoundation/auto_e2e.git
   cd auto_e2e
   make setup                      # CPU torch wheels
   make setup TORCH_CHANNEL=cu118  # or a CUDA build (cu121, ... work too)

make test

Using plain pip ###

git clone https://github.com/autowarefoundation/auto_e2e.git
cd auto_e2e
pip install -r requirements.txt                      # CPU torch wheels
pip install -r requirements.txt --extra-index-url https://download.pytorch.org/whl/cu118  # or a CUDA build (cu121, ... work too)

Documentation

Review our academic paper, access our knowledge base and read through our work on safety verification in our documentation pages, alongside more information about the AutoE2E model at https://autowarefoundation.github.io/auto_e2e/

Next steps

• Explore the Model folder for the model components, training and inference.
• Follow the Trial Guide to run the inference test on AWS EC2.

Architecture at a glance

AutoE2E takes 7 surround and telephoto cameras plus a rendered map tile, along with egomotion and visual history, and predicts a 6.4s future driving trajectory (acceleration and curvature at 10Hz). See the Model architecture guide for the full inputs, outputs and forward signature.

Performance

Up to ~76 FPS (SwinV2-Tiny, feature-concat fusion, RTX 5080, batch 1). Full per-GPU inference benchmarks covering latency, jitter and VRAM across backbones, fusion modes and batch sizes live in BENCHMARKS.md. Run the benchmarking script to add results for your own GPU.