Multimodal-Wireless

Data Collection

This section details the data collection parameters. Our dataset comprises 7 modalities captured within the CARLA simulator. The primary sensor suite includes LiDAR, RGB cameras, depth cameras, radar, and IMU. Additionally, we provide GPS and ground-truth bounding box data for advanced tasks. The wireless channel modality is characterized by multi-path properties, including complex channel gains, path delays, Angles of Departure (AoD), and Angles of Arrival (AoA).

File Hierarchy (V2I)

Multimodal-Wireless
|
|-- Town03
|   |
|   `-- Town03_CBDcrossroad_seed42
|       |
|       |-- cav_1
|       |   |-- 000001.yaml
|       |   |-- 000001.pcd
|       |   |-- 000001_paths.npy
|       |   |-- 000001_camera0.png
|       |   |-- 000001_camera1.png
|       |   |-- 000001_camera2.png
|       |   `-- 000001_camera3.png
|       |
|       |-- cav_2
|       |   `-- ...
|       |
|       `-- rsu_1
|           |-- 000001.yaml
|           |-- 000001.pcd
|           `-- ...
|
|-- Town05
|   `-- Town05_... (Same structure)
|
`-- Town10
    `-- Town10_... (Same structure)

Description of Modalities

The dataset is organized by simulation towns, scenarios, and participating agents (CAVs and RSUs). All collected modalities are synchronized and aligned by a six-digit frame number.

Connected and Automated Vehicle (CAV)

  • LiDAR: 3D point cloud data stored in .pcd files.
  • RGB Cameras: Four forward, backward, left, and right-facing camera images stored in .png files.
  • Metadata (in .yaml):
    • GPS location estimates and IMU information.
    • Ground-truth bounding box info for other vehicles/RSUs detected by the LiDAR. The RSU is identified with an ID of 0.
  • Channel Information: Files ending with _paths.npy contain V2I channel data, representing the downlink from the RSU (transmitter) to the CAV (receiver).

Roadside Unit (RSU)

  • LiDAR: 3D point cloud data in .pcd files.
  • Cameras: Forward-facing RGB and depth camera images as .png files.
  • Metadata (in .yaml):
    • Ground-truth bounding box info for other vehicles/RSUs detected by the LiDAR.
  • Radar: Radar point cloud data stored in .json files.

Modality Configuration Details

CAV

RSU

Channel