Art jazzy di#2
Open
FieldDiTian wants to merge 17 commits into
Open
Conversation
… can be visualize on html
- keep only the newest pointcloud scan to avoid processing stale lidar data - add GNSS integrity veto to reject confident-but-wrong GICP matches - retune GICP and GT recovery thresholds for run_5 replay behavior - harden pipeline and replay wrappers to auto-source ROS env and local libs - fail fast when prep output is incomplete instead of falling through to GLIM - add synthetic RTK-denial hooks and fusion-analysis tooling for debugging
There was a problem hiding this comment.
Pull request overview
Note
Copilot was unable to run its full agentic suite in this review.
Adds an end-to-end, reproducible offline mapping + localization replay pipeline for AV-24 “Putnam-style” datasets, including bag prep from /atlas/* to /gps_p1/*, evaluation utilities, and cached visualization tooling.
Changes:
- Introduces pipeline scripts (bag prep, map export/rendering, replay orchestration) plus a Makefile wrapper and
dlio.envconfig template. - Adds RViz/Marker-based GNSS-vs-GICP error visualization (live + cached replay) and headless map/error renderers.
- Updates GLIM/GICP configs + docs to the Point One Atlas / ROS 2 Jazzy workflow (INS-driven mapping default, new map export tool).
Reviewed changes
Copilot reviewed 40 out of 42 changed files in this pull request and generated 10 comments.
Show a summary per file
| File | Description |
|---|---|
| scripts/show_cached_error_viz.sh | Launches cached error CSV replay into RViz without rerunning localization |
| scripts/run_localization_replay.sh | Orchestrates localization launch, GT bridge, error monitor, recording, and bag playback |
| scripts/run_dlio_pipeline.sh | One-command wrapper for prep → map → export → replay flow (supports dlio.env) |
| scripts/replay_cached_error_viz.py | Republishes cached GNSS/GICP paths + markers + rollercoaster for RViz |
| scripts/render_map.py | Headless PCD renderer for presentation-quality map images |
| scripts/render_error_rollercoaster.py | Headless 3D “rollercoaster” render of localization error |
| scripts/prep_bag.py | Converts raw /atlas/* + Luminar topics into prepped /gps_p1/* bag with restamping and UTM odom |
| scripts/live_gnss_error_monitor.py | Live RViz overlays + CSV+PNG logging of GNSS-vs-localization error |
| scripts/export_map_html.py | Exports a downsampled interactive Three.js map viewer bundle |
| scripts/eval_traj_vs_gnss.py | Evaluates GLIM dump trajectory vs GNSS RTK odometry |
| scripts/eval_odom_vs_gt.py | Evaluates recorded odom vs GT topic with gap filtering and stats |
| scripts/error_viz_common.py | Shared color/marker helpers for error visualization publishers |
| merge_luminar_pcap.py | Adds docstring and trims helper code in the Python PCAP merger reference |
| imgui.ini | Adds an ImGui window layout config file |
| gicp_localization/src/localization.cc | Updates Atlas-specific wording and a small Eigen initialization change |
| gicp_localization/scripts/utm_to_map_odom.py | New node to bridge UTM-frame odom into map frame using T_world_utm.txt |
| gicp_localization/launch/localization_with_tf.launch.py | Changes default GT topic, adds publish_tf and utm_transform_path args |
| gicp_localization/launch/localization.rviz | Adds GNSS path/odom + error marker displays and renames GICP path display |
| gicp_localization/include/gicp_localization/localization.h | Updates comment wording for removed legacy gating |
| gicp_localization/cfg/localization.yaml | Retunes GICP/localization defaults (voxel sizes, iterations, debug pub, deskew note) |
| gicp_localization/README.md | Updates docs from NovAtel to Point One Atlas wording |
| dlio.env.example | Adds tracked template for local pipeline configuration |
| dev_log/google drive.md | Adds a link to an external Google Doc |
| dev_log/20260613.md | Adds a short development log note |
| README.md | Updates top-level workflow/docs for PIPELINE.md, Jazzy, new tools, cached viz |
| PIPELINE.md | Adds a full end-to-end, command-by-command pipeline guide |
| Makefile | Adds build/test helpers and pipeline/viewing targets |
| GLIM/glim_ros2/src/iris_pcap_reader.cpp | Updates comments referencing the Python reference merger script |
| GLIM/glim_ros2/src/glim_pcap_rosbag.cpp | Updates comments around Python reference behavior |
| GLIM/glim_ros2/src/glim_dump_to_pcd.cpp | Adds a headless GLIM dump → PCD exporter executable |
| GLIM/glim_ros2/CMakeLists.txt | Builds/installs the new glim_dump_to_pcd executable |
| GLIM/glim_ext/config/config_gnss_global.json | Retunes/clarifies GNSS factor behavior for INS-driven mapping default |
| GLIM/glim/docs/quickstart.md | Adds a note directing users to DLIO++ workflow docs |
| GLIM/glim/config/config_sensors.json | Disables lidar_concat for mapping and enables lidar deskewing |
| GLIM/glim/config/config_ros.json | Clarifies extension module behavior; keeps viewer modules disabled |
| GLIM/glim/config/config_preprocess.json | Increases preprocess thread count |
| GLIM/glim/config/config_odometry_cpu.json | Clarifies bias estimation and increases thread count |
| GLIM/glim/config/config.json | Switches to INS-driven odometry + CPU mapping defaults and clarifies GNSS config |
| GLIM/README.md | Updates GLIM fork README to match INS-driven default and tuning rationale |
| AGENTS.md | Documents repo policy around machine-local config files |
| .gitignore | Ignores dlio_data/, .deps/, python caches, .vscode/, and dlio.env |
Comments suppressed due to low confidence (4)
scripts/run_localization_replay.sh:1
ros2 bag playtreats--clockas a flag (no value). The trailing100will be parsed as an extra positional arg or an unknown option, causing playback to fail. Remove100and, if the intent is a playback speed, use the supported option (e.g.,--rate 1.0) explicitly.
scripts/prep_bag.py:1- In common ROS 2
rosbag2_pyAPIs,TopicMetadatadoes not accept anidfield; passing it will raise aTypeErrorat runtime. ConstructTopicMetadatausing only the supported fields (typicallyname,type,serialization_format, and optionallyoffered_qos_profiles), and drop the manualtopic_idbookkeeping unless the underlying API explicitly requires it.
scripts/run_localization_replay.sh:1 - Using
pkill -9 -f ...is very broad and can kill unrelated user/system processes outside this script (and-9prevents clean shutdown, risking corrupted output bags/logs). Prefer terminating only the PIDs spawned by this script (and optionally their process groups), use SIGTERM first with a timeout, and reserve SIGKILL for the specific known PID only if it fails to exit.
scripts/replay_cached_error_viz.py:1 xyz.tolist()makes a full Python list copy of the Nx3 array, which can be a noticeable allocation at the default 250k points. Ifcreate_cloud_xyz32accepts annp.ndarray(iterable-of-rows) directly in your ROS 2 environment, passingxyzwithout converting can reduce peak memory and speed up startup.
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
Comment on lines
+1
to
+4
| [Window][Debug##Default] | ||
| Pos=60,60 | ||
| Size=400,400 | ||
| Collapsed=0 |
| @@ -0,0 +1 @@ | |||
| https://docs.google.com/document/d/1FjO1TtHoHTJD9a1jx8ujrE9DRgSY_LKrLvswrLurvr8/edit?usp=sharing No newline at end of file | |||
| VIEWS ?= overview,low,top,start | ||
| HTTP_PORT ?= 8000 | ||
| MAP_WEB ?= $(DATA_ROOT)/map_web | ||
| MAX_POINTS ?= 127000000 |
Comment on lines
+2310
to
+2312
| Eigen::Matrix4f initial_guess = this->deskew_ | ||
| ? Eigen::Matrix4f::Identity() | ||
| : (this->T_prior * T_base_lidar); | ||
| ? Eigen::Matrix4f(Eigen::Matrix4f::Identity()) | ||
| : Eigen::Matrix4f(this->T_prior * T_base_lidar); |
| constexpr uint16_t kEtherTypeQinQ = 0x88a8; | ||
|
|
||
| // Output PointCloud2 layout. MUST be byte-identical to merge_luminar_pcap.py | ||
| // Output PointCloud2 layout. MUST be byte-identical to scripts/merge_luminar_pcap.py |
| constexpr uint32_t kPointStep = 56; | ||
|
|
||
| // Per-point byte offsets inside one row (matches merge_luminar_pcap.py:48-71). | ||
| // Per-point byte offsets inside one row (matches PC2_FIELDS in scripts/merge_luminar_pcap.py). |
|
|
||
| // Bit-level reader for the Luminar Iris payload. Mirrors the Python | ||
| // read_bits() in merge_luminar_pcap.py. | ||
| // read_bits() in scripts/merge_luminar_pcap.py. |
| // Parse an Iris UDP payload into a header and a list of returns. Returns | ||
| // false if the packet is not a v1.3 data packet or the payload is malformed. | ||
| // Mirrors merge_luminar_pcap.py:583-693. | ||
| // Mirrors parse_iris_payload() in scripts/merge_luminar_pcap.py. |
| // --------------------------------------------------------------------------- | ||
| // AssemblerImpl: groups rays into scans, finalizes on key change or timeout. | ||
| // Mirrors merge_luminar_pcap.py:172-314 (ScanAssemblerPC2). | ||
| // Mirrors ScanAssemblerPC2 in scripts/merge_luminar_pcap.py. |
| By Di Tian | ||
| 1. from NovAtel raw IMU to Atlas P1 now | ||
| 2. added Error map between Dlio and GNSS info. GNSS as standard | ||
| 3. added prep_bag.py to filter bag info, compencate timestep |
# Conflicts: # GLIM/glim/config/config.json # gicp_localization/cfg/localization.yaml # gicp_localization/src/localization.cc # scripts/run_dlio_pipeline.sh # scripts/run_localization_replay.sh
# Conflicts: # GLIM/glim/config/config_sensors.json # gicp_localization/cfg/localization.yaml # gicp_localization/src/localization.cc
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Uh oh!
There was an error while loading. Please reload this page.