Perception Autonomy Engineer

About the Role

Caterpillar’s Cat Technology team is seeking a Perception Developer to design, develop, and deploy perception software for autonomous and semi-autonomous machines used in construction. The role focuses on multi-sensor perception, computer vision, 3D point-cloud processing, sensor fusion, and production-quality software for safety-critical, resource-constrained platforms. The ideal candidate combines strong C++/Python development skills with hands-on experience building, optimizing, and validating perception algorithms from prototype through field deployment.

Key Responsibilities

• Design and implement perception algorithms for object and personnel detection, classification, segmentation, tracking, terrain understanding, localization, mapping, and obstacle assessment.
• Develop camera, LiDAR, radar, GNSS, and IMU processing pipelines, including calibration, synchronization, sensor fusion, and uncertainty handling.
• Translate research concepts and state-of-the-art models into robust, maintainable, production-quality C++ and Python software.
• Optimize algorithms and neural-network inference for embedded CPU/GPU platforms using profiling, CUDA, TensorRT, and related acceleration technologies.
• Build automated evaluation workflows, datasets, simulation scenarios, and test tools to measure accuracy, latency, robustness, and safety performance.
• Debug and resolve perception issues using recorded data, simulation, hardware-in-the-loop systems, and field-machine testing.
• Collaborate with systems, autonomy, controls, embedded software, validation, and safety teams to define interfaces and deliver integrated machine capabilities.
• Participate in architecture reviews, code reviews, technical documentation, and continuous improvement of software engineering practices.

Requirements & Tech Stack

• Strong programming skills in modern C++ and Python, with experience developing and debugging software on Linux.
• Practical experience with computer vision, deep learning, 3D geometry, point-cloud processing, or robotic perception.
• Experience with one or more perception tasks such as detection, segmentation, classification, tracking, localization, mapping, or sensor fusion.
• Working knowledge of PyTorch or TensorFlow and common perception libraries such as OpenCV, PCL, Eigen, or similar tools.
• Understanding of camera, LiDAR, radar, GNSS, and IMU sensor characteristics, calibration, coordinate systems, and data quality considerations.
• Experience with ROS/ROS 2, DDS, Git, unit testing, continuous integration, and software documentation practices.
• Strong analytical, problem-solving, communication, and teamwork skills, with the ability to work across research and product-development teams.

Qualifications

• Bachelor’s or master’s degree in Computer Science, Computer Engineering, Robotics, Electrical Engineering, Mechatronics, or a related field.

Preferred Qualifications

• Experience deploying perception models on embedded or edge platforms such as NVIDIA Jetson, DRIVE, or comparable hardware.
• Experience with CUDA, TensorRT, model quantization, profiling, and real-time performance optimization.
• Knowledge of SLAM, visual or LiDAR odometry, Kalman filtering, probabilistic estimation, and graph-based spatial data structures.
• Experience developing perception systems for autonomous vehicles, ADAS, mobile robots, mining, construction, or off-highway equipment.
• Experience with simulation, synthetic data, dataset curation, annotation workflows, MLOps, or field validation.
• Publications, patents, or demonstrated implementation of state-of-the-art perception methods are an advantage.