Learning with Noisy Camera Extrinsics for Robust and Real-Time Omnidirectional Depth Prediction
Conner Pulling, RISS 2021 Cohort
Robotics Institute Summer Scholars: https://riss.ri.cmu.edu/
Real-time omnidirectional depth prediction is vital for aerial robotics to accurately perceive their surroundings. Among methods that use multiple fisheye camera images as inputs, recent work using deep learning has shown promising performance, but is not real-time. While recent non-learning methods have achieved real-time performance, all previous
work fundamentally assumes constant camera positions with no calibration errors. This is not always a practical assumption, especially for low-cost drones and when using standard multi-camera calibration tools. This work presents a novel deep learning model capable of real-time, robust omnidirectional depth prediction using multiple fisheye images while maintaining low GPU memory consumption. To achieve robustness
against calibration errors, this work utilizes a novel synthetic dataset that uses noisy camera extrinsics. With a low memory footprint, fast runtime, and robustness against calibration errors, this model architecture takes a step towards enabling omnidirectional depth prediction capabilities for low-cost drone autonomy applications.
Join us next summer! Apply here: riss.ri.cmu.edu
Carnegie Mellon University’s Robotics Institute is committed to opening doors and creating opportunities for future leaders in robotics. Carnegie Mellon University is home to the top-ranked School of Computer Science, the world’s first university robotics department, the world’s first Ph.D. in robotics, and the largest university-affiliated robotics research group. Launched in 2006, CMU’s Robotics Institute Summer Scholars (RISS) program (http://riss.ri.cmu.edu/) comprises a ten-week summer undergraduate research program that immerses a diverse cohort of scholars in cutting-edge robotics and extensive post-program mentoring. The program provides opportunities for students from across the country and world to conduct research with leaders in the field. The program aspires to foster a diverse and inclusive working and learning environment where all students enjoy the educational benefits of diversity and are actively welcomed, included and supported by the community.
QUICK DATES AND INFORMATION:
- Applications are open November 1
- Program runs June 1 through the first week of August
- Apply at https://riss.ri.cmu.edu/apply/
- Scholarships for both U.S. and International Undergraduate Students Available
- Hosted by the Robotics Institute at Carnegie Mellon University, Pittsburgh
SCHOLARSHIPS: The RISS program partners and community provides extensive scholarships for students selected to participate. Scholarships are available to students from across the US and world.
RESEARCH RESULTS:
Explore our research projects and results at: http://riss.ri.cmu.edu/
Videos & Posters at https://riss.ri.cmu.edu/research_show...
Working Papers Journal at https://riss.ri.cmu.edu/research_show...
SCHOLAR EXPERIENCE: Hear from our scholars! https://youtube.com/playlist?list=PL3Oksl-PEd8uBPt4ydkmFxd5O-AagbfZg
- Scholars contribute to robotics research projects through a guided research experience with multiple layers of mentorship
- Scholars learn how to effectively communicate research ideas to various audiences (e.g., sponsors, academic audience, novice audiences) and in various formats (e.g., elevator pitches, short talks, research papers, and poster presentations).
- Scholars forge long-lasting connections to Carnegie Mellon University researchers and partners.
Robotics Workshops & Talks: The technical professional development series exposes scholars to a wide range of robotics applications and projects through weekly robotics talks, visits to labs, and hands- on workshops.
0 Comments