Robotics | Andreas Ziegler

Event-camera, camera and robot arm calibration

Sun, 21 Feb 2021 00:00:00 +0000

Description

The cognitive systems group at the University of Tübingen uses a table tennis robot system to conduct research on various topics around robotics, control, computer vision, machine learning and reinforcement learning. So far the system uses up to five cameras for the perception pipeline. Recently, the group added event-based cameras to the sensor suite.

Event cameras are bio-inspired sensors that differ from conventional frame cameras: Instead of capturing images at a fixed rate, they asynchronously measure per-pixel brightness changes, and output a stream of events that encode the time, location and sign of the brightness changes. Event cameras offer attractive properties compared to traditional cameras: high temporal resolution (in the order of μs), very high dynamic range (140 dB vs. 60 dB), low power consumption, and high pixel bandwidth (on the order of kHz) resulting in reduced motion blur. Hence, event cameras have a large potential for robotics and computer vision.

With this new sensors in place, the questions arises, how the whole system should be calibrated. Eye-to-hand (calibration of the camera and the robot arm) and camera calibration is a well studied topic in the literature. However, since event-based cameras are still relatively new, there is only very little literature on event-based camera calibration, especially for eye-to-hand calibration.

In a first step, the student should study the state-of-the-art calibration methods relevant for our table tennis robot system. Based on this, the goal of the thesis is to develop new methods which can be used in a calibration toolbox, allowing to calibrate the system in an automatic fashion.

Requirements

Familiar with “traditional” Computer Vision
C++ and/or Python

Spiking neural network for event-based ball detection

Sun, 21 Feb 2021 00:00:00 +0000

Description

So far, most learning approaches applied to event data, convert a batch of events into a tensor and then use conventional CNNs as network. While such approaches achieve state-of-the-art performance, they do not make use of the asynchronous nature of the event data. Spiking Neural Networks (SNNs) on the other hand are bio-inspired networks that can process output from event-based directly. SNNs process information conveyed as temporal spikes rather than numeric values. This makes SNNs an ideal counterpart for event-based cameras.

The goal of this thesis is to investigate and evaluate how a SNN can be used together with our event-based cameras to detect and track table tennis balls. The Cognitive Systems groups has a table tennis robot system, where the developed ball tracker can be used and compared to other methods.

Requirements

Familiar with “traditional” Computer Vision
Deep Learning
Python

Exploration Without Global Consistency Using Local Volume Consolidation

Tue, 03 Sep 2019 00:00:00 +0000

A Representation for Exploration that is Robust to State Estimate Drift

Mon, 30 Apr 2018 00:00:00 +0000

Motivation

Exploration is a fundamental task in the field of robotics. The goal is to build a map of a previously unknown environment. Two tasks have to be performed repeatedly for exploration: mapping the space the robot so far perceived; and planning where to go next. In this thesis we focus on the first task and the goal is to find a map representation that can deal with noisy state estimates. All so far presented map representations either assume perfect state estimates or need to rebuild the map after optimization.

Approach

To achieve this goal we build our representation with polygons. Our polygons represent the boundary between free known space and occupied space or unknown space and the inside of polygons is implicitly free space. We develop two approaches: The first approach builds a global map with polygons by continuously building the union of the polygon from the current field of view and the polygons of the so far explored space. The second approach works with polygons of the local field of view only. For every local polygon the frontiers, the obstacles and the free space is determined and the robot explores as long frontiers are present in the map.

Result

In this thesis we propose a novel representation that can deal with noisy state estimates and does not need to rebuild the map or parts of it, e.g. after a loop closure. By experiments we show that we achieve full coverage of the area to explore with frontier-based exploration, using our proposed representation.

Map Fusion for Collaborative UAV SLAM

Thu, 30 Mar 2017 00:00:00 +0000

Motivation

A correct and accurate common map is crucial for multiple robots to collaboratively performing tasks. In this semester project, an existing multi agent Simultaneous Localisation and Mapping (SLAM) system should be extended to fuse maps of single robots in a way that no false map alignment is guaranteed and that an optimal alignment is achieved by using multiple place matches. Also redundant information, a consequence of the fusion of two maps, should be removed in order to get a good performance of the optimization routines e.g. Bundle Adjustment (BA).

Approach

To achieve the first goal, a new approach is proposed which uses multiple KeyFrame Matchs (KFMs) to fuse two maps. This proposed approach also use a novel idea to spread the KFMs over a bigger area by skipping KeyFrames (KFs) between the detection of KFMs. To remove redundant information, KF culling is performed after all KFMs were detected and before the main map fusion. This way information which were present in both maps appear only once in the fused map and the Pose Graph Optimization (PGO) and the BA does not have unnecessary data to process. To reduce the runtime of the optimization part (PGO and BA), the usage of Powell’s dog leg (DL) non-linear least squares technique instead of the Levenberg-Marquardt (LM) optimization was evaluated and the system was adapted in a way, that the performance is increased.

Result

The proposed map fusion approach reduces drift and achieves better accuracy compared to the previous approach. With the implemented KF culling, the number of KFs which the PGO and the BA have to process is decreased significantly and therefore better timing is achieved. The usage of DL non-linear least squares technique reduced the runtime of the optimization furthermore.