2026 - Master Thesis - Space

Topics

• EU Open Research Repository, AiiDA.net
• CERN, PSI
• 2025 - 📍 ZapBench, PathFinder
• 2026 - How the brain’s wiring changes
• 2026 - SpaceX

Coding

• 2026 - Let your training 8hrs -> 📍 13mins

References

• CVPR

• Earthdata Plugin

• If a team / mentor can tolerate you saying "This has no information" and listen carefully to the rest of your sentence, then it is a very good peer / team.

• DiffusionDrive, CVPR highlight 2025.
• 📍 Disentangling Monocular 3D Object Detection, ICCV 2019.
  • The core method of 3D perception that does not rely on LiDAR laid the foundation for many subsequent 3D Tracking and 3D MOT vision methods.
• Monocular 3D Object Detection Leveraging Accurate Proposals and Shape Reconstruction, CVPR 2019.
• Monocular Quasi-Dense 3D Object Tracking, 2021.
• Multi-Level Fusion based 3D Object Detection from Monocular Images, CVPR 2018.
• Development of the Nervous System, Prof. Dr. Stoeckli Esther

Multi-sensor Input Fusion From Space, Safety Detection

• 1960 - A New Approach to Linear Filtering and Prediction Problems, Kalman
• 2005 - Probabilistic Robotics, Multi-sensor Input Fusion
• 2025 - ACDC Dataset, training and testing semantic perception on adverse visual conditions
• 📍 2019 - Calibration Wizard: A Guidance System for Camera Calibration Based on Modelling Geometric and Corner Uncertainty

Topics

0. Sensor Modalities and Data Types

1. Maritime Search and Rescue

Optical satellite images
+ SAR satellite images
→ Ship Detection
→ Ship Re-Identification (ReID)
→ Trajectory generation & route prediction

2. Input Data Type

3. Fusion Space

Optical image ─┐
               ├─ Dual-head tokenizer → Shared Transformer Encoder → Unified embedding
SAR image     ─┘

4. Output Data

A Dynamic Camera with Multi-modal Input Signal Fusion

          Human perception
        ┌──────────────────┐
        │  Vestibular      │
        │  Vision          │
        └──────────────────┘
                 ▲
                 │
    ┌────────────┴────────────────┐
    │ Wearable System Estimation  │
    └────────────┬────────────────┘
                 │
 ┌───────┬────────┬────────┬────────┬────────┐
 │ Camera│  IMU   │ Eye    │ Depth  │ Others │
 │       │        │ tracker│ / ToF  │        │
 └───────┴────────┴────────┴────────┴────────┘

Core Formulation: Bayesian Multi-Modal Sensor Fusion

Latent State Definition

• At time step t, the latent state is defined as: $x_t = { T_t, \theta, \psi_t }$

• where $T_t$ denotes the device pose, $\theta$ represents the calibration parameters shared across time, and $\psi_t$ denotes user-centric latent variables.

Multi-Modal Observations

• Given heterogeneous sensor measurements at time t: $z_t = { z_t^{cam}, z_t^{imu}, z_t^{eye} }$

• where observations are obtained from the camera, IMU, and eye-tracking modalities.

Bayesian Fusion Objective

• Multi-modal fusion is defined as inference over the joint posterior: $p(x_{1:T} \mid z_{1:T})$

• Using the Markov assumption and conditional independence of observations, the posterior factorizes as: $p(x_{1:T} \mid z_{1:T}) \propto \prod_{t=1}^{T} p(z_t \mid x_t)\, p(x_t \mid x_{t-1})$

Multi-Modal Likelihood Factorization

• Assuming conditional independence between sensor modalities given the latent state: $p(z_t \mid x_t) = p(z_t^{cam} \mid x_t)\, p(z_t^{imu} \mid x_t)\, p(z_t^{eye} \mid x_t)$

State Transition Model

• The temporal evolution of the latent state is modeled as: $p(x_t \mid x_{t-1}) = p(T_t \mid T_{t-1})\, p(\psi_t \mid \psi_{t-1})\, p(\theta)$

• where $\theta$ is treated as a time-invariant latent variable, $p(\theta)$ enforces temporal consistency of calibration parameters.

Interpretation

• Fusion thus corresponds to Bayesian state estimation under uncertainty, where heterogeneous sensor observations impose probabilistic constraints on a shared latent state evolving over time. Calibration parameters are inferred jointly with pose and user states, enabling online self-calibration.

Sensor Models

• $z_t^{imu} = h_{imu}(T_{t-1}, T_t) + \epsilon_{imu}$
• $z_t^{cam} = h_{cam}(T_t, \theta) + \epsilon_{cam}$
• $z_t^{eye} = h_{eye}(T_t, \psi_t) + \epsilon_{eye}$

Filtering Approximation

For online inference, we approximate the posterior using Bayesian filtering.

• Prediction: $p(x_t \mid z_{1:t-1}) = \int p(x_t \mid x_{t-1}) p(x_{t-1} \mid z_{1:t-1}) dx_{t-1}$
• Update: $p(x_t \mid z_{1:t}) \propto p(z_t \mid x_t) p(x_t \mid z_{1:t-1})$

Practical Filtering Choices under Self-Calibrated Camera Constraints

Method Selection Is Constraint-Driven, Not Aesthetic

Why Gaussian

• For Closure Under Bayesian Operations

• Bayesian filtering requires two fundamental operations that are applied recursively over time.

• Prediction
• The prediction step propagates the belief forward in time using the system dynamics:

$p(x_t \mid z_{1:t-1}) = \int p(x_t \mid x_{t-1}) p(x_{t-1} \mid z_{1:t-1}) dx_{t-1}$

• Update
• The update step incorporates the new observation into the predicted belief:

$p(x_t \mid z_{1:t}) \propto p(z_t \mid x_t)\, p(x_t \mid z_{1:t-1})$

• Gaussian distributions possess a crucial closure property under these Bayesian operations:
  • The product of two Gaussian distributions is Gaussian.
  • The marginalization of a joint Gaussian distribution is Gaussian.
• As a consequence:
  • The prediction step preserves Gaussianity.
  • The update step preserves Gaussianity.
• Without this closure property, the posterior distribution does not remain in a tractable functional family, and Bayesian filtering becomes analytically intractable.

Bayesian Filter、Kalman Filter, Gaussian Distribution

┌───────────────────────────────────────────────┐
│               Bayesian Filtering              │
│                                               │
│  p(x_t | z_{1:t}) ∝ p(z_t | x_t) p(x_t | z_{1:t-1}) │
│                                               │
│  • General probabilistic inference framework  │
│  • Arbitrary distributions                    │
│  • Arbitrary nonlinear dynamics               │
│  • Arbitrary observation models               │
│                                               │
│        (Intractable in general)               │
└───────────────────────┬───────────────────────┘
                        │
                        │  Gaussian assumption
                        ▼
┌───────────────────────────────────────────────┐
│            Kalman-style Filtering             │
│                                               │
│  Assumption:                                  │
│  p(x_t | z_{1:t}) ≈ 𝒩(μ_t, Σ_t)              │
│                                               │
│  • Posterior represented only by mean + cov   │
│  • Recursive closed-form updates              │
│  • Efficient and online                       │
│                                               │
│  Includes:                                    │
│   - Kalman Filter (linear)                    │
│   - EKF (local linearization)                 │
│   - UKF (sigma-point)                         │
└───────────────────────┬───────────────────────┘
                        │
                        │  Linear model + Gaussian noise
                        ▼
┌───────────────────────────────────────────────┐
│               Kalman Filter                   │
│                                               │
│  x_t = A x_{t-1} + w_t ,   w_t ~ 𝒩(0, Q)     │
│  z_t = H x_t     + v_t ,   v_t ~ 𝒩(0, R)     │
│                                               │
│  • Exact Bayesian inference                   │
│  • Optimal under linear-Gaussian assumptions  │
└───────────────────────────────────────────────┘

For Each Time Step

Time t-1 belief                    Prediction                    Update
(posterior at t-1)                 (motion model)               (sensor fusion)

   p(x_{t-1}|z_{1:t-1})             p(x_t|z_{1:t-1})             p(x_t|z_{1:t})
   ~ 𝒩(μ_{t-1}, Σ_{t-1})     →       ~ 𝒩(μ_t^-, Σ_t^-)     →       ~ 𝒩(μ_t, Σ_t)
                │                              │                              │
                │                              │                              │
                ▼                              ▼                              ▼
        Gaussian belief              Gaussian prediction            Gaussian posterior

Multiple sensors = multiple Gaussian constraints on the same state

                    z_t^cam
                 (camera likelihood)
                        │
                        ▼
                   ┌────────┐
z_t^imu ───────▶   │  x_t   │   ◀────── z_t^eye
(IMU likelihood)   │ latent │   (eye-tracking likelihood)
                   │ state  │
                   └────────┘

Best Normalization

Brain Signals (Why Median + MAD)

A. Unresolved Core Problems in Modern CVPR (Post-Deep Learning Era)

B. Key Meta-Observation (Critical)

Status Overview

What Is Fundamentally Missing

How ML Makes Camera Errors More Dangerous

Camera as the Global Reference Frame in Vision Systems

Camera Calibration Core Definition