Teaching a store to see. Computer vision and representation learning for autonomous retail — the contribution of IST, IST-ID and ISR to the PT Smart Retail agenda.

Reference implementation of VICAN — Very Efficient Calibration Algorithm for Large Camera Networks. A primal–dual bipartite pose-graph solver that jointly calibrates a marker object and estimates the extrinsic poses of every camera in the network from ArUco marker observations. Includes datasets and Google Colab test scripts.

G. Moreira, M. Marques, J. P. Costeira, A. Hauptmann, “VICAN: Very Efficient Calibration Algorithm for Large Camera Networks”, IEEE ICRA 2024.

Containerized deployment of VICAN. Ships the object-calibration and camera-pose-estimation entry points inside a Docker image, plus helpers for ArUco ID detection and multi-camera configuration generation — turning the calibration algorithm into a dependency-free service that can be provisioned on store hardware in seconds.

The web layer of VisionIST, a fleet of independent Dockerized AI inference services that talk over a shared gRPC protocol with no central orchestrator. Each “box” is a self-contained container; the WebUI exposes them declaratively over HTTP so pipelines can be composed from a browser. Bundled boxes include YOLOv8 (detection and tracking), TAPNext (point tracking), LangSAM (text-guided segmentation), CLIP and Sentence-BERT (multimodal embeddings), VGGT (3D reconstruction), MoGe-3 (monocular depth and normals) and OpenCV primitives.

Head pose estimation robust to occluded faces, together with tools to synthesize realistic occlusions in face datasets (300W-LP, BIWI, AFLW2000). Provides training scripts, inference notebooks, pre-trained models and an evaluation protocol across occlusion levels. Head orientation is a direct proxy for customer attention — which shelf, which product, for how long — in exactly the partially occluded views a store camera network produces.

J. Celestino, M. Marques, J. C. Nascimento, J. P. Costeira, “2D image head pose estimation via latent space regression under occlusion settings”, Pattern Recognition, 137:109288, 2023.

J. Celestino, M. Marques, J. C. Nascimento, “Latent Embedding Clustering for Occlusion Robust Head Pose Estimation”, IEEE Int. Conf. on Automatic Face and Gesture Recognition (FG), 2024.

A synthetic dataset built in Blender for 3D human pose estimation under occlusion: 128 videos, 4 camera views, 3 subjects and 14 actions, with pixel-perfect 2D/3D keypoints, per-keypoint occlusion flags and full camera calibration, in Human3.6M format. Scenarios range from simple environments to a multi-person retail scene with shelves, covering self-, object- and out-of-frame occlusion. Project page: blendmimic3d.github.io.

F. Lino et al., “3D Human Pose Estimation with Occlusions: Introducing BlendMimic3D Dataset and GCN Refinement”, CVPR Workshops 2024.

Identity-preserving face inpainting. A ControlNet-style branch injects identity embeddings from a frozen face recognition encoder (ArcFace) into a pretrained latent diffusion inpainting model, so occluded facial regions are reconstructed as that person rather than drifting to an average face. Only the small control branch is trained; the objective combines reconstruction, identity-consistency and cycle/triplet losses. Evaluated on CelebA-HQ, FFHQ and the new E-Mask dataset of identity-critical occlusions.

J. Santos, C. Santiago, M. Marques, “Face Inpainting with Identity Preserving Latent Diffusion Models”, arXiv:2605.16696, 2026.

PCS-Net, a Siamese CNN that learns the perceived safety of cycling environments directly from street imagery, trained on human pairwise image comparisons (with ties) under a multi-loss framework. It lets planners rank streets and neighbourhoods by how unsafe cyclists feel there, without surveys or in-loco interviews.

M. Costa, M. Marques, C. L. Azevedo, F. W. Siebert, F. Moura, “Which Cycling Environment Appears Safer? Learning Cycling Safety Perceptions From Pairwise Image Comparisons”, IEEE T-ITS, 26(2):1689–1700, 2025.

EG-PCS-Net, a Siamese Vision Transformer that jointly optimizes pairwise safety classification, ranking and attention–gaze alignment: the transformer’s self-attention is supervised with saliency maps derived from human eye-tracking, via a KL-divergence loss, so the model looks where people actually look. Comes with the EG-PCS dataset — 13,623 pairwise comparisons, 2,720 gaze maps, 251 survey participants and 26 laboratory eye-tracking sessions (Zenodo).

L. Perdigão, M. Costa, C. Santiago, M. Marques, “Learning to See Like Humans: Gaze-Aligned Cycling Safety Prediction”, IEEE ITSC 2026.

Code for Hyperbolic vs Euclidean Embeddings in Few-Shot Learning: Two Sides of the Same Coin, the study that settles the geometry everything below is built on. Hierarchical data is widely held to call for hyperbolic embeddings, but their optimization is numerically fragile. Analysing prototypical hyperbolic networks, we show that in high dimensions the embeddings drift to the boundary of the Poincaré ball and that the best few-shot results are all attained at a common hyperbolic radius — so the gain credited to curvature is really a gain from constraining the norm. A fixed-radius encoder with the plain Euclidean metric matches or beats the hyperbolic benchmarks, regardless of embedding dimension. For a catalogue that has to absorb new products from a handful of examples, it means the hierarchy can be carried by a constraint that is cheap and stable to train, rather than by a curved space that is neither.

G. Moreira, M. Marques, J. P. Costeira, A. G. Hauptmann, “Hyperbolic vs Euclidean Embeddings in Few-Shot Learning: Two Sides of the Same Coin”, WACV 2024.

Training code for Learning Visual-Semantic Subspace Representations. A nuclear-norm loss aligns image embeddings with binary label vectors inside a subspace Boolean lattice: each label is a subspace projector, an image lands in the intersection of the subspaces of its concepts, and negation is the orthogonal complement of a subspace. This fixes a known weakness of CLIP-style models, which struggle with negation (“men with a beard but without glasses”). For a product catalogue it means errors stay semantically meaningful — red wine confused with white wine, not with a cardigan.

G. Moreira, M. Marques, J. P. Costeira, A. G. Hauptmann, “Learning Visual-Semantic Subspace Representations”, AISTATS 2025.

Code for Native Hierarchical and Compositional Representations with Subspace Embeddings. Generality is the subspace dimension, hierarchy is subspace inclusion, and logical composition emerges from linear algebra: conjunction as intersection, disjunction as span, negation as orthogonal complement. A differentiable parameterization with soft projection matrices lets the effective dimension of each concept be learned by gradient descent, while staying compatible with optimized Euclidean vector search. Evaluated on WordNet reconstruction and link prediction, HyperLex and SNLI.

G. Moreira, Z. Marinho, M. Marques, J. P. Costeira, C. Xiong, “Native Hierarchical and Compositional Representations with Subspace Embeddings”, ACM SIGKDD (KDD) 2026.

Code for Spectral Guidance for Flexible and Efficient Control of Diffusion Models. As data is corrupted by noise, high-frequency detail disappears while coarse semantics persist; those surviving features are characterized as the leading singular functions of a conditional expectation operator across diffusion timesteps. Learned offline under orthogonality constraints, they form a time-indexed basis onto which arbitrary guidance signals — class labels, CLIP embeddings, segmentation masks — are projected at inference time. No task-specific denoiser retraining, no denoiser gradients during sampling: +37 percentage points in conditional accuracy over the strongest training-free baselines on CIFAR-10, with 4× faster sampling. Useful for controllable synthetic data generation — the same role a simulator plays for calibration and behaviour data.

G. Moreira, M. Marques, J. P. Costeira, C. Xiong, “Spectral Guidance for Flexible and Efficient Control of Diffusion Models”, ICML 2026.