Skip to Main content Skip to Navigation
Journal articles

Real-Time Optical Flow Processing on Embedded GPU: anHardware-Aware Algorithm to Implementation Strategy

Abstract : Determining the optical flow of a video is a compute-intensive task essential for computer vision. For achieving this processing in real-time, the whole algorithm deployment chain must be thought of for efficiency first. The development is usually divided into two parts: first, designing an algorithm that meets precision constraints, then, implementing and optimizing its execution on the targeted platform. We argue that unifying those operations enhances performance on the embedded processor. This paper is based on an industrial use case of computer vision. The objective is to determine dense optical flow in real-time on an embedded GPU platform: the Nvidia AGX Xavier. The CLG (Combined Local-Global) optical flow method, initially chosen, is analyzed to understand the convergence speed of its underlying optimization problem. The Jacobi solver is selected for implementation because of its parallel nature. The whole multi-level processing is then ported to the GPU, using several specific optimization strategies. In particular, we analyze the impact of fusing the solver's iterations with the roofline model. As a result, with a 30W power budget, our implementation runs at 60FPS, on 640 × 512 images, with a four-level processing. Hopefully, this example should provide feedback on the issues that arise when trying to port a method to a parallel platform and serve for further implementations of computer vision algorithms on specialized hardware.
Document type :
Journal articles
Complete list of metadata
Contributor : Nicolas Gac Connect in order to contact the contributor
Submitted on : Tuesday, November 30, 2021 - 12:47:31 PM
Last modification on : Tuesday, January 4, 2022 - 6:29:56 AM


Files produced by the author(s)



Mickael Seznec, Nicolas Gac, François Orieux, Alvin Sashala Naik. Real-Time Optical Flow Processing on Embedded GPU: anHardware-Aware Algorithm to Implementation Strategy. Journal of Real-Time Image Processing, Springer Verlag, 2021, ⟨10.1007/s11554-021-01187-8⟩. ⟨hal-03457011⟩



Les métriques sont temporairement indisponibles