 | Intern Alumni Throughout my career as a research scientist I had the honor to work with many brilliant students. Please find a list of my former interns here, and don't hesitate to reach out to me should you want to intern with me in the future. |
 | Lens Blur Bokeh is an artistic tool that allows photographers to guide attention. This typically requires large and often expensive lenses, but we can also just synthesize this effect on imagery with a shallow depth of field by simulating the underlying physics. |
 | Structure Reference One key ingredient for text-to-image applications is controllability, and we have recently announced Structure Reference as a form of user control. It conditions the text-to-image synthesis guided by the estimated depth of a reference image. |
 | Interpolation Benchmark Frame interpolation is becoming a popular research target. However, the current evaluation of interpolation techniques is not ideal. We strongly believe that the community would greatly benefit from a benchmark, which is what we propose. |
 | Adobe Firefly Together with Oliver Wang, Tobias Hinz, Cynthia Liu, and Kevin Matzen, I was one of the founding members of a project that later became known as Adobe Firefly. Specifically, I was in charge of anything data from data curation to data loading. |
 | Computer Vision Reviewer Awards With the top computer vision conferences now mentioning the top reviewers, I was wondering what the statistics for CVPR / ICCV / ECCV look like. These staristics could, for example, be used to find good candidates for an internship. |
 | While deep learning has brought significant improvements to the area of frame interpolation, using a neural network to synthesize an image is prohibitively expensive. In contrast, we propose to synthesize frames purely based on splatting. |
 | Project Inbetween Being able to take pictures at a whim has left me with collections where many photos look alike. Given similar images, could we animate a transition between them to help us cherish past moments? This is what Project Inbetween is about. |
 | Dark Papers One of the few realms that do not have a dark mode yet are academic papers. This is for obvious reasons but I nevertheless wanted to give it a try to see how a dark mode academic paper would look like. Printing not recommended. |
 | Revisiting Adaptive Convolutions for Video Frame Interpolation Simon Niklaus, Long Mai, and Oliver Wang We show, somewhat surprisingly, that it is possible to achieve near state-of-the-art frame interpolation results with an older and simpler approach, namely adaptive separable convolutions, through a subtle set of low level improvements. |
 | Learned Dual-View Reflection Removal S. Niklaus, X. Zhang, J. T. Barron, N. Wadhwa, R. Garg, F. Liu, and T. Xue Typical dereflection algorithms either use a single input image, which suffers from intrinsic ambiguities, or use multiple images from a moving camera, which is inconvenient. We instead propose an algorithm that uses stereo images as input. |
 | Backward warping has seen broad adoption in tasks like optical flow estimation. In contrast, forward warping has seen less attention since it is unclear how to do it well. It can play a key role though, which is why we propose softmax splatting. |
 | The popular Ken Burns effect is an effective tool for animating still images by zooming into them. We extend this effect into the third dimension and propose a framework that, given just a single image, creates an immersive 3D zoom effect. |
 | Moving Stills We all take pictures to capture events that are dear to our heart. And while taking pictures is fun, viewing them could be more exciting at times. We breath new live into these pictures, ultimately helping us cherish and celebrate past moments. |
 | A popular framework for novel view synthesis is to warp pixels and then refine the outcome using a neural network. But why warp only in color space? We show that also warping the input in feature space helps with and improves the refinement. |
 | Visual Computing A class that I taught at Portland State University when I was a Ph.D. student. It covered the basics of computer vision and deep learning with a heavy focus on implementing the subject matter in Python using OpenCV and PyTorch. |
 | WASM Raytracer How fast is WebAssembly compared to vanilla JavaScript? I wanted to find out and implemented a simplistic raytracer with the same logic in JavaScript as well as C++ and used Emscripten to compile the C++ version to WebAssembly. |
 | A recent approach to perform video frame interpolation convolves input frames with spatially-varying filter kernels. We show that these kernels are separable, which improves computational efficiency and interpolation quality. |
 | Web Development A class that I established at Portland State University when I was a Ph.D. student. I wanted students to obtain some practical web dev skills, and this class ended up being fully booked every time within minutes of the registration opening up. |
 | Frame interpolation is typically done in two steps, motion estimation and compensation. However, this heavily depends on the quality of the motion estimation. We instead propose a single-step approach through spatially-varying convolutions. |
 | Rotating Pipes A classic puzzle game where you have to rotate pipes such that they all connect with each other. It uses minimum spanning trees to generate the puzzles, which is implemented in C++ and run in the browser through WebAssembly. |
 | Rollercoaster Map I consider Rollercoaster Tycoon as one of my guilty pleasures of past times. While my parks are nothing noteworthy, there are some talented people out there, I still thought it would be fun to visualize and share them as a virtual map. |
 | Connect Four A connect four game, directly playable in the browser against a computer opponent with a configurable difficulty. The computer is implemented in C++ using negamax search with alpha-beta pruning and executed through WebAssembly. |
 | Mini Chess A miniature version of chess, playable directly in the browser against a computer opponent on different chess boards. The computer has a configurable difficulty and is implemented using negamax search with alpha-beta pruning. |
 | Artificial Intelligence A class that I taught when I was a Ph.D. student at Portland State on behalf of my dear friend Bart Massey. It covered the theory and practice of finding optimal and satisfying solutions to one-player and two-player combinatorial games. |
 | Youtube Watchmarker Youtube keeps track of your watch history and automatically marks seen videos. However, they only mark recently watched videos, which is kind of disappointing. This addon fixes this issue and keeps track of your entire watch history. |