 | Image Denoising We have recently introduced a new denosier in Lightroom that was pioneered by Michael Gharbi before he left Adobe to found a startup. Since then, Zhihao Xia and I have been the research leads behind this technology. |
 | 3D Mockup A common task for creative professionals is to mockup a design onto an object. This can challenging though, imagine putting a logo onto a t-shirt. Fortunately, we can analyze the 3D structure of the target image and automate this process. |
 | Sky Segmentation Sky segmentation is a core feature in many creative tools, for example to adjust the saturation of the sky separately from the rest. Shipping a sky segmentor across multiple platforms, from mobile to the desktop, is pretty tricky though. |
 | Firefly Video After being one of the founders and key figures behind Adobe Firefly, I have been busy working on a foundational video model in a team led by Jianming Zhang. Like before, I have been in charge of the image data pipeline. |
 | Intern Alumni Throughout my career as a research scientist I had the honor to work with many brilliant students. Please find a list of my former interns here, and don't hesitate to reach out to me should you want to intern with me in the future. |
 | Lens Blur Bokeh is an artistic tool that allows photographers to guide attention. This typically requires large and often expensive lenses, but we can also just synthesize this effect on imagery with a shallow depth of field by simulating the underlying physics. |
 | Structure Reference One key ingredient for text-to-image applications is controllability, and we have recently announced Structure Reference as a form of user control. It conditions the text-to-image synthesis guided by the estimated depth of a reference image. |
 | Benchmarking Video Frame Interpolation Simon Kiefhaber, Simon Niklaus, Feng Liu, and Simone Schaub-Meyer Frame interpolation is becoming a popular research target. However, the current evaluation of interpolation techniques is not ideal. We strongly believe that the community would greatly benefit from a benchmark, which is what we propose. |
 | Adobe Firefly Together with Oliver Wang, Tobias Hinz, Cynthia Liu, and Kevin Matzen, I was one of the founding members of a project that later became known as Adobe Firefly. Specifically, I was in charge of anything data from data curation to data loading. |
 | Computer Vision Reviewer Awards With the top computer vision conferences now mentioning the top reviewers, I was wondering what the statistics for CVPR / ICCV / ECCV look like. These staristics could, for example, be used to find good candidates for an internship. |
 | While deep learning has brought significant improvements to the area of frame interpolation, using a neural network to synthesize an image is prohibitively expensive. In contrast, we propose to synthesize frames purely based on splatting. |
 | Dark Papers One of the few realms that do not have a dark mode yet are academic papers. This is for obvious reasons but I nevertheless wanted to give it a try to see how a dark mode academic paper would look like. Printing not recommended. |
 | Revisiting Adaptive Convolutions for Video Frame Interpolation Simon Niklaus, Long Mai, and Oliver Wang We show, somewhat surprisingly, that it is possible to achieve near state-of-the-art frame interpolation results with an older and simpler approach, namely adaptive separable convolutions, through a subtle set of low level improvements. |
 | Learned Dual-View Reflection Removal S. Niklaus, X. Zhang, J. T. Barron, N. Wadhwa, R. Garg, F. Liu, and T. Xue Typical dereflection algorithms either use a single input image, which suffers from intrinsic ambiguities, or use multiple images from a moving camera, which is inconvenient. We instead propose an algorithm that uses stereo images as input. |
 | Backward warping has seen broad adoption in tasks like optical flow estimation. In contrast, forward warping has seen less attention since it is unclear how to do it well. It can play a key role though, which is why we propose softmax splatting. |
 | The popular Ken Burns effect is an effective tool for animating still images by zooming into them. We extend this effect into the third dimension and propose a framework that, given just a single image, creates an immersive 3D zoom effect. |
 | A popular framework for novel view synthesis is to warp pixels and then refine the outcome using a neural network. But why warp only in color space? We show that also warping the input in feature space helps with and improves the refinement. |
 | Visual Computing A class that I taught at Portland State University when I was a Ph.D. student. It covered the basics of computer vision and deep learning with a heavy focus on implementing the subject matter in Python using OpenCV and PyTorch. |
 | WASM Raytracer How fast is WebAssembly compared to vanilla JavaScript? I wanted to find out and implemented a simplistic raytracer with the same logic in JavaScript as well as C++ and used Emscripten to compile the C++ version to WebAssembly. |
 | A recent approach to perform video frame interpolation convolves input frames with spatially-varying filter kernels. We show that these kernels are separable, which improves computational efficiency and interpolation quality. |
 | Web Development A class that I established at Portland State University when I was a Ph.D. student. I wanted students to obtain some practical web dev skills, and this class ended up being fully booked every time within minutes of the registration opening up. |
 | Frame interpolation is typically done in two steps, motion estimation and compensation. However, this heavily depends on the quality of the motion estimation. We instead propose a single-step approach through spatially-varying convolutions. |
 | Rotating Pipes A classic puzzle game where you have to rotate pipes such that they all connect with each other. It uses minimum spanning trees to generate the puzzles, which is implemented in C++ and run in the browser through WebAssembly. |
 | Rollercoaster Map I consider Rollercoaster Tycoon as one of my guilty pleasures of past times. While my parks are nothing noteworthy, there are some talented people out there, I still thought it would be fun to visualize and share them as a virtual map. |
 | Connect Four A connect four game, directly playable in the browser against a computer opponent with a configurable difficulty. The computer is implemented in C++ using negamax search with alpha-beta pruning and executed through WebAssembly. |
 | Mini Chess A miniature version of chess, playable directly in the browser against a computer opponent on different chess boards. The computer has a configurable difficulty and is implemented using negamax search with alpha-beta pruning. |
 | Artificial Intelligence A class that I taught when I was a Ph.D. student at Portland State on behalf of my dear friend Bart Massey. It covered the theory and practice of finding optimal and satisfying solutions to one-player and two-player combinatorial games. |
 | Youtube Watchmarker Youtube keeps track of your watch history and automatically marks seen videos. However, they only mark recently watched videos, which is kind of disappointing. This addon fixes this issue and keeps track of your entire watch history. |