Developing a Flipnote-Inspired Shader
For this project, I tried to capture the visual style of Flipnote Studio on the Nintendo DSi and 3DS, translating that unique aesthetic into a real-time 3D environment. The challenge was tackled through HLSL shader programming within Unity.
If you'd like to see the details of my process, feel free to look at my journal for it here!
Approach to Design
The shader design process was a great look through the following techniques:
- Screenspace Dithering: Using Python, I generated a volumetric mapping texture that finds the nearest visually equivalent shade from the 3DS color palette and points to the top left corner of an 8x8 square of that 'dither palette'. Then, in the shader, the pixel coordinates are used to determine its position in the 8x8 cell, leading to a screenspace dithering effect.
- Depth-Normals: This approach involved a combination of two maps: a normal map of the scene and a depth map. They were multiplied together to create a reliable way of outlining close objects to the camera.
- Edge Detection: For this, I applied a Sobel Filter, which involves convolving a kernel on the x and y coordinates of an image. In this case, it was applied to the depth normals.
- Noise-Accented Outline: Applying a pseudo-random number generator to add a 'sketchy' feel to outlines, reminiscent of the Flipnote's casual, low-res feel.
Despite challenges with interfacing with Unity's built-in functions and parsing through dense documentation, with assistance from peers and substantial research, I managed to pull through and learn a ton about programming shaders at the base level!
Visualizations & Previews
Optimization and Future Plans
I identified potential improvements to the system that could significantly enhance performance. For instance, merging the dither palette with the volumetric texture that is sampled for each fragment would reduce the number of sample calls—a significant area for performance gain. I also plan to explore less computationally expensive math calculations for the noise function.
Application and Integration
The shader, packaged as a user-friendly Unity package file, can be easily implemented as a post-processing effect on any camera. You can download it here!
FlipnoteShader.unitypackageInspiration and Learning
This project was an invaluable learning journey. Various super cool online peeps, such as Acerola, Freya Holmér, and Ronja, were amazing introductions into this sort of thing. I found this project to be a wonderful fusion of technical exploration and nostalgic aesthetic. It's been a thrilling process and I'm looking forward to pushing the boundaries of what's possible with shader programming in the future!