Gravitational lensing in Video Games

While developing the concept for Star Impact, I had a deep urge to use gravitational lensing in the game. If the rare material Red Matter can warp the fabric of space-time to create the first interstellar spaceship, then player should be able to see it. i.e. gravitational lensing: "ooooo ahhh, cool."

Gravitational lensing is really just a simple distortion effect, and distortion effects are not unheard of in games, however they tend to be one-time post processing magic affecting the entire screen. Perhaps the the player gets hit, or it is a water level (gross). Red Matter is different, it can have multiple Red Matter collectibles on the screen at once. And, gravitational lensing needs to occur at every collectible on screen. Potentially, that is 20 collectibles, 20 intense calculations, and 20 passes on a post processing shader attempting to run 60 fps on mobile. A recipe for failure.

I needed a way to resolve this, and I will admit, I was stumped at first. But with time, anything can be solved, and the answer soon became clear. In games, we have light maps: a pre-rendered texture that describes how lighting should be applied a scene or object. This speeds up things, because the lighting at a location can now be calculated with only a simple lookup in your lighting texture. Maybe I could create a... wait for it, a distortion map...

This is a Distortion Map. Unreadable to humans but food for my shader voodoo magic!

Yes, a distortion map. That is, a texture that describes how pixels should be distorted (moved) on the screen. If it works, I should be able to render these distortion textures all over the screen and write some shader voodoo magic to interpret these distortion pixels and apply them in a one pass post-processing shader. Yes, the insanity is real and it works. *pats self on the back*

The encoding of the distortion map is simple, essentially it is uses RGB values to determine how many pixels a color should be shifted on the x and y axis. This can be optimized into more efficient encoding, but the principal stands: Pre-render distortion equations to distortion maps, draw distortion maps to a render texture at runtime, and use a single pass post-processing shader to bring it to life.

As a developer, and more importantly a problem solver, I find problems likes these a great part of making games. They start with a simple desire or dream, and although they may cause much initial pain and confusion, the persistence and creativity of the developer can lead to innovative and unexpected solutions. Plus gravitational lensing: "ooooo ahhh, cool."

See video below to see the distortion in motion, beautiful.