Week 4 - Intensity Profiles

Post-processing for 3D applications involves rendering a scene to a texture, referred to as a frame buffer object, and manipulating the data contained in the texture such that it looks different. 

This week I completed 3 post-processing shaders. I completed a shader for controlling the brightness and contrast of an image, a shader for intensity profiles, and a curves adjustment shader. This blog will focus on the on the shader for intensity profiles.

Intensity Profiles/Levels Control

Levels control is a more precise way of controlling brightness and contrast in an image. To control levels the shader requires a minimum input (sometimes referred to as the input black-level), a maximum input (input white-level), a minimum output (output black-level), a maximum output (output white-level) and a value for gamma (middle grey).

Below is the shader code for levels

uniform sampler2D scene;

uniform float minInput;

uniform float maxInput;

uniform float gamma;

uniform float minOutput;

uniform float maxOutput;

void main()

{

        //sample scene texture

vec3 color = texture2D(scene,gl_TexCoord[0].st).rgb;

//levels input range

color = min( max(color - vec3(minInput), vec3(0.0)) / (vec3(maxInput) - vec3(minInput)), vec3(1.0));

//gamma correction

color = pow(color, vec3(1.0 / gamma));

//levels output range

color = mix(vec3(minOutput), vec3(maxOutput), color);

gl_FragColor.rgb = color;

}

The levels input range line essentially rejects RGB values outside of the range of the minimum and maximum input exchanging them for either black or white (black if the color is less than the minimum input specified, and white if it's greater than the maximum input specified). Below is an example substituting the following values:

minInput = 0.2
maxInput = 0.9
gamma =  0.6;
minOutput = 0.2
maxOutput = 0.9
color = (0.1,0.1,0.1)

color = min( max(color - vec3(minInput), vec3(0.0)) / (vec3(maxInput) - vec3(minInput)), vec3(1.0));

color = min( max((0.1,0.1,0.1) - vec3(0.2), vec3(0.0)) / (vec3(0.9) - vec3(0.2)), vec3(1.0));

         = min( max( (-0.1,-0.1,-0.1), (0.0,0.0,0.0) ) / (0.7,0.7,0.7), (1,1,1))

         = min( (0.0,0.0,0.0)/(0.7,0.7,0.7), (1,1,1) )

         = min(  (0.0,0.0,0.0), (1,1,1) )

         = (0,0,0)

As seen above, the value was below the minimum input and so it was rejected and exchanged for black. Had the color been (1.0,0.9,0.9) the color would have been rejected and replaced with white since the color was greater than the maximum input. If the color was within the range the final result of the above equation would simply be the original color with an adjustment. For example an original pixel color of (0.3,0.3,0.3) would return a color of (0.1428,0.1428,0.1428), and a color of (0.8,0.3,0.5) would result in a color of (0.8571, 0.1428,0.4285).

Next is a simple gamma correction. Gamma correction is used to adjust the luminance of pixels such that they are either brighter or darker. In the case of my shader increasing the gamma value will make the image brighter and decreasing the gamma value will make the image darker. Usually the equation is the same except we do not set our exponent to 1.0/gamma, it's usually just gamma. My small issue with this is that increasing the gamma value results in a darker image and decreasing it results in a lighter image, and that just didn't feel natural to me so I reversed it.

The levels output range uses the color value, adjusted from the levels input range and gamma, to linearly interpolate between the minimum and maximum outputs. The GLSL mix function does this for us:

color = mix(vec3(minOutput), vec3(maxOutput), color);

The mix function itself however uses the following logic:

mix(x,y,a) = vec3(dot(x,(1 - a)) + dot(y,a))

If we substitute a value for color of (0.1428,0.1428,0.1428) and use the minimum and maximum outputs stated above:

color = dot((0.2,0.2,0.2),(1,1,1) - (0.1428,0.1428,0.1428) ) +  dot((0.9,0.9,0.9),(0.1428,0.1428,0.1428))
         = dot((0.2,0.2,0.2),(0.8572,0.8572,0.8572)) + 0.38556
         = 0.51432 + 0.38556
         = vec3(0.89988)

Below are screenshots of an image with my shader effect being applied, and one without any effects.

Week 3 - Bloom Shader

This week we talked about the bloom effect.

Bloom is used to produce the real-world effect of bright-light causing a sort of over-exposure.

Looked out the window and God's graphics guy applied a bloom.

The idea is very simple; it is to make everything appear more vibrant and pretty. Luckily the implementation is just as simple as the concept. It only involves 3 steps:

1. Extract highlights from image
2. Blur the extracted highlights
3. Composite the filtered image together

The first step before any of the above, in terms of programming, is to render your scene to a frame buffer object (FBO). The reason we render the scene to an FBO is because bloom is a post-processing effect; it happens after the scene is rendered. In other words, it is not a real-time effect. Once the scene has been rendered to an FBO we can apply our effects.

The first step is a simple pass that basically keeps the brightest colors in the image mainly highlights, leaving the darker colors slightly out of it. This first pass is also rendered to an FBO for further post-processing. The second step is to blur the highlights. This step requires a creating convolution matrix. There are two ways we can do this: using a box filter or a Gaussian filter. In a box filter each source pixel is weighted equally, meaning all the values in the convolution matrix are the same. In a Gaussian the highest weight is in the center of the matrix. Generally the Gaussian filter produces better results due to its smooth distribution.

The final step is to combine the original image with the bright blurred image. Below is the effect I achieved in the shader itself.

Week 2 - Hatching

This week we talked about lighting and we touched on the concept of toon shading.

When I went through some of the shader assignments I saw that one of the toon shader upgrades was to add hatching support. I looked up screenshots from a hatching shader and found several images similar to the one below.

The first step is to create multiple textures of differing density. One texture with very low density, up to one with high density. Next we want to calculate the diffuse (intensity) of the surface based on the direction of the light. Based on the intensity of the light at that point, we can assign 1 scalar, acting as a weight, for each texture. In our fragment shader we sample all the textures at the same specified texture coordinate, each time we sample we multiply by the assigned weight:

vec4 color1 = texture2D(HatchTexture1,texCoord) * weight1;

Once we have sampled all the textures we add all the colors together per fragment. This should give us the line drawing effect we are looking for.

Week 1 - Polish Plans for GDW

Our GDW game in its current state doesn't look as good as it should. After looking at the current state of our game, aesthetically, it needs improving. Due to the cartoon art style our artists already took inspiration from, our game seems perfectly suited for a toon shader.

In our research we looked at different games that apply toon shaders to see the variety of toon shaders currently being used. We looked at games such as Champions Online (screenshots), the Borderlands Series (screenshots), Dragon Ball Z Budokai Series (screenshots), and then we stumbled upon the game Killer is Dead, set to release in summer 2013. The aesthetic of this game is highly stylized and if I had to identify to shaders being used I'd say it was a toon shader and a bloom shader. We hope to achieve a similar effect in our game.

Killer is Dead Debut Gameplay Trailer - Looks awesome.

I took some screenshots of models in maya and tried to replicate the effect in Photoshop. I wasn't able to replicate the bloom as I imagined it in my head, but I was able to get a bit of a toon effect just by applying a simple find edges algorithm. I tried applying bloom by using a soft low opacity brush and going over the brighter areas but it appears to glow rather than bloom. The textures that were applied to the models don't have much detail, and the models have a low-poly count, so when I applied the simple find edges effect in Photoshop and layered it on top of the original screenshot, it already looked toon, the only thing that would need to be applied properly would be a cel shading effect.