11.11 - Combining Surface Properties¶

This has been a long chapter and perhaps you have forgot what it is even about! So let’s review for a moment.

A 3D model is represented by a triangular mesh. To simulate real-world objects the surface properties of the triangles must be modeled. In this chapter we have discussed how to model surface properties using the following techniques:

Modeling the diffuse, specular, and ambient color of a surface.
Simulating a curved surface by varying the normal vectors of vertices (smooth shading).
Mapping colors from an image onto a surface (image texture mapping).
Mapping colors from a procedural calculation onto a surface (procedural texture mapping).
Transformations on texture coordinates to modify (or animate) a surface pattern.
Modifying the actual geometry of a surface using heightmaps or displacement maps.
Modifying how light reflects from a surface using bump maps or normal maps.

We have examined each of these techniques in isolation, but the rendering of a model is typically performed by a combination of many (or all) of these techniques. This lesson discusses implementation issues related to combining these techniques in shader programs.

Software Development Issues¶

Good software development habits are invaluable when developing complex shader programs. Make it your habit to do the following:

Procedural Decomposition - break a problem down into simpler sub-problems. Implement each sub-problem as a distinct procedure.
Incremental Development - develop and debug distinct procedures independent from the overall problem.
Systematic Integration - create a solution to the overall problem by systematically adding in pieces to the overall solution.
Data vs. Algorithm - a rendering can be incorrect due to bad data, a bad algorithm, or the incorrect coding of an algorithm. Always make sure you have good data before attempting to debug your algorithms.

The output of a shader program is a 2D image. As you are testing and debugging a shader program put the values you are trying to debug into the gl_FragColor output variable. Specific examples of how to do this are described below.

Tasks for Creating a WebGL program¶

The following WebGL program renders a “bunny” by combining smooth shading, image texture mapping, bump maps, and an ambient, diffuse, and specular lighting model. Before experimenting with the demo program, please review the following tasks that were required to create this program:

The model of a “bunny” was downloaded from the internet: http://graphics.stanford.edu/data/3Dscanrep/
Using Blender:
1. The model was defined in a .ply data file. It was imported into Blender and exported as an .obj data file.
2. The original polygonal mesh had 35,947 vertices and 69,451 triangles. To reduce the size of the mesh, the X (delete memu) –> Limited dissolve command was used. This created multi-sided polygons that were changed into triangles using CRTL f (face menu) –> Triangulate Faces. The model was reduced to 22,271 vertices and 42,128 triangles. The CRTL v (vertices) –> Smooth Vertex tool was also used to smooth out the location of the vertices.
3. Triangular mesh errors were fixed. For example, some of the triangles had their normal vectors pointing inward (instead of outward). In addition, every face was given the “smooth face” property to enable smooth shading.
4. A material was created and assigned to all of the model faces.
5. A texture map was created to color all the faces: water.png
6. A texture map was created to bump map the faces: leather-bump-map.jpg
7. Texture coordinates were created for all the faces using the “unwrap” command. However, they were very poor texture coordinates.
8. The original model had 4 “holes” in the bottom of the model which made the assignment of texture coordinates problematic. Most of the triangles on the bottom of the model were deleted to create one large hole. The “unwrap” command then produced more reasonable texture coordinates.
9. The model was exported to an .obj file with only vertices and texture coordinates. This makes the file as small as possible. (The function CreateModelsFromOBJ() will calculate normal vectors after the file has been downloaded from a server.)
10. The triangles that formed the bunny’s eyes were manipulated to create a cleaner, more circular model for each eye.
11. The texture coordinates for the triangles that form the bunny’s eyes were move to the upper-right corner of the texture map image. Gimp was used to color a black rectangle in this part of the texture map image. Therefore, the bunny’s eyes are colored black.
12. The assignment of texture coordinates took the most work. The image below shows the layout of the texture coordinates for the entire model.
  
  All texture coordinates
  
  In the image below, only the texture coordinates for the head and ears are shown highlighted in orange. The head and ears received very little “texturing” while most the pattern in the water.png image is being applied to the body of the bunny. This makes for a nice rendering, but an infinite number of other texture coordinates assignments are possible.
  
  Only the head texture coordinates
13. The “unwrap” command created texture coordinates for parts of the bunny’s ears that were so condensed they were equal to each other. This caused the local coordinate system of these triangles to calculate zero length vectors, which caused the bump map calculations to produce an invalid normal vector, which created a black color for these triangles. In Blender the texture coordinates for these triangles were moved to a different part of the bump map image and scaled to larger values, but still values that mapped to a single pixel. This fixed the problem with the bump map calculations and kept the ears predominately the same color.
Gimp was used to resize the the bump map image to a power of 2 and the Colors –> Brightness-Contrast tool was used to darken the image, which made the “bumps” smaller. The original image and its final version are shown below.

Original image Final “bump map” image
The shader program was created in several stages, adding successive layers of complexity one at a time.

This brief overview of the work required to create the WebGL program below is typical of the issues you will face when you create new WebGL programs. Your tasks and issues may vary widely based on your models and your desired final output.

A WebGL Program¶

Experiment with the following program and study its shader program. The bump map used is very subtle. Please manipulate the bump map transformation parameters to visually see the “bumps”. After investigating the program, please perform the experiments listed below the demo program.

Show: Code Canvas Run Info

bunny.vert
bunny.frag

// Vertex Shader
precision mediump int;
precision mediump float;

// Scene transformations
uniform mat4 u_To_clipping_space; // Projection, camera, model transform
uniform mat4 u_To_camera_space;   // Camera, model transform

// Light model
uniform vec3  u_Light_position;
uniform vec3  u_Light_color;
uniform float u_Shininess;
uniform vec3  u_Ambient_intensities;

// The texture units
uniform sampler2D u_Color_map_unit;
uniform sampler2D u_Bump_map_unit;

// Original model data
attribute vec3 a_Vertex;
attribute vec3 a_Color;
attribute vec3 a_Normal;
attribute vec2 a_Texture_coordinate;

// Bump map data to calculate a triangle's local coordinate system
attribute vec3 a_P2;
attribute vec3 a_P3;
attribute vec2 a_Uv2;
attribute vec2 a_Uv3;

// Data (to be interpolated) that is passed on to the fragment shader
varying vec3 v_Vertex;
varying vec4 v_Color;
varying vec3 v_Normal;

// Data (to be interpolated) that is passed on to the fragment shader
varying vec2 v_Texture_coordinate;

// The U, V texture coordinate axes in 3D space;
// That is, a local coordinate system for a triangle
varying vec3 v_U3d;
varying vec3 v_V3d;

//-------------------------------------------------------------------------
// Calculate the local coordinate system for this vertex's triangle
void calulate_triangle_coordinate_system(vec3 p1,  vec3 p2,  vec3 p3,
                                         vec2 uv1, vec2 uv2, vec2 uv3) {
  float u1 = uv1[0];
  float v1 = uv1[1];
  float u2 = uv2[0];
  float v2 = uv2[1];
  float u3 = uv3[0];
  float v3 = uv3[1];

float divisor = (u3-u1)*(v2-v1) - (u2-u1)*(v3-v1);

v_U3d = ((v2-v1)*(p3-p1) - (v3-v1)*(p2-p1)) /  divisor;
  v_V3d = ((u2-u1)*(p3-p1) - (u3-u1)*(p2-p1)) / -divisor;

v_U3d = normalize(v_U3d);
  v_V3d = normalize(v_V3d);
}

//-------------------------------------------------------------------------
void main() {

calulate_triangle_coordinate_system(a_Vertex, a_P2, a_P3,
                                      a_Texture_coordinate, a_Uv2, a_Uv3);

// Put the vertex's location into camera space.
  v_Vertex = vec3( u_To_camera_space * vec4(a_Vertex, 1.0) );

// Put the vertex's normal vector into camera space.
  v_Normal = vec3( u_To_camera_space * vec4(a_Normal, 0.0) );

// Put triangle's local coordinate system into camera space
  v_U3d = vec3( u_To_camera_space * vec4(v_U3d, 0.0) );
  v_V3d = vec3( u_To_camera_space * vec4(v_V3d, 0.0) );

// Pass the vertex's color to the fragment shader.
  v_Color = vec4(a_Color, 1.0);

// Pass the vertex's texture coordinate to the fragment shader.
  v_Texture_coordinate = a_Texture_coordinate;

// Transform the location of the vertex for the graphics pipeline.
  gl_Position = u_To_clipping_space * vec4(a_Vertex, 1.0);
}

// Fragment shader program
precision mediump int;
precision mediump float;

// The texture unit to use for the color lookup
uniform sampler2D u_Color_map_unit;
uniform sampler2D u_Bump_map_unit;

uniform mat3 u_Transform_color_tex_coords;

// The dimensions of the bump map image: (width, height)
uniform vec2 u_Image_size;

// Fragment data
varying vec3 v_Vertex;
varying vec4 v_Color;
varying vec3 v_Normal;

// Bump map data for the fragment shader to calculate a triangle's
// local coordinate system
varying vec3 v_U3d, v_V3d;

// Data coming from the vertex shader
varying vec2 v_Texture_coordinate;

// Lighting model
uniform vec3  u_Light_position;
uniform vec3  u_Light_color;
uniform vec3  u_Ambient_intensities;

// Model surfaces' shininess
uniform float u_Shininess;

//-------------------------------------------------------------------------
// Given texture coordinates, calculate the "finite difference"
// between neighboring pixels.
vec2 get_normal_offsets(vec2 texture_coordinates) {
  float pixel_delta_u = 1.0 / u_Image_size[0];
  float pixel_delta_v = 1.0 / u_Image_size[1];

vec2 up    = vec2(0.0,  pixel_delta_v);
  vec2 down  = vec2(0.0, -pixel_delta_v);
  vec2 left  = vec2(-pixel_delta_u, 0.0);
  vec2 right = vec2( pixel_delta_u, 0.0);

vec4 right_color = texture2D(u_Bump_map_unit, texture_coordinates + right);
  vec4 left_color  = texture2D(u_Bump_map_unit, texture_coordinates + left);
  vec4 up_color    = texture2D(u_Bump_map_unit, texture_coordinates + up);
  vec4 down_color  = texture2D(u_Bump_map_unit, texture_coordinates + down);

return vec2(right_color[0] - left_color[0], up_color[0] - down_color[0]);
}

//-------------------------------------------------------------------------
// Given a normal vector and texture coordinates, "bump" the
// normal vector based on the offsets from the bump texture map.
vec3 bump_map_normal(vec3 normal, vec2 texture_coordinates) {
  vec2 offsets = get_normal_offsets(texture_coordinates);

if (length(normal) == 0.0) {
    // If the normal vector has gone to zero length, recalculate it.
    normal = normalize(cross(v_U3d, v_V3d));
  }
  normal = normal + offsets[0] * v_U3d + offsets[1] * v_V3d;
  return normalize(normal);
}

//-------------------------------------------------------------------------
// Given a normal vector and a light,
// calculate the fragment's color using diffuse and specular lighting.
vec3 light_calculations(vec3 fragment_color,
                        vec3 fragment_normal,
                        vec3 light_position,
                        vec3 light_color) {

vec3 specular_color;
  vec3 diffuse_color;
  vec3 to_light;
  vec3 reflection;
  vec3 to_camera;
  float cos_angle;
  float attenuation;
  vec3 color;

//- - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -
  // General calculations needed for both specular and diffuse lighting

// Calculate a vector from the fragment location to the light source
  to_light = light_position - v_Vertex;
  to_light = normalize( to_light );

//- - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -
  // DIFFUSE  calculations

// Calculate the cosine of the angle between the vertex's normal
  // vector and the vector going to the light.
  cos_angle = dot(fragment_normal, to_light);
  cos_angle = clamp(cos_angle, 0.0, 1.0);

// Scale the color of this fragment based on its angle to the light.
  diffuse_color = vec3(fragment_color) * light_color * cos_angle;

//- - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -
  // SPECULAR  calculations

// Calculate the reflection vector
  reflection = 2.0 * dot(fragment_normal,to_light) * fragment_normal - to_light;
  reflection = normalize( reflection );

// Calculate a vector from the fragment location to the camera.
  // The camera is at the origin, so just negate the fragment location
  to_camera = -1.0 * v_Vertex;
  to_camera = normalize( to_camera );

// Calculate the cosine of the angle between the reflection vector
  // and the vector going to the camera.
  cos_angle = dot(reflection, to_camera);
  cos_angle = clamp(cos_angle, 0.0, 1.0);
  cos_angle = pow(cos_angle, u_Shininess);

// If this fragment gets a specular reflection, use the light's color,
  // otherwise use the objects's color
  specular_color = light_color * cos_angle;

color = diffuse_color + specular_color;
  color = clamp(color, 0.0, 1.0);

return color;
}

//-------------------------------------------------------------------------
void main() {

vec3 color, normal;

// Texture mapping to get fragment color.
  vec4 frag_color = texture2D(u_Color_map_unit, v_Texture_coordinate);

// Transform the texture coordinates for the bump map calculations.
  vec2 bump_tex_coords = vec2(u_Transform_color_tex_coords * vec3(v_Texture_coordinate, 1.0));

// Modify the fragment normal vector using the bump map image data.
  normal = bump_map_normal(normalize(v_Normal), bump_tex_coords);

// Perform the lighting calculations.
  color = light_calculations(vec3(frag_color), normal, u_Light_position, u_Light_color);

// Add in the ambient light
  color = color + u_Ambient_intensities * vec3(frag_color);

gl_FragColor = vec4(color, 1.0);
}

Experiment with Multiple Surface Properties

Virtual World A Bunny

camera eye (0.0, 0.0, 5.0)	camera center (0.0, 0.5, 0.0)
X: -5.0 +5.0	X: -5.0 +5.0
Y: -5.0 +5.0	Y: -5.0 +5.0
Z: -5.0 +5.0	Z: -5.0 +5.0

light position(3.0, 0.0, 5.0)	light color (1.00, 1.00, 1.00)
X: -5.0 +5.0	Red:	0.0 1.0
Y: -5.0 +5.0	Green:	0.0 1.0
Z: -5.0 +5.0	Blue:	0.0 1.0

ambient intensities (0.30, 0.30, 0.30)
Red:	0.0 1.0
Green:	0.0 1.0
Blue:	0.0 1.0
Change all intensities at once.

shininess = 30.0
0.1 128.0

tx = 0.00	-4.0 4.0
ty = 0.00	-4.0 4.0
sx = 0.00	-4.0 4.0
sy = 0.00	-4.0 4.0
Scale both axes
angle = 0	-360.0 +360.0

Show: Process information Warnings Errors

Open this webgl program in a new tab or window

Debugging Shader Programs¶

Please make the following changes to the shader program above and study the results. These exercises will help you understand how to debug shader programs.

Look at the vertices by changing line 151 in the fragment shader from gl_FragColor = vec4(color, 1.0); to gl_FragColor = vec4(v_Vertex, 1.0);. You should see smooth colors across the model’s surfaces. (Remember that color values are always positive, so you can scale and offset the vertices to get more details in the dark areas using
```
gl_FragColor = vec4(v_Vertex * 0.5 + 0.5, 1.0);
```
Study the normal vectors: gl_FragColor = vec4(v_Normal, 1.0);
Study the texture coordinates: gl_FragColor = vec4(v_Texture_coordinate, 0.0, 1.0);. Note that the areas where you see a “solid color” are areas where the texture coordinates are not changing very much and were a single pixel color from the texture map image will be used over large areas.
Study the texture mapped colors: gl_FragColor = frag_color;
Examine the diffuse color calculation. Since the diffuse color is calculated inside a procedure, create a temporary global variable at the top of the fragment shader (add vec3 temp;). In the light_calculations function, around line 103, calculate the diffuse color using white for the surface color, temp = vec3(1.0, 1.0, 1.0) * light_color * cos_angle;. Change the last line in the fragment shader to gl_FragColor = vec4(temp, 1.0);.
Suppose there is a problem with calculating the local coordinate system for each triangle. Examine the value of the vectors v_U3d and v_V3d. (Use gl_FragColor = vec4(v_U3d, 1.0);.)
Further investigating the local coordinate system for each triangle, determine if any of the coordinate system axes are degenerate vectors that have zero length. The code might look like this:
```
if (length(v_U3d) == 0.0) {
  color = vec3(1.0, 0.0, 0.0);
} else {
  color = vec3(0.0, 1.0, 0.0);
}
gl_FragColor = vec4(color, 1.0);
```
If you see any red in the rendering, those are locations in the triangular mesh that have an invalid local coordinate system.
GPU’s have floating point errors just like CPU’s. For example, study the output of this code:
```
if (length(normalize(v_Normal)) == 1.0) {
  color = vec3(1.0, 0.0, 0.0);
} else {
  color = vec3(0.0, 1.0, 0.0);
}
gl_FragColor = vec4(color, 1.0);
```
The output shows a random set of red and green pixels. This is because the length of a vector is often not exactly 1.0. It might be 0.999999 or 1.000001. Try this code:
```
if (length(normalize(v_Normal)) >= 0.999) {
  color = vec3(1.0, 0.0, 0.0);
} else {
  color = vec3(0.0, 1.0, 0.0);
}
gl_FragColor = vec4(color, 1.0);
```
The model should now be all red. When working with floating point values in shader programs, take into account round-off errors as necessary.
Come up with you own experiments. Be creative!

Summary¶

Creating beautiful renderings requires significant work. Hopefully this lesson has helped you understand some of the processes required and some of the issues you might encounter. With experience, the work becomes easier and less confusing.

Beautiful renderings typically require a combination of modeling techniques. The number of possible combinations is very large!

Self Assessment¶

If you performed all of the above experiments and understood the resulting output, then you have already completed a good self-assessment. If the experiments were confusing, please try them again!

Next Section - 12.1 - Introduction to Advanced Rendering