CS 315 Homework 5 - The Great Outdoors

Due Date: Sun Nov 10 at 11:59pm
Ongoing deliverables are required; see below

Overview

For this assignment, you'll be taking what you've learned about OpenGL modeling and adding further rendering techniques in order to build a simple animated scene. The focus of this assignment is on lighting and texturing, but in order to see the effects properly you'll also be working with bigger, more complex models---such as the models you might find available online or construct through a 3D-modeling application like Blender.

Because many computer scientists (and professors of computer science) spend far too much time inside, your program should render an animated image of an outdoor scene. For example, maybe your scene shows a pleasant farm, cheerful camping on a moutain, or joyous boats in a harbor. It can have happy little clouds and happy little trees. Most any idea is fine--this is another chance to be creative! (If you're unsure or want to confirm that you're not thinking too big, check in with me). Your one restriction will be the memory and processing speed available on your phone/emulator (so smaller, less complex models can be better).

In the end, your scene should include:

An animated light source (e.g., the sun) that changes position/direction over time
A different appearance during "day" and "night"
At least one (1) object loaded from a mesh file (using some mesh OTHER than the ones provided)
At leas one (1) object that moves in some way (animation)
At least three (3) objects (different models) shaded according to the Phong reflectance model
At least one (1) object that has been textured
At least one (1) object that uses an advanced mapping technique: bump mapping OR environmental mapping
- Note that this portion of the assignment has been reduced in scope and weight, but includes further changes for extra credit.

This assignment should be completed individually. You are encouraged to discuss high-level issues with other students, but your code must be your own. We will also be going over pieces of this assignment in class as examples--you are welcome to use and adapt that code for your assignment.

Objectives

Review working with transformations and animation
Be able to find and load 3D meshes
Be able to write GLSL shaders
Be able to apply illumination models (i.e., Phong) to shade a scene
Be able to work with textures and texture mapping
Practice advanced texture mapping and fragment processing techniques

Necessary Files

You will need to download and extract a copy of the starter code for this assignment. This project includes a basic GLES program to get you started--if you run the program you should be able to see the teapot spinning on the screen!

Be sure and read through the starter code. It includes some changes from the previous code:

Separated Shader Files: Because you'll be modifying and writing GLSL shaders for this assignment, I have refactored the shader code from being constants in the Renderer class to being separate files that are loaded in at run-time. These files can be found in the assets folder of the Android project (I have given all shaders the .glsl extension, though the extension really doesn't matter).
- diffuseVertex.glsl is the vertex shader used in the last assignment that includes some basic lighting; flatVertex.glsl is the vertex shader without lighting from Homework 3; and fragment.glsl is our fragment shader.
- The shaders are plain text, though I found reference to at least one eclipse plugin that claims to enable syntax highlighting.
- You will need to edit shader files for this assignment--you are welcome to modify the provided shaders, or to create your own using them as a base (recommended!)
Indexed Drawing: As mentioned in class, meshes can lead to a lot of excess data as vertices and their attributes are duplicated. In order to get around this, we use indexed vertex arrays--effectively we have an array of vertices, and then an array of indicies into that array that tells us where each face is. We can draw based on an index of vertices by using the glDrawElements() method.

I have provided a method, drawIndexedTriangleBuffer() that will let you draw an indexed triangle buffer. Be sure and read over the method and make sure you understand what it is doing! Note that currently you will need to pass in a separate buffer for the vertices and the normals; making a method to work with a packed index array is left as an exercise to the reader.
- The difficulty is that some object models (like the provided cube.obj) use a different set of indices for the vertices and the normals. Rendering objects like this require producing a packedBuffer and rendering as a non-indexed array. Think about why this is the case!
Mesh Class and Object Loading: Finally, I have provided a class to handle loading and storing triangle meshes loaded from .obj files. This is the Mesh class. A Mesh object is instantiated based on a file found in the assets folder; this file is parsed into the appropriate float[]s. A Mesh object also has methods to fetch FloatBuffer objects as needed; this helps encapsulate the code and avoids duplicating memory if you want to use multiple copies of the same mesh.
- Unlike the ModelFactory class this is not a static class--you can instantiate a different Mesh object for each mesh. But even if you want to draw multiple teapots, only load the mesh once (you can simply tell the model to be drawn again!)
The easiest way to understand this class is to look at the source code (it's just String/File parsing) and how the class is used in the starting sample. It should be self-explanatory

You will need to make modifications to this class. In Part 3 you'll need to modify the loader so it also collects texture coordinates (this should be simple based on the provided code). You also might need to tweak the file parsing if you want to support more complicated models or material files (see below).
Note that the ModelFactory class and its ability to create cube and spheres is still available; you can always develop your own things as combinations of models (like re-using your robot!). You can also just make new models by hard-coding points or adding new procedural methods to the ModelFactory.

And of course, the zip file also contains a README.txt file that you will need to fill out. DO FILL THIS OUT.

Platform Options

As an experiment in order to ease the development process for people who are still struggling with the Android emulator, for this assignment you are welcome to develop in an alternate platform: in particular, using WebGL. WebGL is OpenGL ES for the web browser--it's exactly like what we're doing, but loading in an HTML5 Canvas rather than an Android View. See here for details.

Development Schedule

This assignment can be divided into a few different parts, described below. Each part corresponds to a topic we will be covering in class over the next week. I have included "due dates" for each part--these deadline will help to break up the assignment into more smaller parts (though each part builds on the first).

By each deadline, you will be asked to upload your working code that demonstrates that you have reached that milestone. I will try and "spot check" this code as time allows--if you want me to be sure and check your progress at a deadline, let me know.

The deadlines are at midnight on the following days:

Part 1 (Modeling & Animation): Wed 10/30
Part 2 (Lighting): Sun 11/03
Part 3 (Texturing): Wed 11/06
Part 4 (Advanced Texturing): Sun 11/10

Part 1: Modeling & Animation

Part 1 of this assignment should be primarily review: you will be working with models and the simple shaders already provided in order to create a simple animation. There are a few new details, but understanding them simply involves understanding the starting code

Note that this Part of the assignment effectively covers requirements 1-4 of your final scene. That may seem like a lot, but that's because much of it is provided for you!

Modeling the Scene & Loading a Mesh

The first step of this part involves being able to load and render a new triangle mesh.

The very first thing you should do is make sure you can run the starter code and load the teapot mesh. Be sure and look through the starter code and make sure you understand how this teapot has come to be.
Try modifying the code to load the legoman. Note that you may have to adjust the transformations to make sure he appears nicely (this should be old hat by this point).
- The program will take a few seconds longer than normal to load, since it has to read and process the .obj file. There is an optimization whereby you save the binary representation of the Mesh object (or even the Buffers themeslves), in order to avoid reprocessing it.
The next step will be to plan your scene. Come up with a basic idea, and think of some simple objects that you might model and render.
- I want to emphasize simple: the Android emulator has finite memory and speed. The more complex the model, the greater the chance of things crashing. You should still be able to use most simple models, but keeping an eye on the number of triangles is a good idea. Note that reusing the same model doesn't require extra memory (though will require extra rendering time). If you think you hit the limits of the emulator or your phone, let me know!.
- Keep in mind the overall requirements: you can get away with only having 5 objects in your scene (e.g., if your mesh is shaded and animated) Also note that the cubes and spheres provided by the ModelFactory each count as an object.
Once you know what objects are in your scene, you'll need to get some models to represent them. In particular, you'll need at least one mesh that has not been provided for you.

There are lots of 3D models made available for free on the Internet--indeed, being able to find and use these models (with proper credit) is a big step towards developing "real-world" graphics applications. You can google for particular models, or peruse various repository sites (if you find particularly good ones, let me know!)
- Note that the mesh loader I've provided loads basic files in Wavefront .obj format--we mentioned this briefly in class. So you'll want to search for ".obj" or "object" files as a format.
- Depending on the model, you may need to modify the hand-made Mesh loader class somewhat to make sure the parsing is robust. Similarly, if you want to load in different mesh formats, you are welcome to create a loader (if you wish to use a 3rd party loader, please check in with me first!)
- Watch out for meshes defined with quadrilaterals---these are meshes that have faces defined with 4 vertices (each of which has up to 3 components: position/texture/normal), rather than 3. If you have a quadrilateral mesh, you'll need to do some work to convert it to a triangle mesh. This can involve running it through a simple converter program (which need not be in Android) to produce a new mesh, or something more complicated.
- If you're feeling bold (or have some experience), you can also try creating a simple model in a 3D modeling program such as Blender. These models can be easily exported as .obj files.
- Models often also come with material files (.mtl) that specify the color, lighting, textures, etc. for a model. The loader as provided does not yet support these specifications, though adding that functionality would be a handy extension!
  - The challenging part of this extension is not parsing the .mtl, but dealing with the fact that a model might have multiple materials for different sets of faces! You won't be able to have just one indexBuffer, since each will get a different set of attributes passed to the shader.
I recommend finding and loading a single model at a time, to make sure things work. Getting one model loaded is a good target for this part of the assignment.
Loading a mesh should be trivial with the provided code. You'll need to place the mesh file in the assets folder of your Android project in order to have access to it. Then just tweak the given code to load in that mesh.
- Remember that you may need to scale or rotate the model in order to make it show up right. For example, the teapot is really big so has been scaled down; legoman is a little small so I tend to scale him up.

The Sun

Once you have a model loaded and your scene is beginning to take shape, you should add in functionality to model the sun moving across the sky in your outdoor scene. The actual model of the sun isn't important--it can easily be off-camera (who wants to look at the sun anyway?), or simply a yellow sphere or even a very large point. What is important is that your light source moves in a circular arc around your scene.

As we will disuss in more detail through lecture, we often think of light as coming from a particular location--from a light source (like a lightbulb). Thus we can think of light as coming from a particular position, and often having a particular direction (think like a spotlight). In the second part of the assignment you will be implementing a more complicated illumination model, but for now we'll just focus on the position of the light and making that change over time--the direction of the light will be from the light's position to each of our vertices.

Consider the provided diffuseVertex shader (which is the same "shading" shader you used in the last assignment). Notice that this has a variable that is the lightPos. This represents (gasp) the light position! The shader initially hard-codes the position, so the light is always coming from a spot on the z-axis (around where the camera is).
Your task is to make this variable change over time (the same way you had your modelMatrix change over time in order to support animation--that is, it was calculated each frame). So instead of hard-coding the variable, you'll want to pass it in as a variable to the shader.
- Important question: should this position be an attribute of the vertex or a uniform variable?
This is not as difficult as it may seem (it's only a few lines of code--though you will need to tweak the shader code). After you've finished, you should be able to see the "light" for your model revolve around it--that is, which side of the model is lit and which is dark should rotate.
- This would be an equivalent effect to if you rotated the camera around the model but didn't actually move the light source!
- Testing this with something simple like the cube is a good idea.

The Night Sky

Because your sun moves, you should be able to differentiate between day and night. That is, half the time there should be a sun (at a roving position in the sky), and half the time the scene should be "at night". At night, the sun should not be visible (and should not be lighting things!), but you should be able to see at stars (at least 4).

Stars can again be modeled however you want: larger points (this is easy and recommended), very tiny spheres, star-like models, etc. Note if you use models for stars, they will have their own illumination model so that they can seem to "glow."
- Stars can either "fade in" or appear instantly at sunset. But they shouldn’t be visible in bright daylight.
- Note that as the sun "rises" the amount of overall light will increase, and it will decrease as the sun "sets". This will be controlled through the full illumination model details in Part 2.
I suggest that this is a good time to make sure that your scene has a model representing the "ground" (so you can see the difference between the ground and the sky). A simple 2D rectangle parallel to the xy plane can do the job nicely (the ground can count as one of your required models if you texture it!). To color the sky, you can either change OpenGL's clearColor, or you can also use a 2D rectangle as a "background" (like they do in the movies! And if you texture that, you are basically modeling a matte painting!)

Animation Controls

In order to ease testing and grading, you should provide an interaction mechanism to both control the movement of the sun and whether it is day or night. A button that toggles day and night and a button that starts and stops the sun would be sufficient, though slides/etc. are of course welcome.

Note that I should have the XML with the GLSurfaceView set up correctly in this starter code, so hopefully less headaches there! But when you rename the application package, you may need to tweak the XML slightly.

Having done these four things you should be well on your way to having a nice animated scene! Feel free to add more models (ideally in a format that makes them easy to comment out for testing), and add some animations to the models if you want your cows to wander over the farm or whatever.

Remember that Part 1 of this assignment is "due" Wed 10/30 at midnight.

Part 2: Lighting

In Part 2 of this assignment you will apply more complex lighting to your scene. You will be modifying or writing new shaders in order to effectively shade and render the objects in your scene. The part covers requirement 5 of your final scene.

Your program should implement the Phong Reflectance Model of illumination, applying this lighting to three separate "objects" in your scene. This model should include ambient, diffuse, and specular components. Note that it is okay to have 0 ambient light if you have enough light sources to suffice, such as through three-point lighting.

Objects can be either custom meshes, provided primitives (cubes or sphere), or even hand-created models (e.g., if you want to make a tetahedron).
You may wish to use a different shader to light different objects (or the scene at day or night). You can load multiple shaders in your onSurfaceCreated() method, and then switch between them by uing the glUseProgram() method.
- The in-class demo code from 10/28 has sample code of loading in multiple shaders, as well as some tweaks to the starter code and shader loading that may fix issues. If there are problems, let me know.
- Note that there is a run-time cost of switching shaders; it's best to draw all your objects with one shader first, then with the next, then with the next.
In any case, each object should have a different set of reflectance coefficients representing different materials. In the simplest variety, some objects may be "shinier" (have higher specular coefficients) than others.
- There are a few different ways to choose different coefficients. One involves "playing" with the materials by testing different values to see what you get. There are interactive programs that may make this easier--for example, Nate Robins lightmaterial tutorial will let you play around with coeffients on a per-color basis.
- You can also load up a modeling program like Blender and use that to play with material settings. You might find This wikibook informative, though Blender materials do not map directly to the coefficients of the class Phong model.
Optional If you want to get really advanced (going above and beyond the requirements), you can define a different BRDF for different objects. This is essentially writing a different illumination model (following the same principles as Phong). You might look at Disney's BRDF Explorer and the MERL database for examples of different BRDFs. Having a mesh that supports such a model could be worth extra credit.

Your scene only needs to include one light source (the sun, which should be a point light), but you are welcome to include more. In particular, adding point lights or spot lights (with finite distance) could make for a compelling night scene!

Again, your scene should have different lighting models for day and for night. You can achieve this by using a separate shader for each, or by sending in a uniform variable (a bool?) to act as a "switch" in your single shader.

Night time might be a good time to add a very low level of ambient light--the effect of the stars!

Finally, you are welcome to adapt the demo code from lecture. Your program should use Phong (per pixel) shading rather than Gouraud (per vertex) shading for at least one object.

Part 3: Texturing

In Part 3 of this assignment you will add textures to at least one (1) of your objects. Note that this will need to be a different object than you applied Phong lighting to (even though it is possible to combine the two techniques)--you will need to have at least one object that is lit without being textured.

Texture-mapping is relatively straight-forward, and we will discuss it in detail during class. You can also reference this tutorial for further examples.
You should generate mipmaps for your texture in order to make it render smoothly.
You are welcome to use any school-appropriate image for your texture. I recommend material-based textures, rather than wallpapering your mesh with the Seahawks logo. There are lots of texture-appropriate images online; just be sure and cite your sources in the README.
- Textures, as with meshes, should be stored in the assets folder for easy access.

I highly recommend you modify the ModelFactory to specify texture coordinates for the cube; this will help you test if your texture loading and shading code works.

Be careful about applying textures to the triangles in your mesh so that you get a consistent appearance! Particularly when trying to map a square texture to a triangular mesh!

If you wish to apply textures to one of your loaded meshes (and I expect you will), you'll need to make a couple of further changes:

The mesh loader that is provided does not process texture coordinates (the "vt" coordinates in a .obj file). You will need to modify the loader to properly parse these coordinates. This should be a simple adaptation of the existing code.
You will also need to tweak the "face" processing to store the texture coordinates as part of the indexed mesh.
- Note that if your mesh didn't include texture coordinates, you will need to come up with a way to generate them programmatically (e.g. assign one vertex as 0 and one as 1).

Part 4: Advanced Texturing

In the final part of the assignment, you will be adding a more "complicated" lighting or texturing effect to one of the objects in your scene. There are a couple of different options for you:

Bump Mapping

You can texture an object using a Bump Map (normal map), rather than calculating the normals based on the geometry. This will allow you to add more detail "textures" (in the tactile sense) to objects without needing to increase the complexity of your model. For example, maybe you create an embossed cube, or a bumpy sphere, a watery quad, or some other effect.

You will need to find and utilize a normal map image--there are plenty of examples on the Internet (just make sure you cite your sources in the readme!)
You will need to modify your shaders to account for the texture-based normals, as we've done in class.
Note that it is possible to load in multiple texture samples (multiple sampler2D variables), and then mix and match them--so you can have a normal map AND a texture map, using both at the same time!
Alternatively, you can programmatically determine a "bump map" for your object. Rather than using an image loaded from a file, you can simple perturb the normals to produce a textured effect. For example, using a periodic function such as: cos(a*2*PI*u) * cos(a*2*PI*v) should produce a nice "wavey" normals that can look like water ripples. This is a suitable alternative to loading in a normal mapping (assuming your perturbing produces an indentifiable effect!)

Environmental Mapping

You can also texture an object with an environmental cube mapping, so that it looks like it is "reflecting" the environment.

You will need to utilize a texture to map on the inside of the environmental cube. You can either find one online (search for "cube map" images or "skybox" images), or create your own. You can create your own image simply by sampling any image with the "t" format we used in class, or by compiling a series of scenes to map onto your cube faces.
- Note that it is possible, with some work, to produce a cube mapping that reflects your scene's environment, so it looks like your objects are reflecting! You adjust the eye to be looking out from your "reflecting" object along a particular axis, render your scene, screenshot the scene, and then compile those screenshots into your cube map image.
- It is also possible to automate this process--to render the scene to an image buffer and then rerender your scene using that image buffer as an environmental texture map. As we won't go over how to do this, such a process would be worth significant extra credit. I recommend you start here to look for the relevant methods and techniques. Note I'm happy to talk about the details--I just haven't had time to get it working on my own!
Note that by combining the environmental sampling with the object color, you can reduce the "perfect mirror" effect of the object. In fact, you might experiment with having only the specular component of your lighting include environmental mapping (so that the specular reflection is actually a reflection).

Shadow Mapping (Optional! Extra Credit!)

Finally, fulfilling the requirements both of this part of the assignment and earning extra credit, you can implement shadow mapping for one or more objects. This will require doing a two-pass rendering--you'll need to render the scene using the light source as the eye, rendering the depth to a texture. You can then use this depth to calculate whether a fragment is in shadow or not.

We will discuss this algorithm from a high level in class, but you are welcome to try and implement it for this assignment. I recommend you start here, but there may be other resources to use as well (if you find any that are good, let me know!)

Extensions

There are numerous extensions for this assignment described sporatically above. The most prominent chance for extra credit involves creating changes or utilities that can help the rest of the class. For example, if you come up with a way to convert quads to triangles and share that with me so I can distribute it to the class, you can earn extra points. You get rewarded for helping others!

Adding further detail to your scene and rending can also lead to some extra credit--for example, processing material files, implementing a BRDF, adding special effects (fog, fire, etc), or even just making a complicated scene animation.

Adding extra interaction (e.g., the ability to move around the scene in a first-person view, push objects around, etc) would also be worth something. The possibilities are endless!

Submitting

BEFORE YOU SUBMIT: make sure your code is fully functional, but also documented and readable. Again, grading will be based primarily on functionality. I cannot promise that I can find any errors in your code that keep it from running, so make sure it works. But I will look over your code to check your implementation, so please make sure that is readable :)

Upload your entire project (including all the src, resources, layouts, libraries, etc) to the Hwk5 submission folder on hedwig. Also include a filled-in copy of the README.txt file in your project directory!

The homework is due at midnight on Sun, Nov 10.

Grading

This assignment will be graded based on approximately the following criteria:

1x mesh loaded from a file [20+1%]
Animated object [5%]
Animated light source (sun) [10+1%]
Different lighting for day and night [5+1%]
3x objects rendered using the Phong model [20+1%]
1x object that has been textured [15+1%]
1x object with advanced texturing (bump mapping, environmental mapping, or shadow mapping) [10%]
Code is documented and readable, with assets in the proper folder [5%]
README is completed (including asset source citations!) [5%]