Skip navigation

As discussed earlier, my final project was to create the decalling system similar to what Unreal Engine 4 has. In my presentation before, I discussed an algorithm, where I was changing the mesh data in the vertex buffer itself, by inducing an additional parameter called  index. This index decided what texture should the geometry use. But, as I moved along with this project, I realized some key problems with this algorithm:

1. The change of textures was happening at vertex level, hence the decals were not covering the geometry accurately by pixels.

2. This algorithm was changing the mesh itself, which creates problems in instancing.

Hence, I came up with the 2nd algorithm. This algorithm assumes symmetrical decals, which means that to detect the coverage of a decal volume over a geometry, we don`t need pixel accurate collision detection. Here is how the algorithm works:

ICGAlgo2

This algorithm manually sets the uniforms corrosponding to all the decals and then you have to manually compare the fragment (in world coordinates) with the min-max range of every decal. This created a pixel level accurate decal system for decal volumes like cube, sphere, and a cylinder. But, this had problems:

1. It involves a lot of manual work, like setting up of uniforms for every decal. Because of this, this algorithm defeats the purpose of this system. This system is intended to be added as a feature in a game engine. Hence, it is absolutely required to make this algorithm generic for any number of decal volumes.

2. Bleeding occurs while using spherical decals and flat meshes to put decals on. But, even Unreal Engine 4 has the same issue. Hence, I decided to make this problem out of scope of this project.

Hence, I came up with another algorithm. This algorithm renders all the meshes in the scene for every decal using a decal shader. This shader renders the fragments lying within the bounds of the decal with the decal properties, and rejects all other fragments. This basically, puts a “decal sticker” on a mesh. Because of this though, the problem is that the fragments that decal shader wants to color have already been colored by the mesh specific shaders. Hence, the only way to color these fragments is to disable the depth test. But, because of that, depth errors were created because of these decal stickers. Here is how it looked:

ICGWithoutStencil

I used stencil buffer to solve this. This is how the algorithm works:

ICGAlgo3

Some of the results using this algorithm:

ICGFinal1

The picture has 3 decals a pink cube, a blue sphere and a yellow cylinder. Pink cube covering parts of the sphere mesh and the 404Sight`s character mesh.

 

ICGFinal2

The picture shows that the decal not only chnages color of a mesh, but also makes that part toon shaded.

 

ICGFinal3

This picture shows the pink cude decal affecting a part of the sphere and a part of the right wall.

 

The biggest advantage of using this system that I found while using this in my Thesis game is that this reduces the number of renderable objects in a scene. Because of this, a lot of memory load, like shadow maps, is decreased. In 404Sight, the map sizes decreased from 3GB for 4 levels to 900MB for 14 levels. Other than this, this project helped me a lot in developing graphics algorithm skills. Also, this marks the end of the blog series for this class. It was an amazing class. I learnt so much from it. I will try to come back to the blogs with another interesting series.

 

Here is the list of the bugs/ requests I worked on:

1. The mouse cursor was not appearing when you end the game. This bug was a silly one. I forgot to add a function call that enables the mouse cursor at the end of the game.

2. Play button was not responding. This was not really a bug. This was reported because of the fact that we did not have a loading screen at the start of the game, which, on slow PCs felt like the game had frozen there. I fixed this by removing some unnecessary sanity checks that were slowing the game at the start of the game. Other than that, I added a loading screen as well.

3. Falling through levels at the end of the loading tunnel. This was a bug in the sanity check. While working on getting the wall behind the player in the tunnels, me and/or James reset the sanity Check boolean before it could even put up a loading screen. So, I fixed it.

4.  Controller a press crash. This bug occurred because of the fact that even after the game had started, somehow one of the isHovered boolean was left activated. So, because of that when you press a, it caused the system to trigger the onButtonClicked function for one of the back buttons. I fixed it by wrapping all the buttons under the logic that the function can only be called if you are NOT in game.

5. Added resolution/ windowed mode option.

6. Added options for setting up the levels for field of view, sound volume and effects volume.

newOptions

 

So, 404Sight has been published on Steam. Yay. Here are some facts:

1. We have about 80,000 downloads.

2. Almost 86% of the reviews we got are positive.

3. We got an article published in The Verge.

4. We got articles in the major Indian newspapers, including The Times of India and The Hindu.

HinduArticle

5. There have been some bugs reported and some requests made by the Steam community. We made up a priority list of how we want to approach them.

6. We are kicking ass. TOUCH WOOD.

10317552_894193953955618_4666112191061351182_o

WHAT : Decal system UE4

How much I have done: For now, I have it so that you can add a box of any size in to the world and can move it around. This decal volume would change the texture of COLLIDING PIXELS.

Algorithm:

1. Pass renderable objects to the decaling system

2. Check if the vertices in the world coordinates lie inside or outside world bounds of the cube (computed by first and last vertex of the cube)

3. If found to lie within bounds, change a parameter, called index in the vertex array, to 0.0f

4. In VS,  interpolate index.

5 In FS, if the index < 0.5, use texture1, else use texture2

 

Problems faced:

1. Changing the buffer on the fly.

2. Induce additional data in mesh

 

1. Decal volume completely out

noneCovered

 

2. Decal volume covering roughly half the cube

clipped

 

3. Decal volume covering the whole cube

fullyCovered

 

 

WHAT WILL I DO:

1. Introduce more decal volume shapes: sphere, cylinder

2. Introduce more properties to be changed by the decal : normal maps, etc.

3. If time left, will port some shaders from my game to UE4.

 

 

A couple of days left before we publish our game. The whole team is working extremely hard. I have worked on or helped on so many small things. But, I mainly worked on 2 main things. First was to get new audio in, with the awesome bookends made by Rachel. For the audio, we wanted a different track to be played for every level. I was able to get the new sounds in using my old system. So, all I do is I play all the sounds the same way, I just change the audio reference based on the level id, whenever a level is loaded. For the bookends, Jame and Sid made UMG classes using Unreal`s movie textures. On my end, I load the bookends at the start and the end of the game. The tough part to play the audio associated with the bookends at the right time (because audio doesn`t play with the movie textures. AAAAAghh). So, to solve this, I had to time the audio to play correctly with the bookend, and then switch the audio reference immediately, so that that can be matched up to the next bookend.

 

Other big thing that I worked on was to add additional features to the loading tunnel. The first was to reset the timer the moment someone comes out of the tunnel into the new level. We needed this because of the fact that players were loosing their time in the level while crossing this tunnel, which the players who start their game straight from the level select menu don`t. Other thing was that we needed to create a wall at the start of the tunnel once the player enters the tunnel, so that when the player turns around, he/she can`t go back (because there is nothing back there :-p). James and me worked on this together.

bookEnd

 

black_oem_controller_1

Alright. Another simple sounding task, going haywire. So, UMG has nothing that allows you to highlight a button by a controller key press. So, we had to fake it somehow. The first idea that came to my mind was to move the mouse cursor using an analog stick. I saw a game dev team doing something similar in their menus. So, I started to look it up on forums on how we might achieve it. Thankfully, I found a user made plugin, called the Victory plugin, made by someone called Rama. This guy has been really useful in general helping people out on Unreal Forums. So, this plugin had functions like getMouse position and setMouse position. Using this, I wrote the following blueprint code that allowed me to move the mouse with my analog stick:

 

controller1

controller2

controller3

 

Once this was done, the next problem was that there was no way to simulate a mouse click using a controller. Tony found a way with which you can detect a mouse hover is you surround all the buttons with a UMG border. This gave me a heads up on how to get the controller working. So, what I did was to cover all the buttons with borders. With each button, I associated a bool named isButtonHovered. So, whenever this button is hovered, all the bools on this screen are equated to false, and the bool corresponding to this button is turned on. Now, whenever controller key a is pressed, I call the onButtonClicked function for the button that has its hoveredBool selected.

controller4

controller5

 

 

This assignment was about normal mapping. I did that in the previous semester as well, in DirectX. Though, in this semester, I understood this a lot more. So, normal mapping is essentially altering the normals based on a texture. Since, the objects are transformed in different ways, the normals from the normal map texture need to go under some transformations. So, we take the normal from the normal map texture to a space where basis is formed by tangent, bi-normals and the normals of the fragment. These 3 are taken as inputs while exporting the files from Maya. Personally, I get my tangent, bi-normal and normals in view space and then make basis out of it. This leads the normal from the normal map texture to be transformed correctly and can be used straight away in the lighting equation. This is how it looks like (normal mapping done on walls and the floor):

normalMapping

 

In the picture above, the light is coming from the top. The next thing that I did was to implement toon shading. Toon shading helps to make an object look a 3D cartoon (just like Borderlands 2). Toon shading involves 3 features: darkened silhouette edge, stepped diffuse shading, and stepped specular highlight. First, I did the stepped diffuse shading. For this, I basically clamped values to a minimum threshold. For example, all the diffuse intensity values from 0.9 are all clamped to 0.9, and the values between 0.5 and 0.9 and all clamped to 0.5. Similar idea can be used to make the stepped specular highlights. To make a silhouette edge, I did a very simple technique(whose concept I read on a graphics forum). So, I render the object counter clockwise with the stepped phong shading, and then render the same object clockwise (so that the back facing polygons are rendered this time). While drawing it clockwise, I use a shader that essentially extends the vertices along the normal. This is what the final result looks like:

toonShading

 

Controls:

W,A,S,D to move camera around. I,K to increase/decrease the intensity of light. z to change between phong shading and toon shading.

 

This is unbelievable. Our game has been greenlit on Steam in 6 dayssssss!!!!!!!!!!!!! OMG!!!!!!! I think this might be some sort of a record. Thanks a lot to all the people who voted us up. Thanks a lot to Steam. This is amazing. My game will be available to play on Steam. But, this is not it. We recently won a grant from Epic Games worth 13K dollars.

https://www.unrealengine.com/blog/epic-announces-unreal-dev-grant-winners-march26

This is how it feels when hard work pays off. But, wait, what does this all mean. We have less than a month to publish our game. That means we need to work really hard to finish and polish the game. The whole team is working really hard on building levels, and arting them. I, personally, have a lot of things to build/fix, like getting controller support in menus, building level select option, getting new audio in game, etc. I am almost done with getting the level select screen in game, along with adding sounds to button presses. Next, I will work on getting controller support to menus.

A lot of work to do. But, good things are happening as well. So, can`t complain 🙂

 

GreenLit

Alright, we are back. This assignment was on shadows. I implemented 2 types of shadow techniques. Projective shadows and shadow mapping. Let`s go through projective shadows first.

This technique uses the very basics of shadows, projection. Since, we know that a shadow on a plane is essentially the projection of an object on the plane with respect to the light. Hence, we use the projection matrix that can project an object in the world coordinate system to a plane, and then draw this squashed object in the scene. To make it look as a shadow, we render this object all black. Since, we need to see the objects below this shadow object as well (because in real life, things that are under shadow can still receive some light in the form of light reflections in the surrounding areas), we blend the underlying pixels to the projection shadow objects, using the simple blend equation:

glBlendFunc(GL_SRC_ALPHA,
GL_ONE_MINUS_SRC_ALPHA);

The other issue with this approach is that the projection assumes an infinite plane. Hence, the shadows are rendered outside the plane as well. Other than this, there is z fighting between shadows from different objects at the same place as well. To solve these 2 issues, we use stencil buffer. Before rendering all the geometry, I used:

glStencilFunc(GL_ALWAYS,1 , 0XFF);
glStencilOp(GL_KEEP, GL_REPLACE, GL_REPLACE);

This would replace all the fragments that are covered by the objects in the world by 1(stencil buffer was initialized to 0). Next, before the projection shadows are rendered, I used:

glStencilFunc(GL_EQUAL, 0X01, 0X01);
glStencilOp(GL_KEEP, GL_KEEP, GL_ZERO);

This solves 2 purposes, first, it just renders at places where the stencil fragment is 1, and hence, no shadow would go out of the bounds of the geometry bounds. Further, it resets the stencil bits wherever a projective shadow passes the depth test. This would not allow the shadows from other geometries to rewrite on the fragments where a shadow was written previously, hence avoiding the z fighting between shadows. This is how it looks like:

 

shadowProjection

But, this technique has some unsolvable issues. This technique can only be used to cast shadows on planer polygons. Further, it can`t be used to do self shadowing. The last issue with this technique is that it requires the rendering of all the objects number of planes number of times. Hence, objects rendered = n*m +n.

 

Hence, we move to the second technique, shadow mapping. This method uses the idea of projection as well. But, here, we create a shadow texture. This is essentially a depth texture created by copying the contents of the frame buffer (depth) to a texture object. To create this texture,  we create a framebuffer object, and render the occluders (objects that would cast shadows. Teapot and the cube in this case) with light`s perspective onto the framebuffer. The depth buffer part of this frame buffer stores the depths of the objects near the light. Now, we set the openGL state to render onto the screen. The, we set the shader to the fragment shader that takes in this shadow texture as an input. In the vertex shader, we send the the model-view matrix and the light-model view matrix(instead of the camera, we use light as the eye). Using these 2, we compute the position in the view space as well as the light space. Now, we send this data to the fragment shader. In the fragment shader, we use the x,y coordinate of the light`s interpolated position as an input to the texture function in the fragment shader. This function would return us the depth stored corrosponding to that uv. We compare this depth to the z value of the interpolated position (light`s perspective). If Depth < Z, then the point is in shadow. The easiest way to do this is to make a variable called visibility, and inititialize it to 1. If the fragment is found to be in shadow, change the visibility to 0.5. Now, multiply the visibility to the final color. Using this, this is how the shadows look:

ShadowMapping

 

Here, you can see the self shadows on the occluders as well.

This assignment took me much more time that I expected. I just kept running into issues. The first issue that I ran into was that I was not multiplying the occluder geometry (while making shadow map) with the projection matrix. Such a rookie mistake. But, it took me forever to solve. I found out that the problem was in my shadow map itself, by making the texture of the ground as the shadow map. In this way, I was able to visualize the shadow map (which was blank w at that time). The second major issue that I ran into was that my stencil buffer was not being created. I spent over 6 hours trying to figure out the problem. THe prblem was that the superbible framework was creating a window wothout the stencil buffer. The problem was that I could`nt find the place I could change the window creation code. Finally, Gagan told me about a way to that. After that, I was good to go.

The control scheme for this assignment is:

W,S to move the light in Z axis.

A,S to move the light in X axis.

Q,E to move the light in Y axis.

I,K to increase/decrease the light intensity.

Z to toggle between shadow mapping and shadow projection.

 

GDC is done. I wouldn`t say it was as positive for our game as it was negative. So, we presented at the Intel competition. I must say, Tina and Rachel did an amazing job at presenting our game. But, sadly, we did not win any award. I must say, it was pretty disheartening, mainly because I felt that our game was definitely a good contender in the visual effects. Other than this, we found out that if there is no internet while playing the game, the game crashes while trying to connect to the internet. I was soooooo angry at Unreal. i mean I tested this last semester on one of the older versions. The game never had any issues back then. This problem occurred in one of the newer version. I posted it on Unreal`s forums. they replied back saying they are looking into it. They even told us that there was not even a simple function that could check if there was no internet connection available, ie, a simple timeout function that returns false, if there is no internet connection. Hence, we had to rip off the data sending part from the demo. I hope that the analytics could be used somewhere else in the game, like achievements (if we get greenlit on Steam). We love Unreal Engine a lot, but sometimes, it can be a pain in the butt.

Moving to positives though, we got some really good feedback from playtesters. I got to pitch my game to a lot of industry professionals, which was nice. We also have put up our game on Steam Greenlight, and are hoping that to happen this year. Since, we need to publish our game somewhere to graduate, we might go for Desura as well. But, let us see how things roll. We are still pretty confident about the future of this game, and will continue to work hard on it. Hopefully, good things will come up.

math-and-frustration