LineWars VR Blog Posts

Mar 3rd, 2019 - Cruiser Death Handling, Multiplayer Work, Oculus Go Performance Issues

Cruiser Death handling

After I got the trailer released, I began working on Mission 7, where the goal is to destroy an enemy convoy consisting of several cruisers. I could not get very far with this mission, as there was no way yet to kill the cruisers. You could only damage them, but there was no cruiser death handling implemented yet. Thus, in order to get further in my game, I needed to implement a death handling for a cruiser.

I first started experimenting with some kind of an explosion when the cruiser dies, something like what I had for the asteroids. However, this seemed quite difficult to get looking in any way convincing, and I also had trouble figuring out how the exploded fragments would then vanish from the scene. With the asteroid it is sort of expected that the fragments crumble and thus get smaller and smaller and then vanish, but why would some exploded parts of a ship do that? If I did not make the fragments disappear, though, I would need to keep them alive for the duration of the scene, and that would make them also need collision handling, you should be able to shoot at them, and so on. This seemed like a lot of work.

In the end I decided to have the cruiser just explode into two parts, so it kind of gets cut in half. This way I only needed to model two fragments, and I decided it would be OK if these parts would just stay in the scene tumbling, and you could actually collide with these tumbling hulks. I used the fire animation on most of the body panels of these destroyed parts, so they look like the remnants of a big explosion that cut the ship in half. I still need to fix the texturing (and perhaps also the modeling) of the area closest to where the explosion happened, but other than that, the parts seem to work okay.

I still need to work on improving my PDC gun handling to allow several cruisers in the scene, for the Mission 7 to work properly. Currently I only have one array for cruiser PDC targets, but when there are several cruisers in the scene, these all should have their own targets for their PDCs.

Multiplayer Work

I also decided it was high time for me to start at least looking into the multiplayer features of LineWars VR. As it is rather difficult to get started with such a major feature, I thought I'd start simple, and first look into getting VoIP working between two players.

I looked into the Oculus VrVoiceChat sample that is included in the Oculus Unity Integration package. It seemed to suit my needs pretty well, so I implemented a similar Oculus Rooms -based system, where you can invite a friend that also has LineWars VR into a room that you created, and then a VoIP connection is automatically created between the two players. I am having some issues with this yet, and it seems like there are some issues with the Oculus forums as well, as many replies to similar questions seem to be missing. For example How to remove room invites forcefully. Anyways, the issues I am currently having are:

Working around the above issues, though, allowed me to have a VoIP conversation with a friend while we both played the game. The next step is obviously to add some networking code to allow the players to play in the same game. I planned to start working on this next, however, the performance problems I had been having with my Oculus Go made me want to look into those first.

Oculus Go Performance Issues

Since I received the Oculus Start Development Kit early last month, I have been testing my game on the Oculus Go. I was expecting Oculus Go to be in many ways better than my Samsung Galaxy S6 using the Gear VR device, however it seemed the opposite was true. This is what my impressions with Oculus Go were, compared to my Gear VR:

I had assumed the minor frame drops I had with my Galaxy S6 would all vanish when running on Galaxy S7 or faster devices (including Oculus Go, which has the internals of Galaxy S7), but it actually performed much worse.

I actually spent about two weeks fighting with the performance issues, which did not seem to make much sense. I got big frame drops even in the main menu of the game, which only had a few buttons and nothing else. I even tested with hiding everything from the main menu, except the camera and the skybox, and STILL I got constant dropped frames, as in the following graph!

In the above image you can see the graphs from my loading screen and then the main menu. The loading screen looks rather normal, with the constant CPU load causing the framerate to drop to 70fps, and the GPU not doing much of anything. When the main menu begins, though, there are occasional CPU jumps from the base level of 60% usage to about 65% usage or above, and each of these jumps seems to immediately cause frame drops! I just could not figure out how such a small CPU usage increase would immediately drop frames, and what would even cause such periodic CPU usage increase when just sitting in the main menu?!

After a few days of not getting any further with the frame drops in the main menu, I decided to focus on the actual game instead. I used the profiler to look into code that takes a lot of time to execute. Here also there were some very strange results, where a simple code that normally takes almost no time suddenly took several milliseconds to run. This below is what the average frame rate graph looked like when playing the actual game.

After adding some UnityEngine.Profiling.Profiler.BeginSample() calls into my code, I noticed that the code that handles the cockpit instruments seemed to sometimes take quite a lot of time to run. When digging deeper into this code, it turned out that the problem was the CrosshairsCheck code, which sends a ray forward from the player's ship, and checks what the ray hits. I am using my own KDTree implementation for the ray-mesh intersection tests, and when the target was a space station (my most complex object), this test took a lot of time (0.80ms or over 80% of all the time spent in all my scripts) even on the PC, which is about 10 times faster than the Oculus Go! That was not good.

I realized that the KDHit test routine I had written in C# code was a good candidate for porting over to native code. I did that, and after that change this test took less than 0.1ms to run on the Oculus Go. This did not fix my main performance issues, though.

After a lot more tests it began to look like every time I activated a new game object, there was a frame drop. Thus, I began changing my object pools to not actually contain disabled GameObjects, but instead have all the GameObjects active, but just scaled to zero. This of course increases the number of draw calls. Since I do not seem to be bound by anything graphics related, instead by some Unity GameObject activation-related issue, I thought this might be a good trade-off. While I was doing this, I also decided to switch my "AsteroidHit" handling (which actually does not have much to do with asteroids, it is just an animated sprite that displays a flash and some ejecta from any kind of hit to a ship hull or asteroid) to use native code. I created just a single GameObject to contain 20 separate square "panels", each by default having a zero size. These panels can then be located in world coordinates where such a hit animation needs to be shown at, and rotated and scaled to face the camera. Thus, I did not need to have any GameObject activation when such a hit happens. This improved the frame drops somewhat, but still many frame drops remained.

Finally, after I had fought with this problem for almost two weeks, it occurred to me to check what would happen if I switched my background music from "Streaming" to "Compressed in memory" in the Unity audio clip settings. To my surprise, this finally got rid of the periodic frame drops in the main menu, and also improved the frame drops in game quite noticeably! I had assumed that the background music streaming would happen in a separate thread, so it had not even occurred to me that playing music could be the reason for my frame drops! After this change, and including some (but not all) of my GameObject activation changes, I got the following frame rate graph (showing the loading screen, main menu, and start of a mission, including a level change). So, not quite a straight line yet in the actual mission, but much better than what I had managed before.

I still had an issue of an occasional sudden major frame drop. I began to suspect this was some kind of Just-In-Time compiler issue, where the first run of any code takes a lot longer than subsequent runs. This would be a problem in my game, as there are a lot of complex code that gets executed when something specific (like a collision) happens. If the first collision always causes frame drops, that would be pretty bad. I decided to properly test this theory, using my Mission 9 scene which I had not finished yet. I made the scene contain only three ships, which fly towards each other at their maximum speed, so they all collide with each other at almost the same time. I then connected the profiler to my Oculus Go and ran this test scene.

The above image from the profiler shows the frame where the collision happened for the first time in the game, the below image shows the exact same collision after going back to the menu and restarting the same scene.

As you can see, the first time the ships collide the CollisionDetect() code takes 16.82ms (much more than a single frame time) to execute, and even allocates 366 bytes of memory. However, the exact same code when executed again takes only 0.42ms and allocates no memory. I had been wondering about those strange memory allocations, as my code should not allocate any memory during the time it runs. Okay, it pretty clearly looks like there is some code compilation going on when the collision code is first run. I could get rid of this by porting that code to native C code, but that would be quite a lot of work. I began to wonder if there would be a way to force Unity to compile this code even before it gets executed?

I found an interesting post called "JIT Methods at runtime", which had a technique for compiling the methods of a class before they get executed. I tested this, but sadly it did not seem to work. It gave no errors, but the methods just took as long to execute as before. Then I googled for something similar with Unity, and that finally gave me the answer I was looking for, from Unity Answers. I changed the inner contents of my code to use this GetFunctionPointer() method, and that finally got rid of the large frame drops when the first collision happens!

I added the following code to each of my main classes (this is from the Ships class, I am sure there is a way to get the type of the owning class inside a static method, but I didn't bother hunting for that). I then added a call to all these methods into my loading scene, and got much nicer framerates in the actual game!

    // Precompile all the methods of this class
    public static void Jit()
    {
        var methods = typeof(Ships).GetMethods(System.Reflection.BindingFlags.DeclaredOnly |
                                               System.Reflection.BindingFlags.NonPublic |
                                               System.Reflection.BindingFlags.Public |
                                               System.Reflection.BindingFlags.Instance |
                                               System.Reflection.BindingFlags.Static);
        foreach (var method in methods)
        {
            method.MethodHandle.GetFunctionPointer();
        }
    }

There are still some slight frame drops when a lot of action happens in the game, but that is as expected, and I can now use the profiler to check the actual method time usage, and not get confused by the weird results of the first call of the method. Any frame drops in a cockpit game are quite distracting (unless they happen during a level change or ship teleport or some such situation where a sudden jump is to be expected). I think I will move more code into native code, as that seems to help quite a lot with various performance issues. I have also tested the IL2CPP backend instead of the default Mono backend, but that does not cause a noticeable difference in the frame rates.

Feb 8th, 2019 - Trailer Released, Oculus Start Dev Kit Arrived

Trailer Released!

Okay, a bit of an unscheduled blog post this time, as I just released the first trailer for my game! Yay! I guess making a trailer while the game is still pretty far from finished was a bit premature, but it began to bother me that I had no proper clips or screen shots to show off my game. I have been working on the nitty gritty details for a long time, which began to make me feel like there is nothing working correctly yet in the game. Thus, to help with my motivation to keep working on the game, I decided to take some time off the actual game development and make a trailer, using stuff that actually does already work. With some clever camera work and editing I was able to hide most of the non-working stuff and focus on getting some pretty clips from the gameplay. The trailer even includes some cheesy voice over lines (special thanks to Tom Doughty for the voice!) and a lot of explosions!

Oculus Start Development Kit Arrived!

As I mentioned in an earlier blog post, I got accepted into the Oculus Start program on the 1st of November last year. I had been waiting for my development kit to arrive since that time, and finally on the 28th of January it arrived! Even though the kit had nothing that I absolutely needed (as I already had an Oculus Rift and the Gear VR device), I wanted to test my game on the Oculus Go device (which the kit also included). Now I can make sure my game runs great also on the Oculus Go, and not just on the Gear VR devices. Here is a picture of the development kit contents (the new Oculus Rift is in the cardboard box underneath all the other stuff).

Also, and completely unrelated to LineWars VR: Happy Birthday to my sister!

Jan 20th, 2019 - Pirate Cockpit Work, Music for LineWars VR, and Other Progress

Happy new year, and welcome to a new blog page! The old page where I had written all the blog posts from 2018 had pretty much overgrown, so I decided to split it into two separate pages, and begin a new blog page for posts from this year.

Pirate Cockpit Work

After writing the previous blog post, I continued working on the Pirate ship cockpit. I needed still to do quite a bit of modeling, and even more texturing. I decided to use the same system as I did with the Cobra cockpit where I have two separate meshes, where the major difference is how they use the common texture atlas. One mesh uses the texture atlas as four separate greyscale textures, which allows me to have very detailed legends for the various switches and instruments. This mesh also uses only sharp edges with no round shapes. Then I have another mesh containing all the round shapes and using the same texture atlas as a normal RGB texture. For the Pirate ship cockpit, I also needed a third mesh for the canopy rim, where I used some volumetric rendering techniques as I described in my previous blog post.

I actually needed to revisit my canopy rim volumetric rendering algorithm, as the idea I had about using just a two-dimensional intersection test was actually flawed. When I moved my head sideways (using the 6DOF Oculus Rift I use for development) the canopy rim shape changed weirdly, so obviously my algorithm was not correct. Instead of trying to fix my algorithm, I decided to check how much of a performance hit I would need to take to use a proper ray-cylinder intersection test. I used the intersection test code based on Íñigo Quílez's list of primitive intersection tests, which actually turned out to be cheaper than the code I used originally! This intersection test only needs one sqrt operation while my original code had two of those. I was able to simplify the intersection code quite a bit because my cylinder has one cap at the origin and is oriented along the Z axis.

After the canopy rim was fixed, I then alternated between working on the modeling, texturing, and shadow algorithm work for the rest of the cockpit. The shadow algorithms were somewhat more complex because of the round canopy rim, and also because of the bubble canopy. The cockpit will get shadows also from the pilot's helmet, which obviously should not be rectangular. Because the canopy rim is actually slightly angled relative to the instrument panel, it causes the closer shadow plane to actually intersect the instrument panel around the middle of the panel. This meant I had to split all my instrument panel polygons along this intersecting shadow plane, as my shadow algorithms only work when all the vertices of a triangle are on the same side of the shadow plane.

I still haven't implemented quite all shadow planes I would need (for example the back of the ship should also cause shadows on the instrument panel and on the canopy rim when the sun is very low behind the ship), but most of the shadows already work. The CanopyRim fragment shader (using volumetric rendering and self-shadowing, along with shadows from the pilot's helmet and from the horizontal rim of the cockpit) has performance characteristics like this:

  4 work registers used, 1 uniform registers used, spilling not used.

                          A       L/S     T       Bound
  Instructions Emitted:   33      3       1       A
  Shortest Path Cycles:   1.5     1       0       A
  Longest Path Cycles:    12      3       1       A
So, not quite under the 10 GPU cycles I am aiming at, but as my GPU cycle budget is at up to 20 GPU cycles per fragment, I am still well within that budget. Sadly, this will still get slightly slower when I add the missing shadows. On the other hand, the canopy rim is not very big on the screen, so that helps somewhat.

The current performance of the main Pirate cockpit fragment shader is as follows:

  4 work registers used, 1 uniform registers used, spilling not used.

                          A       L/S     T       Bound
  Instructions Emitted:   32      7       1       A
  Shortest Path Cycles:   2.5     2       1       A
  Longest Path Cycles:    10.5    7       1       A
This too is slightly over the 10 GPU cycles, and it also is missing the shadow from the rear of the ship.

Currently my pirate ship cockpit is at a sort of passable level as far as stuff in front of the pilot is considered. Anything behind the pilot is still quite unfinished, which is also the case with the older Cobra cockpit and the Cruiser bridge. I will next switch to working on some other missing aspects of my game before then doing a new pass over all of my cockpits at some later time. Here below is an animated GIF image showing the current state of the cockpit, along with some dynamic procedural shadows. The round shadow is caused by the helmet of the pilot. I had to make this GIF image greyscale, to get rid of the very bad color banding. That is a normal issue with GIF images, as they can only contain 256 distinct color values.

Music for LineWars VR

In the beginning of this year I thought it might be a good time to finally get some background music for LineWars VR. Back in June of last year I had gotten an offer by freesound.org user AlienXXX for some music, but sadly (for me) he got a new job and became very busy, so that offer sort of fell through. Since that time, I have been very busy with other aspects of my game, so hadn't thought about the music issue. Now I began to think that perhaps I should start hunting for some music. I first checked the good free music site incompetech which I had used in the past for various YouTube videos. I found a couple of pretty good tracks, but nothing that would really cause a "wow!" effect. I was not very aware of other royalty-free music sites, so I asked for help on the Oculus Start discord channel.

I was recommended a site called AudioJungle. The tracks there are not free, but still reasonably priced and royalty-free. I knew I wanted something rather epic-sounding, so I used search terms like "Epic" and "Cinematic" and the like. These pointed me to a category called "Dramatic, Action & Adventure", which I thought was just what I was after. But there were 48,273 tracks in that category! How could I find just the right tracks for me! Luckily there was a search filter called "looped" which I could use, to look for tracks that have parts suitable for looping, which I would need for my background. But there were still over 2400 tracks to choose from!

After I had downloaded preview files for about 10 different tracks, some quite good, I encountered a track called "Epic Adventure Dubstep Trailer". The track was rather weird, but I decided to download the preview anyway. It was an interesting track, though too weird and also too slow for my purposes. I then continued previewing more tracks, until I had found about 30 or 40 pretty neat tracks. However, I noticed I always came back to replaying that weird Dubstep track by Agroglyph and finding I liked it better than any of the other tracks. I decided to contact him, to ask if he could perhaps make a faster version of that track for my game.

We emailed back and forth, until he offered to actually compose new tracks for all my missions, for a price I can afford! That was a good solution for me, as I could get tracks with similar style for all my missions, and could have a say in how the tracks would be structured and instrumented! I believe that is money well spent, as music is an important part of the gaming experience. I had tested my game with some faster preview tracks I had found, and the game felt like a totally different much more polished game with some good music playing in the background!

As I write this, I have already received a preliminary version of the first new track he will be composing for my game, and even that preview version already works very well. I am quite happy to have gotten the music for my game sorted!

Missions 1 to 4 playable!

As I now got background music coming for all of my nine missions, I thought that perhaps I should start actually implementing more of my missions! Until now I had only implemented a mission that I needed for testing some feature (like Mission 3 for testing the Cruiser PDC stuff and teleporting between different ship types). Since I already had missions 1 and 3 running, it was logical to next work on mission 2. This is the "First Blood" mission, where for the first time the player gets to shoot at other ships and also gets shot at. This is a rather simple mission, so it only took around a day to build it, as everything the mission needed was pretty much implemented already. Just change the skybox, some spawn locations and the narration file in the beginning of the mission.

After I got mission 2 done, it was time to work on mission 4. This is called "Pirate Ambush", and it happens inside a dark asteroid field. At first, I thought I will need some help with creating a skybox that would show the asteroid field, but then I decided to try to tackle this one myself. I took my Asteroid object (which is actually just a sphere) with the procedural textures in Cinema 4D, and used an FFD deformer to make it have a more interesting shape. Then I began experimenting with the Array modeling object of Cinema 4D. I created three arrays of different sized asteroids and in different orientations, and then finally put all of these three arrays into another array. The inside arrays have 8 large asteroids, 21 medium-sized and 59 small asteroids, and the overall array creates 7 copies of these three arrays, for a total of 7*(8+21+59) = 616 asteroids in my asteroid field.

I decided to experiment with a Red Dwarf as the star illuminating this asteroid field, using a very deep red color for the sunlight. I think this created a suitably dark feel for this asteroid field. The great thing about my creating the asteroid field skybox using the same object as what I use for the actual in-game asteroids was that you will have a hard time telling the difference between asteroids in the skybox and the real asteroids. In the screen copy below there are 30 real asteroids visible, all the rest are just part of the skybox. Can you spot the real asteroids?

This mission became very entertaining, it is great fun flying between the asteroids chasing and shooting at the enemy ships, while trying to avoid collisions with the asteroids!

Simulated Dynamic Global Illumination

After I had been playing the fourth mission with the very red sunlight for a while, it began to bother me how the illuminated cockpit areas have a very reddish tint, but the areas that are in shadow have no reddish tint at all. In real life the red light would get reflected from the illuminated areas to the cockpit areas in shadow, giving them a red tint as well. This is often called Global Illumination in 3D graphics.

Performing proper accurate Global Illumination would not be possible in real time, but it occurred to me that perhaps I could fake this rather cheaply. I already had a _SunColor uniform vector (with RGBA values) in my cockpit shaders, however the A channel of this vector was not used. I also thought that I could estimate the amount of light getting into the cockpit simply by using my cockpit window shadow planes (as described in my May 20th, 2018 blog post, with the shadow plane image from that blog post copied below). If I used the dot product between the normals of these shadow planes and the sun direction, I should get a quite nice estimate of the amount of light entering the cockpit.

I added code into my cockpit fragment shader to multiply the _SunColor RGB values with the _SunColor.a value and add that to the color of the fragment (both when it is in shadow and when it is lit). I then added code to my C# script to calculate the sum of the dot products for all three shadow planes (the sign is reversed in the code below, because my light vector goes from light towards the cockpit, while the shadow plane normals point away from the cockpit). I then multiply this with a Global Illumination Intensity factor 0.1f (an experimental value for now), and put it into the fourth component of the _SunColor vector.

   float d1 = Vector3.Dot(lightDir, cobra_N1);
   float d2 = Vector3.Dot(lightDir, cobra_N2);
   float d3 = Vector3.Dot(lightDir, cobra_N3);
   float gi = 0f;
   if (d1 < 0)
       gi += -d1;
   if (d2 < 0)
       gi += -d2;
   if (d3 < 0)
       gi += -d3;
   // Setup the sun color, including simulated global illumination.
   Vector4 sunColGI = Movement.SunColor * 6.0f * Movement.ShadowMult;
   sunColGI.w = 0.1f * gi;
   Shader.SetGlobalVector("_SunColor", sunColGI);
If you are wondering what those multipliers of 6.0f and Movement.ShadowMult are, the first is just a global brightness factor (so that I can have non-white sun colors without lessening the sun brightness), and the second one gets zeroed if the cockpit is in the shadow of some other object, like an asteroid or another ship.

This worked reasonably well, but did not take into account any Ambient Occlusion, that is, parts of the cockpit that would not receive much indirect light. At first, I thought this will be a problem I cannot cheaply solve, until I realized that I already have Ambient Occlusion emulation for the ceiling light! This is handled by having a multiplier for each vertex telling how much the ceiling light illuminates that vertex. Since the Global Illumination is not directly related to the Ceiling Light intensity, I could not use the value directly. However, I thought that if I simply decrease the global illumination intensity for those vertices that also have very little illumination from the ceiling light (like the foot wells), the end result might be pretty close to what I want. Thus, I added the following code to my fragment shader:

    // i.lightData.x contains the ceiling light multiplier for the vertex
    fixed4 shCol = tmp > 0.5 ? col : col * i.lightData.x + oculusdither(i.vertex.xy) + tmp * _SunColor * min(_SunColor.a, i.lightData.x);
That is, I multiply the sun color with the minimum of the Global Illumination Intensity from _SunColor.a and the Ceiling Light intensity from i.lightData.x. The tmp value is the (greyscale) color from my cockpit texture.

The cost of this change in the shader was only 0.5 GPU cycles, as the fragment shader cycle count went from 8 to 8.5 after this change. The image below shows a comparison shot with and without this Global Illumination emulation. I think the result is worth the very minor performance hit.

I decided to also make a short video from the Mission 4 I mentioned above. My old pre-alpha gameplay video had gotten rather old already, as it didn't have the trails from the ships, the new more visible lasers, or any music. This new alpha gameplay video should more closely reflect the current state of the game.

That's about it for this blog post, thanks again for your interest in my blog and LineWars VR!

Previous blog posts

See here for blog posts from 2018 and before.