LineWars VR Blog Posts

Mar 31st, 2019 - Status Update

Lots of varied progress during the past month, so I could not figure out a better title to this blog post than a plain "Status Update". Let's get on with the update!

Cruiser PDC Handling to Native Code

As I had been working on improving the performance even before I wrote the previous blog post, and as the new Mission 7 I started work on needed to have several cruisers firing their PDC guns, I decided to check how much of a performance hit does the Cruiser PDC gun moving and firing actually cause. It turned out that the PointPDC code (which determines where to point each PDC gun) took between 0.1 and 0.5 milliseconds, depending on how many guns need moving. That was for a single cruiser, so with several cruisers in the scene that could jump up to several milliseconds. That is rather high, considering that at 72fps everything that happens in a frame needs to fit within about 13.8 milliseconds, including all the rendering that needs to be done. Thus, I decided to port this code over to native C code.

The reason I hadn't ported this code over earlier, was that I needed to access the positions and directions of all the close enemy ships, and I also used a lot of Quaternion routines like LookRotation and RotateTowards in the code. I had to figure out a way to do all that in C code, without accessing the Unity C# helper routines. I decided to use a Matrix to pass the location and direction data of all the ships to the native code. To do that I split my existing ShipData class array into two separate structure arrays, ShipArray for data containing only such structures that can be passed to native code, and ShipData for an array of data consisting of C# class instance pointers. I added Matrix4x4 structures called l2wm (for LocalToWorldMatrix) and w2lm (for WorldToLocalMatrix) into the ShipArray array, so that I can access the positions and directions of all the ships also in the native code. I just need to remember to assign the matrix from the transform of the gameObject once per frame, like this:

    shipArray[idx].l2wm = shipData[idx].gameObject.transform.localToWorldMatrix;
    shipArray[idx].w2lm = shipArray[idx].l2wm.inverse;

Now I could easily transform the ship positions between the local coordinate systems of each of my ships, or specifically transform the world coordinates of the ships into the local coordinate system of my Cruisers in order to aim the PDC guns correctly. For example, when in C# I would write Vector3 outPoint = shipData[idx].gameObject.transform.InverseTransformPoint(inPoint), in my native code I could do the same thing by calling TransformPoint(&outPoint, &inPoint, &(shipArray[idx].w2lm)). I added the following helper functions into my native code to perform such transforms for a point and for a direction vector. I did not handle scale here, as practically all my objects have no scaling:

static inline void TransformPoint(VECTOR3 *o, VECTOR3 *p, MATRIX4X4 *m)
{
    o->x = p->y * m->col[1].x + p->x * m->col[0].x + p->z * m->col[2].x + m->col[3].x;
    o->y = p->y * m->col[1].y + p->x * m->col[0].y + p->z * m->col[2].y + m->col[3].y;
    o->z = p->y * m->col[1].z + p->x * m->col[0].z + p->z * m->col[2].z + m->col[3].z;
}

static inline void TransformDirection(VECTOR3 *o, VECTOR3 *d, MATRIX4X4 *m)
{
    o->x = d->y * m->col[1].x + d->x * m->col[0].x + d->z * m->col[2].x;
    o->y = d->y * m->col[1].y + d->x * m->col[0].y + d->z * m->col[2].y;
    o->z = d->y * m->col[1].z + d->x * m->col[0].z + d->z * m->col[2].z;
}

Okay, that was part of the problem solved, but I still needed to replace the Quaternion operations with something else, to get all my PDC code ported to native C code. I could perhaps have used matrices instead, but I did not want to figure out how to do all those LookRotation and such operations using matrices, so I decided to look into what those Quaternion routines actually do.

The first step was to find out how a vector multiplication by a quaternion works. I found implementations of various Quaternion functions from OGRE3D source codes. These included the vector multiplication, which I then implemented in plain C language for my native code, as follows:

// Based on https://github.com/ehsan/ogre/blob/master/OgreMain/src/OgreQuaternion.cpp
static inline void QuatMul(VECTOR3 *o, QUATERNION *q, VECTOR3 *v)
{
    // nVidia SDK implementation
    VECTOR3 uv, uuv;
    //Vector3 qvec(x, y, z);
    //uv = qvec.crossProduct(v);
    uv.x = q->y * v->z - q->z * v->y;
    uv.y = q->z * v->x - q->x * v->z;
    uv.z = q->x * v->y - q->y * v->x;
    //uuv = qvec.crossProduct(uv);
    uuv.x = q->y * uv.z - q->z * uv.y;
    uuv.y = q->z * uv.x - q->x * uv.z;
    uuv.z = q->x * uv.y - q->y * uv.x;
    //uv *= (2.0f * w);
    //uuv *= 2.0f;
    //return v + uv + uuv;
    o->x = v->x + 2.0 * q->w * uv.x + 2.0 * uuv.x;
    o->y = v->y + 2.0 * q->w * uv.y + 2.0 * uuv.y;
    o->z = v->z + 2.0 * q->w * uv.z + 2.0 * uuv.z;
}

Next, I looked for the LookRotation implementation, and found an algorithm from Unity Answers, where someone had implemented the code in C#. I converted this algorithm to C for my uses. I am not sure if this is the most efficient way to do this, but it seems to work. I might still implement all of these in NEON ASM if needed, to get them to run even faster.

// Implementation of Quaternion.LookRotation, based on https://answers.unity.com/questions/467614/what-is-the-source-code-of-quaternionlookrotation.html
// NOTE! f and u vectors need to be normalized!
static inline void LookRotation(QUATERNION *o, VECTOR3 *f, VECTOR3 *u)
{
    VECTOR3 vector2, vector3;
    float m00, m01, m02, m10, m11, m12, m20, m21, m22, num, num2, num3, num4, num5, num6, num7, num8;

    // Vector3.Cross(up, Vector3.Normalize(forward))
    vector2.x = u->y * f->z - u->z * f->y; 
    vector2.y = u->z * f->x - u->x * f->z; 
    vector2.z = u->x * f->y - u->y * f->x;
    // vector2 = Vector3.Normalize(Vector3.Cross(up, vector));
    m00 = 1.0 / sqrt(V3DOT(vector2, vector2));
    vector2.x *= m00;
    vector2.y *= m00;
    vector2.z *= m00;
    // vector3 = Vector3.Cross(vector, vector2);
    vector3.x = f->y * vector2.z - f->z * vector2.y;
    vector3.y = f->z * vector2.x - f->x * vector2.z;
    vector3.z = f->x * vector2.y - f->y * vector2.x;

    m00 = vector2.x;
    m01 = vector2.y;
    m02 = vector2.z;
    m10 = vector3.x;
    m11 = vector3.y;
    m12 = vector3.z;
    m20 = f->x;
    m21 = f->y;
    m22 = f->z;
 
    num8 = (m00 + m11) + m22;
    if (num8 > 0.0)
    {
        num = sqrt(num8 + 1.0);
        o->w = num * 0.5;
        num = 0.5 / num;
        o->x = (m12 - m21) * num;
        o->y = (m20 - m02) * num;
        o->z = (m01 - m10) * num;
        return;
    }
    if ((m00 >= m11) && (m00 >= m22))
    {
        num7 = sqrt(((1.0 + m00) - m11) - m22);
        num4 = 0.5 / num7;
        o->x = 0.5 * num7;
        o->y = (m01 + m10) * num4;
        o->z = (m02 + m20) * num4;
        o->w = (m12 - m21) * num4;
        return;
    }
    if (m11 > m22)
    {
        num6 = sqrt(((1.0 + m11) - m00) - m22);
        num3 = 0.5 / num6;
        o->x = (m10+ m01) * num3;
        o->y = 0.5 * num6;
        o->z = (m21 + m12) * num3;
        o->w = (m20 - m02) * num3;
        return; 
    }
    num5 = sqrt(((1.0 + m22) - m00) - m11);
    num2 = 0.5 / num5;
    o->x = (m20 + m02) * num2;
    o->y = (m21 + m12) * num2;
    o->z = 0.5 * num5;
    o->w = (m01 - m10) * num2;
}

Another algorithm I needed was the RotateTowards algorithm, to get the PDC gun turrets to slowly rotate towards the enemy ships they are shooting at. I also needed to know the angle difference between the current angle of the gun and the target angle, so I implemented my version of the code to return the angle difference to the caller. Thus, I do not need to calculate this separately, as I had to do in my C# code.

// From http://www.opengl-tutorial.org/intermediate-tutorials/tutorial-17-quaternions/
// The function returns the angle difference between q and q2
static inline float RotateTowards(QUATERNION *o, QUATERNION *q, QUATERNION *q2, float maxAngle)
{
    float cosTheta, angle, fT, t1, t2, t3;
    QUATERNION q1;

    q1.x = q->x;
    q1.y = q->y;
    q1.z = q->z;
    q1.w = q->w;

    //cosTheta = dot(q1, q2);
    cosTheta = q1.w * q2->w + q1.x * q2->x + q1.y * q2->y + q1.z * q2->z;

    // Avoid taking the long path around the sphere
    if (cosTheta < 0)
    {
        q1.x = -q1.x;
        q1.y = -q1.y;
        q1.z = -q1.z;
        q1.w = -q1.w;
        cosTheta = -cosTheta;
    }
    angle = acos(cosTheta);

    if (maxAngle < 0.001)
    {
        o->x = q->x;
        o->y = q->y;
        o->z = q->z;
        o->w = q->w;
        return angle;
    }
    // q1 and q2 are already equal, or the difference is less than we are allowed.
    if (cosTheta > 0.9999 || angle < maxAngle)
    {
        o->x = q2->x;
        o->y = q2->y;
        o->z = q2->z;
        o->w = q2->w;
        return angle;
    }

    fT = maxAngle / angle;

    //quat res = (sin((1.0f - fT) * maxAngle) * q1 + sin(fT * maxAngle) * q2) / sin(maxAngle);
    t1 = sin((1.0 - fT) * maxAngle);
    t2 = sin(fT * maxAngle);
    t3 = 1.0 / sin(maxAngle);
    o->x = (t1 * q1.x + t2 * q2->x) * t3; 
    o->y = (t1 * q1.y + t2 * q2->y) * t3; 
    o->z = (t1 * q1.z + t2 * q2->z) * t3; 
    o->w = (t1 * q1.w + t2 * q2->w) * t3; 

    //res = normalize(res);
    t1 = 1.0 / sqrt(o->x * o->x + o->y * o->y + o->z * o->z + o->w * o->w);
    o->x *= t1;
    o->y *= t1;
    o->z *= t1;
    o->w *= t1;
    return angle;
}

After I had implemented all those, it began to look to me like actually handling the rotation of each of the vertices of the PDC gun mesh would be faster using the TransformDirection() routine using a rotation matrix, instead of doing this using a Quaternion. My understanding has been that Quaternions are used because they are fast, but looking at these implementations it seemed that multiplying a Vector with a Quaternion needs 21 multiplications, while matrix multiplication only needs 9 multiplication operations. So, I decided to add code to convert the Quaternion into a rotation matrix after I had calculated the direction where the gun should point at, and before performing the actual vertex adjustment. This code is also based on the OGRE3D source codes.

// From https://github.com/ehsan/ogre/blob/master/OgreMain/src/OgreQuaternion.cpp
static inline void Quat2Mat(MATRIX4X4 *o, QUATERNION *q)
{
    float fTx  = q->x + q->x;
    float fTy  = q->y + q->y;
    float fTz  = q->z + q->z;
    float fTwx = fTx * q->w;
    float fTwy = fTy * q->w;
    float fTwz = fTz * q->w;
    float fTxx = fTx * q->x;
    float fTxy = fTy * q->x;
    float fTxz = fTz * q->x;
    float fTyy = fTy * q->y;
    float fTyz = fTz * q->y;
    float fTzz = fTz * q->z;

    o->col[0].x = 1.0f-(fTyy+fTzz);
    o->col[1].x = fTxy-fTwz;
    o->col[2].x = fTxz+fTwy;
    o->col[0].y = fTxy+fTwz;
    o->col[1].y = 1.0f-(fTxx+fTzz);
    o->col[2].y = fTyz-fTwx;
    o->col[0].z = fTxz-fTwy;
    o->col[1].z = fTyz+fTwx;
    o->col[2].z = 1.0f-(fTxx+fTyy);
}

Using all of these helper routines I was able to port the whole PDC gun pointing and shooting code over to native code. I timed the native code performance using the Unity profiler, and it gave around 8x performance boost over the C# version. This time the performance improvement was not quite 10x, as much of the inner code was just copying vertex positions from one array to another, which is not much faster in native code.

Oculus Go Performance Issues Continued

I ended my last blog post mentioning how I still suffered some frame drops even after "precompiling" all my scripts. After I got the PDC gun handling ported to native code, I decided to continue troubleshooting these performance issues. I disabled various parts of the code, but could not get rid of the frame drops. It then occurred to me to check what kind of performance End Space (which I consider to be the main competition for my game) has on my Oculus Go. I was pretty surprised to see that even though End Space forces both CPU and GPU levels to as high as they can be set, it still suffers from major frame drops. The frame drops in End Space were much more severe than in my game, even though I used much lower CPU and GPU levels with higher resolution.

As you can see from the above graph (second from top), when running LineWars VR at CPU level 2, I get almost constant frame drops. Looking at the CPU level graph it seems like the dynamic throttling does not work quite as it should. Most of the time the CPU does not dynamically switch to a higher level, even though that might avoid a frame drop. Checking End Space finally made me test how my game behaves with forcing CPU level to 4 (which basically disables dynamic throttling). This got almost completely rid of the frame drops, and visually the gameplay was also very smooth. So, it seems like dynamic throttling just does not play well with my game. Increasing GPU level has practically no effect on the framerate, so I can leave that at the default level of 2.

I decided to use this CPU=4 GPU=2 level combination on Oculus Go from now on. I still want to port the slowest methods over to native code, but it looks like I can already reach quite steady 72fps when using CPU level 4. The device gets a bit warmer than it did on level 2, and the battery consumption increases, but as the gameplay is much smoother, I think this is an acceptable tradeoff.

Mission 7 Work

Okay, now that the performance issues were again mostly solved, I could get back to working on Mission 7. When porting the PDC code over to native code, I had checked all the algorithms I needed to change to handle several cruisers shooting their PDC guns, so now it was time to implement these changes. This worked quite fine, except that now the mission became much too difficult! With the enemy cruisers having their main gun active and all of them shooting their PDC guns, it took only a short while for them to kill all the friendly fighters and also the player's ship! That was no fun.

I decided that the enemy convoy cruisers will not actively use their main guns against the attacking friendlies, instead they rely on the escort fighters and their PDC guns to kill the attackers. This made the mission somewhat easier, but still those PDC guns did short work of the attackers. I decided to test reducing the PDC gun firing speed, and that turned out to be a good way to balance the PDC gun effectiveness. With the PDC fire speed reduced (compared to Mission 3, where the cruiser is friendly) the PDC fire in Mission 7 was survivable enough that the mission is winnable. It does get rather challenging towards the higher levels, with more enemy cruisers and also more escort fighters in the convoy. Since there are only two missions remaining after this one, I think it is OK for this mission to be somewhat challenging.

Alien Mothership

For missions 8 and 9 I needed an Alien Mothership object. I have been thinking about this object for a few months now, but have not been able to come up with anything that I would be happy with. One morning I did yet another Google image search about alien motherships, and this time I finally found an image that gave me an inspiration for the look of my Alien Motherships! I wanted to have a ship design that looks organic and asymmetric, and quite sinister and intimidating. I also decided to make this ship mostly black, with a lot of specular shininess on it. To help with the sense of scale, I added a lot of small windows on the top part of the ship.

The Alien Mothership currently has 1487 vertices and 2106 triangles, which is a bit on the heavy side in terms of collision detection. As I use mesh collisions using a KDtree approach, so many triangles make the corresponding KDTree to have 760 nodes in the KDtree, so it takes a bit too much CPU time to handle the collision detection. As the object is very irregularly shaped, there is even no good way to limit the required collision tests. I will probably attempt to reduce the polygon count of this object still further before releasing the game.

This is what the Alien Mothership currently looks like. It is still missing some texturing, but the overall look is like in this image.

Mission 9 Work

I decided to skip Mission 8 for now and move directly to Mission 9 "The Final Battle". This is a desperate battle to save the Earth from an alien invasion. I have very rarely managed to win this mission in my original LineWars II game, and it should be almost unwinnable also in LineWars VR. But not quite unwinnable, as that would be cheating, in my opinion. In my original game the alien motherships were escorted by Cobra fighters, but I thought that does not make much sense story-wise, so I decided to create some specifically alien escort fighters for this mission.

I decided to create simple black shiny spheres as the alien drone ships. These are very small (currently only 2 meters in radius) and very fast (only a slightly slower than the missiles) and super manoeuverable. In addition to this, they shoot faster than the fighter ships and have no engine trails. Thus, they are very difficult to see (them being black with no trails) and difficult to shoot at (them being faster and much more manoeuverable than your ships). However, they are not very durable, and the PDC guns of the friendly cruisers can track them, especially when they fly straight towards the cruiser. These drones are very annoying to try to shoot at, and I think that suits my goal of making this mission very difficult and desperate-feeling.

What makes these drones even more annoying is that the Alien Motherships launch new drones to replace any killed drones. Thus, in this mission you need to kill (the VERY tough and hard to kill!) Alien Motherships first, if you wish to have any chance of surviving the mission! The Alien Motherships are quite heavily armed themselves, they have lasers that will kill a fighter in one shot, and will severely damage even a cruiser with every shot. In addition to this, the Alien mothership does not need to face the target it is shooting at, unlike the fighter ships and alien drones. Luckily the Alien Motherships only use their lasers for close range defense, so if you stay away from the mothership, it will most likely not shoot at you.

I also worked on the skybox for Mission 9. Since this mission happens around Earth, it should have the Earth in the background, and I also wanted to have the correct star map (as seen from Earth) in the skybox. This will of course not look quite as nice as some colorful nebulas I have in other missions, but perhaps such dark background increases the darkness and desperation of this mission. I would like to have both the Earth and the Moon in the skybox, but if I use realistic sizes of them, either one or both of them would be pretty small. Perhaps I will exaggerate the size of the Moon to make it look nicer in the skybox.

All in all, Mission 9 turned out to be very action-packed, with several cruisers shooting their PDC guns, and dozens of drones and around 15 friendly fighters mixing it up in a large furball around the cruisers. Quite like what I was aiming for with this mission! I tried to take some screen shots, but as static screenshots don't show the action at all, they do not look very good. I also don't want to spoil the mission by posting a video of it, so I don't have anything to show for this mission at the moment, sorry.

An Article About Me and LineWars VR in a Local Newspaper!

The local newspaper Keskisuomalainen wrote an article about me and my LineWars VR game. The article came out on the 27th of March. That was some nice free publicity for my game, even though the audience was somewhat limited, the newspaper being a local one. My game is targeted to global audience, but any publicity is good publicity, as they say! There was even a small picture on the front page of the newspaper, which was a nice bonus!

Onko tässä seuraava virtuaalipelihitti? Jyväskyläläisen Patrick Aallon, 52, LineWars VR valmistuu pala palalta

That's it for this blog post! Thanks again for your interest in my game and in this blog! I will continue fine-tuning Mission 9, then work on Mission 8, and after that I should be able to focus on the multiplayer features. Then there are a lot of bugs to fix and minor improvements to texturing and other aspects of the game to do before I can get it finished, so the release is still several months away.

Mar 3rd, 2019 - Cruiser Death Handling, Multiplayer Work, Oculus Go Performance Issues

Cruiser Death handling

After I got the trailer released, I began working on Mission 7, where the goal is to destroy an enemy convoy consisting of several cruisers. I could not get very far with this mission, as there was no way yet to kill the cruisers. You could only damage them, but there was no cruiser death handling implemented yet. Thus, in order to get further in my game, I needed to implement a death handling for a cruiser.

I first started experimenting with some kind of an explosion when the cruiser dies, something like what I had for the asteroids. However, this seemed quite difficult to get looking in any way convincing, and I also had trouble figuring out how the exploded fragments would then vanish from the scene. With the asteroid it is sort of expected that the fragments crumble and thus get smaller and smaller and then vanish, but why would some exploded parts of a ship do that? If I did not make the fragments disappear, though, I would need to keep them alive for the duration of the scene, and that would make them also need collision handling, you should be able to shoot at them, and so on. This seemed like a lot of work.

In the end I decided to have the cruiser just explode into two parts, so it kind of gets cut in half. This way I only needed to model two fragments, and I decided it would be OK if these parts would just stay in the scene tumbling, and you could actually collide with these tumbling hulks. I used the fire animation on most of the body panels of these destroyed parts, so they look like the remnants of a big explosion that cut the ship in half. I still need to fix the texturing (and perhaps also the modeling) of the area closest to where the explosion happened, but other than that, the parts seem to work okay.

I still need to work on improving my PDC gun handling to allow several cruisers in the scene, for the Mission 7 to work properly. Currently I only have one array for cruiser PDC targets, but when there are several cruisers in the scene, these all should have their own targets for their PDCs.

Multiplayer Work

I also decided it was high time for me to start at least looking into the multiplayer features of LineWars VR. As it is rather difficult to get started with such a major feature, I thought I'd start simple, and first look into getting VoIP working between two players.

I looked into the Oculus VrVoiceChat sample that is included in the Oculus Unity Integration package. It seemed to suit my needs pretty well, so I implemented a similar Oculus Rooms -based system, where you can invite a friend that also has LineWars VR into a room that you created, and then a VoIP connection is automatically created between the two players. I am having some issues with this yet, and it seems like there are some issues with the Oculus forums as well, as many replies to similar questions seem to be missing. For example How to remove room invites forcefully. Anyways, the issues I am currently having are:

Working around the above issues, though, allowed me to have a VoIP conversation with a friend while we both played the game. The next step is obviously to add some networking code to allow the players to play in the same game. I planned to start working on this next, however, the performance problems I had been having with my Oculus Go made me want to look into those first.

Oculus Go Performance Issues

Since I received the Oculus Start Development Kit early last month, I have been testing my game on the Oculus Go. I was expecting Oculus Go to be in many ways better than my Samsung Galaxy S6 using the Gear VR device, however it seemed the opposite was true. This is what my impressions with Oculus Go were, compared to my Gear VR:

I had assumed the minor frame drops I had with my Galaxy S6 would all vanish when running on Galaxy S7 or faster devices (including Oculus Go, which has the internals of Galaxy S7), but it actually performed much worse.

I actually spent about two weeks fighting with the performance issues, which did not seem to make much sense. I got big frame drops even in the main menu of the game, which only had a few buttons and nothing else. I even tested with hiding everything from the main menu, except the camera and the skybox, and STILL I got constant dropped frames, as in the following graph!

In the above image you can see the graphs from my loading screen and then the main menu. The loading screen looks rather normal, with the constant CPU load causing the framerate to drop to 70fps, and the GPU not doing much of anything. When the main menu begins, though, there are occasional CPU jumps from the base level of 60% usage to about 65% usage or above, and each of these jumps seems to immediately cause frame drops! I just could not figure out how such a small CPU usage increase would immediately drop frames, and what would even cause such periodic CPU usage increase when just sitting in the main menu?!

After a few days of not getting any further with the frame drops in the main menu, I decided to focus on the actual game instead. I used the profiler to look into code that takes a lot of time to execute. Here also there were some very strange results, where a simple code that normally takes almost no time suddenly took several milliseconds to run. This below is what the average frame rate graph looked like when playing the actual game.

After adding some UnityEngine.Profiling.Profiler.BeginSample() calls into my code, I noticed that the code that handles the cockpit instruments seemed to sometimes take quite a lot of time to run. When digging deeper into this code, it turned out that the problem was the CrosshairsCheck code, which sends a ray forward from the player's ship, and checks what the ray hits. I am using my own KDTree implementation for the ray-mesh intersection tests, and when the target was a space station (my most complex object), this test took a lot of time (0.80ms or over 80% of all the time spent in all my scripts) even on the PC, which is about 10 times faster than the Oculus Go! That was not good.

I realized that the KDHit test routine I had written in C# code was a good candidate for porting over to native code. I did that, and after that change this test took less than 0.1ms to run on the Oculus Go. This did not fix my main performance issues, though.

After a lot more tests it began to look like every time I activated a new game object, there was a frame drop. Thus, I began changing my object pools to not actually contain disabled GameObjects, but instead have all the GameObjects active, but just scaled to zero. This of course increases the number of draw calls. Since I do not seem to be bound by anything graphics related, instead by some Unity GameObject activation-related issue, I thought this might be a good trade-off. While I was doing this, I also decided to switch my "AsteroidHit" handling (which actually does not have much to do with asteroids, it is just an animated sprite that displays a flash and some ejecta from any kind of hit to a ship hull or asteroid) to use native code. I created just a single GameObject to contain 20 separate square "panels", each by default having a zero size. These panels can then be located in world coordinates where such a hit animation needs to be shown at, and rotated and scaled to face the camera. Thus, I did not need to have any GameObject activation when such a hit happens. This improved the frame drops somewhat, but still many frame drops remained.

Finally, after I had fought with this problem for almost two weeks, it occurred to me to check what would happen if I switched my background music from "Streaming" to "Compressed in memory" in the Unity audio clip settings. To my surprise, this finally got rid of the periodic frame drops in the main menu, and also improved the frame drops in game quite noticeably! I had assumed that the background music streaming would happen in a separate thread, so it had not even occurred to me that playing music could be the reason for my frame drops! After this change, and including some (but not all) of my GameObject activation changes, I got the following frame rate graph (showing the loading screen, main menu, and start of a mission, including a level change). So, not quite a straight line yet in the actual mission, but much better than what I had managed before.

I still had an issue of an occasional sudden major frame drop. I began to suspect this was some kind of Just-In-Time compiler issue, where the first run of any code takes a lot longer than subsequent runs. This would be a problem in my game, as there are a lot of complex code that gets executed when something specific (like a collision) happens. If the first collision always causes frame drops, that would be pretty bad. I decided to properly test this theory, using my Mission 9 scene which I had not finished yet. I made the scene contain only three ships, which fly towards each other at their maximum speed, so they all collide with each other at almost the same time. I then connected the profiler to my Oculus Go and ran this test scene.

The above image from the profiler shows the frame where the collision happened for the first time in the game, the below image shows the exact same collision after going back to the menu and restarting the same scene.

As you can see, the first time the ships collide the CollisionDetect() code takes 16.82ms (much more than a single frame time) to execute, and even allocates 366 bytes of memory. However, the exact same code when executed again takes only 0.42ms and allocates no memory. I had been wondering about those strange memory allocations, as my code should not allocate any memory during the time it runs. Okay, it pretty clearly looks like there is some code compilation going on when the collision code is first run. I could get rid of this by porting that code to native C code, but that would be quite a lot of work. I began to wonder if there would be a way to force Unity to compile this code even before it gets executed?

I found an interesting post called "JIT Methods at runtime", which had a technique for compiling the methods of a class before they get executed. I tested this, but sadly it did not seem to work. It gave no errors, but the methods just took as long to execute as before. Then I googled for something similar with Unity, and that finally gave me the answer I was looking for, from Unity Answers. I changed the inner contents of my code to use this GetFunctionPointer() method, and that finally got rid of the large frame drops when the first collision happens!

I added the following code to each of my main classes (this is from the Ships class, I am sure there is a way to get the type of the owning class inside a static method, but I didn't bother hunting for that). I then added a call to all these methods into my loading scene, and got much nicer framerates in the actual game!

    // Precompile all the methods of this class
    public static void Jit()
    {
        var methods = typeof(Ships).GetMethods(System.Reflection.BindingFlags.DeclaredOnly |
                                               System.Reflection.BindingFlags.NonPublic |
                                               System.Reflection.BindingFlags.Public |
                                               System.Reflection.BindingFlags.Instance |
                                               System.Reflection.BindingFlags.Static);
        foreach (var method in methods)
        {
            method.MethodHandle.GetFunctionPointer();
        }
    }

There are still some slight frame drops when a lot of action happens in the game, but that is as expected, and I can now use the profiler to check the actual method time usage, and not get confused by the weird results of the first call of the method. Any frame drops in a cockpit game are quite distracting (unless they happen during a level change or ship teleport or some such situation where a sudden jump is to be expected). I think I will move more code into native code, as that seems to help quite a lot with various performance issues. I have also tested the IL2CPP backend instead of the default Mono backend, but that does not cause a noticeable difference in the frame rates.

Feb 8th, 2019 - Trailer Released, Oculus Start Dev Kit Arrived

Trailer Released!

Okay, a bit of an unscheduled blog post this time, as I just released the first trailer for my game! Yay! I guess making a trailer while the game is still pretty far from finished was a bit premature, but it began to bother me that I had no proper clips or screen shots to show off my game. I have been working on the nitty gritty details for a long time, which began to make me feel like there is nothing working correctly yet in the game. Thus, to help with my motivation to keep working on the game, I decided to take some time off the actual game development and make a trailer, using stuff that actually does already work. With some clever camera work and editing I was able to hide most of the non-working stuff and focus on getting some pretty clips from the gameplay. The trailer even includes some cheesy voice over lines (special thanks to Tom Doughty for the voice!) and a lot of explosions!

Oculus Start Development Kit Arrived!

As I mentioned in an earlier blog post, I got accepted into the Oculus Start program on the 1st of November last year. I had been waiting for my development kit to arrive since that time, and finally on the 28th of January it arrived! Even though the kit had nothing that I absolutely needed (as I already had an Oculus Rift and the Gear VR device), I wanted to test my game on the Oculus Go device (which the kit also included). Now I can make sure my game runs great also on the Oculus Go, and not just on the Gear VR devices. Here is a picture of the development kit contents (the new Oculus Rift is in the cardboard box underneath all the other stuff).

Also, and completely unrelated to LineWars VR: Happy Birthday to my sister!

Jan 20th, 2019 - Pirate Cockpit Work, Music for LineWars VR, and Other Progress

Happy new year, and welcome to a new blog page! The old page where I had written all the blog posts from 2018 had pretty much overgrown, so I decided to split it into two separate pages, and begin a new blog page for posts from this year.

Pirate Cockpit Work

After writing the previous blog post, I continued working on the Pirate ship cockpit. I needed still to do quite a bit of modeling, and even more texturing. I decided to use the same system as I did with the Cobra cockpit where I have two separate meshes, where the major difference is how they use the common texture atlas. One mesh uses the texture atlas as four separate greyscale textures, which allows me to have very detailed legends for the various switches and instruments. This mesh also uses only sharp edges with no round shapes. Then I have another mesh containing all the round shapes and using the same texture atlas as a normal RGB texture. For the Pirate ship cockpit, I also needed a third mesh for the canopy rim, where I used some volumetric rendering techniques as I described in my previous blog post.

I actually needed to revisit my canopy rim volumetric rendering algorithm, as the idea I had about using just a two-dimensional intersection test was actually flawed. When I moved my head sideways (using the 6DOF Oculus Rift I use for development) the canopy rim shape changed weirdly, so obviously my algorithm was not correct. Instead of trying to fix my algorithm, I decided to check how much of a performance hit I would need to take to use a proper ray-cylinder intersection test. I used the intersection test code based on Íñigo Quílez's list of primitive intersection tests, which actually turned out to be cheaper than the code I used originally! This intersection test only needs one sqrt operation while my original code had two of those. I was able to simplify the intersection code quite a bit because my cylinder has one cap at the origin and is oriented along the Z axis.

After the canopy rim was fixed, I then alternated between working on the modeling, texturing, and shadow algorithm work for the rest of the cockpit. The shadow algorithms were somewhat more complex because of the round canopy rim, and also because of the bubble canopy. The cockpit will get shadows also from the pilot's helmet, which obviously should not be rectangular. Because the canopy rim is actually slightly angled relative to the instrument panel, it causes the closer shadow plane to actually intersect the instrument panel around the middle of the panel. This meant I had to split all my instrument panel polygons along this intersecting shadow plane, as my shadow algorithms only work when all the vertices of a triangle are on the same side of the shadow plane.

I still haven't implemented quite all shadow planes I would need (for example the back of the ship should also cause shadows on the instrument panel and on the canopy rim when the sun is very low behind the ship), but most of the shadows already work. The CanopyRim fragment shader (using volumetric rendering and self-shadowing, along with shadows from the pilot's helmet and from the horizontal rim of the cockpit) has performance characteristics like this:

  4 work registers used, 1 uniform registers used, spilling not used.

                          A       L/S     T       Bound
  Instructions Emitted:   33      3       1       A
  Shortest Path Cycles:   1.5     1       0       A
  Longest Path Cycles:    12      3       1       A
So, not quite under the 10 GPU cycles I am aiming at, but as my GPU cycle budget is at up to 20 GPU cycles per fragment, I am still well within that budget. Sadly, this will still get slightly slower when I add the missing shadows. On the other hand, the canopy rim is not very big on the screen, so that helps somewhat.

The current performance of the main Pirate cockpit fragment shader is as follows:

  4 work registers used, 1 uniform registers used, spilling not used.

                          A       L/S     T       Bound
  Instructions Emitted:   32      7       1       A
  Shortest Path Cycles:   2.5     2       1       A
  Longest Path Cycles:    10.5    7       1       A
This too is slightly over the 10 GPU cycles, and it also is missing the shadow from the rear of the ship.

Currently my pirate ship cockpit is at a sort of passable level as far as stuff in front of the pilot is considered. Anything behind the pilot is still quite unfinished, which is also the case with the older Cobra cockpit and the Cruiser bridge. I will next switch to working on some other missing aspects of my game before then doing a new pass over all of my cockpits at some later time. Here below is an animated GIF image showing the current state of the cockpit, along with some dynamic procedural shadows. The round shadow is caused by the helmet of the pilot. I had to make this GIF image greyscale, to get rid of the very bad color banding. That is a normal issue with GIF images, as they can only contain 256 distinct color values.

Music for LineWars VR

In the beginning of this year I thought it might be a good time to finally get some background music for LineWars VR. Back in June of last year I had gotten an offer by freesound.org user AlienXXX for some music, but sadly (for me) he got a new job and became very busy, so that offer sort of fell through. Since that time, I have been very busy with other aspects of my game, so hadn't thought about the music issue. Now I began to think that perhaps I should start hunting for some music. I first checked the good free music site incompetech which I had used in the past for various YouTube videos. I found a couple of pretty good tracks, but nothing that would really cause a "wow!" effect. I was not very aware of other royalty-free music sites, so I asked for help on the Oculus Start discord channel.

I was recommended a site called AudioJungle. The tracks there are not free, but still reasonably priced and royalty-free. I knew I wanted something rather epic-sounding, so I used search terms like "Epic" and "Cinematic" and the like. These pointed me to a category called "Dramatic, Action & Adventure", which I thought was just what I was after. But there were 48,273 tracks in that category! How could I find just the right tracks for me! Luckily there was a search filter called "looped" which I could use, to look for tracks that have parts suitable for looping, which I would need for my background. But there were still over 2400 tracks to choose from!

After I had downloaded preview files for about 10 different tracks, some quite good, I encountered a track called "Epic Adventure Dubstep Trailer". The track was rather weird, but I decided to download the preview anyway. It was an interesting track, though too weird and also too slow for my purposes. I then continued previewing more tracks, until I had found about 30 or 40 pretty neat tracks. However, I noticed I always came back to replaying that weird Dubstep track by Agroglyph and finding I liked it better than any of the other tracks. I decided to contact him, to ask if he could perhaps make a faster version of that track for my game.

We emailed back and forth, until he offered to actually compose new tracks for all my missions, for a price I can afford! That was a good solution for me, as I could get tracks with similar style for all my missions, and could have a say in how the tracks would be structured and instrumented! I believe that is money well spent, as music is an important part of the gaming experience. I had tested my game with some faster preview tracks I had found, and the game felt like a totally different much more polished game with some good music playing in the background!

As I write this, I have already received a preliminary version of the first new track he will be composing for my game, and even that preview version already works very well. I am quite happy to have gotten the music for my game sorted!

Missions 1 to 4 playable!

As I now got background music coming for all of my nine missions, I thought that perhaps I should start actually implementing more of my missions! Until now I had only implemented a mission that I needed for testing some feature (like Mission 3 for testing the Cruiser PDC stuff and teleporting between different ship types). Since I already had missions 1 and 3 running, it was logical to next work on mission 2. This is the "First Blood" mission, where for the first time the player gets to shoot at other ships and also gets shot at. This is a rather simple mission, so it only took around a day to build it, as everything the mission needed was pretty much implemented already. Just change the skybox, some spawn locations and the narration file in the beginning of the mission.

After I got mission 2 done, it was time to work on mission 4. This is called "Pirate Ambush", and it happens inside a dark asteroid field. At first, I thought I will need some help with creating a skybox that would show the asteroid field, but then I decided to try to tackle this one myself. I took my Asteroid object (which is actually just a sphere) with the procedural textures in Cinema 4D, and used an FFD deformer to make it have a more interesting shape. Then I began experimenting with the Array modeling object of Cinema 4D. I created three arrays of different sized asteroids and in different orientations, and then finally put all of these three arrays into another array. The inside arrays have 8 large asteroids, 21 medium-sized and 59 small asteroids, and the overall array creates 7 copies of these three arrays, for a total of 7*(8+21+59) = 616 asteroids in my asteroid field.

I decided to experiment with a Red Dwarf as the star illuminating this asteroid field, using a very deep red color for the sunlight. I think this created a suitably dark feel for this asteroid field. The great thing about my creating the asteroid field skybox using the same object as what I use for the actual in-game asteroids was that you will have a hard time telling the difference between asteroids in the skybox and the real asteroids. In the screen copy below there are 30 real asteroids visible, all the rest are just part of the skybox. Can you spot the real asteroids?

This mission became very entertaining, it is great fun flying between the asteroids chasing and shooting at the enemy ships, while trying to avoid collisions with the asteroids!

Simulated Dynamic Global Illumination

After I had been playing the fourth mission with the very red sunlight for a while, it began to bother me how the illuminated cockpit areas have a very reddish tint, but the areas that are in shadow have no reddish tint at all. In real life the red light would get reflected from the illuminated areas to the cockpit areas in shadow, giving them a red tint as well. This is often called Global Illumination in 3D graphics.

Performing proper accurate Global Illumination would not be possible in real time, but it occurred to me that perhaps I could fake this rather cheaply. I already had a _SunColor uniform vector (with RGBA values) in my cockpit shaders, however the A channel of this vector was not used. I also thought that I could estimate the amount of light getting into the cockpit simply by using my cockpit window shadow planes (as described in my May 20th, 2018 blog post, with the shadow plane image from that blog post copied below). If I used the dot product between the normals of these shadow planes and the sun direction, I should get a quite nice estimate of the amount of light entering the cockpit.

I added code into my cockpit fragment shader to multiply the _SunColor RGB values with the _SunColor.a value and add that to the color of the fragment (both when it is in shadow and when it is lit). I then added code to my C# script to calculate the sum of the dot products for all three shadow planes (the sign is reversed in the code below, because my light vector goes from light towards the cockpit, while the shadow plane normals point away from the cockpit). I then multiply this with a Global Illumination Intensity factor 0.1f (an experimental value for now), and put it into the fourth component of the _SunColor vector.

   float d1 = Vector3.Dot(lightDir, cobra_N1);
   float d2 = Vector3.Dot(lightDir, cobra_N2);
   float d3 = Vector3.Dot(lightDir, cobra_N3);
   float gi = 0f;
   if (d1 < 0)
       gi += -d1;
   if (d2 < 0)
       gi += -d2;
   if (d3 < 0)
       gi += -d3;
   // Setup the sun color, including simulated global illumination.
   Vector4 sunColGI = Movement.SunColor * 6.0f * Movement.ShadowMult;
   sunColGI.w = 0.1f * gi;
   Shader.SetGlobalVector("_SunColor", sunColGI);
If you are wondering what those multipliers of 6.0f and Movement.ShadowMult are, the first is just a global brightness factor (so that I can have non-white sun colors without lessening the sun brightness), and the second one gets zeroed if the cockpit is in the shadow of some other object, like an asteroid or another ship.

This worked reasonably well, but did not take into account any Ambient Occlusion, that is, parts of the cockpit that would not receive much indirect light. At first, I thought this will be a problem I cannot cheaply solve, until I realized that I already have Ambient Occlusion emulation for the ceiling light! This is handled by having a multiplier for each vertex telling how much the ceiling light illuminates that vertex. Since the Global Illumination is not directly related to the Ceiling Light intensity, I could not use the value directly. However, I thought that if I simply decrease the global illumination intensity for those vertices that also have very little illumination from the ceiling light (like the foot wells), the end result might be pretty close to what I want. Thus, I added the following code to my fragment shader:

    // i.lightData.x contains the ceiling light multiplier for the vertex
    fixed4 shCol = tmp > 0.5 ? col : col * i.lightData.x + oculusdither(i.vertex.xy) + tmp * _SunColor * min(_SunColor.a, i.lightData.x);
That is, I multiply the sun color with the minimum of the Global Illumination Intensity from _SunColor.a and the Ceiling Light intensity from i.lightData.x. The tmp value is the (greyscale) color from my cockpit texture.

The cost of this change in the shader was only 0.5 GPU cycles, as the fragment shader cycle count went from 8 to 8.5 after this change. The image below shows a comparison shot with and without this Global Illumination emulation. I think the result is worth the very minor performance hit.

I decided to also make a short video from the Mission 4 I mentioned above. My old pre-alpha gameplay video had gotten rather old already, as it didn't have the trails from the ships, the new more visible lasers, or any music. This new alpha gameplay video should more closely reflect the current state of the game.

That's about it for this blog post, thanks again for your interest in my blog and LineWars VR!

Previous blog posts

See here for blog posts from 2018 and before.