Android M Runtime Permission Issue in Unity Game Development

关于有史以来遇见过的最麻烦的游戏bug: Android M Permission

问题描述:

Android M 允许玩家在游戏安装后,修改游戏访问设备的权限。当玩家禁止游戏访问设备摄像头之后,游戏中的AR scene 依旧可以照常运行并不显示任何提示信息。

具体权限访问改变参考此处:https://blog.xamarin.com/requesting-runtime-permissions-in-android-marshmallow/

经过调查,游戏AR scene 的错误监控系统来自 Vufornia 的自带错误监控系统。然而,糟糕的是,Vufoirnia 并没有兼容 Android 的最新版本。每次AR scene 初始化时,访问的 Init 信息为 success。 这意味着,我必须自创一种检测方式,来检测游戏是否有权限访问设备的摄像头。

最开始,我尝试了两种方法:WebCamTexture (http://docs.unity3d.com/ScriptReference/WebCamTexture.html) 和 CameraDevice( https://developer.vuforia.com/library/sites/default/api/unity/classVuforia_1_1CameraDevice.html)。

前者是 Unity 提供的 api,后者是 Vuforinia 提供的对象。我本以为,如果摄像头访问权限受限后,api 返回的 active device 数目应该为 0。但是,实际返回对象数目依旧是 2 (前置和后置摄像头)。其次,我试图通过摄像头的各种参数,来区分权限访问情况。但是,无论是检测 Camera 的 buffer 是否在上一帧有更新,或者是检查 Camera 的 field of view,即使 permission 被禁止了,这些信息依旧可以被访问。

于是,我不得不把方向转向 Android M 的常规解决办法。http://jijiaxin89.com/2015/08/30/Android-s-Runtime-Permission/ 和 https://developer.android.com/training/permissions/requesting.html

这里的关键 api 是 checkSelfPermission。要调用该 api,最直接的办法是,创建自己的 jar,并添加到项目中。之后再用 C# 代码调用。但是,还有一种更简单的办法,即用 Unity 提供的 api 直接调用 Android 函数。方法在此:http://docs.unity3d.com/ScriptReference/AndroidJavaClass.html

我们既可以通过 AndrpidJavaClass 调用 static 函数,可以通过 AndroidJavaObject 调用 non-static 函数。以下是我的测试代码:

  1.             AndroidJavaClass jc = new AndroidJavaClass(“com.unity3d.player.UnityPlayer”);
  2.             AndroidJavaObject activity = jc.GetStatic<AndroidJavaObject>(“currentActivity”);
  3.             AndroidJavaObject context = activity.Call<AndroidJavaObject>(“getApplicationContext”);
  4.             AndroidJavaObject ContextCompat = new AndroidJavaObject(“android.support.v4.content.ContextCompat”);
  5.             AndroidJavaClass ActivityCompatClass = new AndroidJavaClass(“android.support.v4.app.ActivityCompat”);
  6.             int hasCameraPermission = context.Call<int>(“checkSelfPermission”, “android.permission.CAMERA” );
  7.             ActivityCompatClass.CallStatic(“requestPermissions”, context, “android.permission.CAMERA”, 1000 );
  8.             //int hasCameraPermission = ContextCompat.Call<int>(“checkSelfPermission”, context, “android.permission.CAMERA” );

但是糟糕的是,如果我用 ContextCompat.checkSelfPermission 调用 static 函数,程序将会报错显示无法找到该 static 函数。如果我使用 context.checkSelfPermission 调用 non-static 函数,虽然 method 会被调用,但是返回的值永远都是 0 (granted)。

后来经过再三检查,发现其问题在于我们使用的 library 不是最新版本,无法支持 Android api 23 及以上的版本。如果更新 lib 的话,会花费相当多的时间去清理和调整。

最后,我们尝试更改 Manifest 中的 targetVersion 来禁止用户修改游戏的访问权限。但是,这种方法也失败了。原因在于 targetVersion 的具体意思是指告知设备该程序已经在通过 targetVersion 的目标版本测试。然而我们的游戏实际并不满足目标版本,但是设备依然会按照应有的方式处理我们的程序。详细见此:

http://www.cnblogs.com/popapa/p/android_uses-sdk-element.html
http://developer.android.com/guide/topics/manifest/uses-sdk-element.html

最后是网上另外两名用户的解决办法: http://blog.trsquarelab.com/2015/11/  和 http://stackoverflow.com/questions/35027043/implementing-android-6-0-permissions-in-unity3d/

 

Unity: Gimbal Lock Again

I have to say, I have heard the word “gimbal lock” tons of time before. But recently a gimbal lock bug in Unity definitely reminded me of how naive I was about this problem. Actually, gimbal lock problem should not be seen in Unity. However, this is not always true.

What is gimbal lock?

Gimbal lock is the loss of one degree of freedom in a three-dimensional, three-gimbal mechanism that occurs when the axes of two of the three gimbals are driven into a parallel configuration, “locking” the system into rotation in a degenerate two-dimensional space.

This problem is frequently seen when we use euler angle for game object’s rotation, as one axis is set to 90 (in Unity axis order, this axis is x), then we lost one dimension. You could easily reproduce this problem in Unity Editor by set one game object’s rotation as (90, 0, 0). Then no matter how you change the value for y or z, the game object would only rotate in one dimension.

Detailed Information about Gimabl Lock

How Unity deals with euler?

Later, I checked the source code of Unity API. Here is what I found:

For transform.eulerAngles

public Vector3 eulerAngles
{
      get
      {
        return this.rotation.eulerAngles;
      }
      set
      {
        this.rotation = Quaternion.Euler(value);
      }
}

For transform.Rotate

public void Rotate(Vector3 eulerAngles, [DefaultValue("Space.Self")] Space relativeTo)
    {
      Quaternion quaternion = Quaternion.Euler(eulerAngles.x, eulerAngles.y, eulerAngles.z);
      if (relativeTo == Space.Self)
        this.localRotation = this.localRotation * quaternion;
      else
        this.rotation = this.rotation * Quaternion.Inverse(this.rotation) * quaternion * this.rotation;
    }

It is obvious that Unity internally uses quaternion instead of euler. In fact, euler is just the data showing to users. According to resources online, people believe gimbal lock could be solved by using quaternion. But now, I have doubt about it.

We could make a simple experiment like what two bloggers did here:

Candycat1992 made this test:

transform.rotation = Quaternion.Identity;
transform.Rotate(new Vector3(0, 0, 40));  
transform.Rotate(new Vector3(0, 90, 0));  
transform.Rotate(new Vector3(80, 0, 0)); 

And Scott did this:

void Start () { 
		     newDirection = new Quaternion();
		     newDirection  = Quaternion.Euler(-90, 45, 0);
     //newDirection  = Quaternion.Euler(-90, 0, 45);
	}
 
	void Update () { 
		     transform.rotation = Quaternion.Lerp(transform.rotation, newDirection, Time.deltaTime);
}

And gimbal lock appeared! But wait, how could quaternion fail to avoid gimbal lock?

Then I made my own test:

  • Set the rotation of the target object as (90, 0, 0)
  • Call transform.Rotate( 0, rate, 0 )
  • Call transform.Rotate( 0, 0, rate )

Gimbal lock never happens.

So what actually happened?

There are two concepts I misunderstood:

  1. Unity rotation order: In Unity, the order of rotation is ZXY. So if I set the original rotation of the object as (90, 0, 0), and then transform.Rotate(0, rate, 0) and transform.Rotate(0, 0, rate). It would not generate the gimbal lock. Because here it rotates in the order of XYZ, as rotation in X would not make any affect. But, if we call transform.Rotate( 90, y, z) directly, Unity would rotate in order of ZXY. In this way, it would generate gimbal lock.
  2. The way to use Quaternion: Quaternion could avoid gimbal lock, which is true. But remember, “could avoid” doesn’t mean always avoid. We could consider Quaternion as four dimension, comparing to three dimension of Euler, it has an extra dimension to avoid gimbal lock. However, if you rotate it the same as Euler axes, Quaternion does the same as Euler.  Therefore, we should be very careful about the way we use Quaternion.

Quaternion and gimbal lock is not easy to understand. So far I only know what went wrong, and I need to do more research about Quaternion and gimbal lock.

Unity UI adjustment based on different screen resolution

There are three ways which we could use to adjust UI elements based on different screen resolution:

  1. Surrounding fill out
  2. Relative size and fixed size
  3. Two patterns

Surrounding fill out

For surrounding fill out, it means that all game elements are placed in the center of the screen, and fill out rest of the screen space by using other elements, like merely black color. Here is the example from “Fallout:Shelter”:

16:9 ( for iphone5)

Screen Shot 2015-08-17 at 11.29.16 AM.png

4:3 ( for ipad)

Screen Shot 2015-08-17 at 11.28.19 AM.png

You can see, all UI elements are places in the center square. No matter what screen resolution it is, each UI element stays in the same position, and rest of space whether be cut (4:3) or filled in black (16:9). It is obviously a smart solution. Meanwhile, it presents a clean style for users. Personally I like this solution the most.

Pros:

  • Easy to implement
  • Clean style, looks perfect.

Cons:

  • All UI elements must be in the center
  • All UI elements’ size could not be adjusted

Summary

If there are a few interactive UI elements in the game scene, and each element’s size is not so small (like button), I suggest you use this method to solve UI adjustment issue.

Relative size and fixed size

For relative size and fixed size, UI element’s size and position are set as relative or fixed. Usually, the size is set as relative, and position is set as fixed. In Unity, we could use NGUI stretch and NGUI anchor to achieve it. In general, we want UI element’s size would be automatically adjusted according to screen size, and UI element’s position stays the same as resolution changes in order to make layout unchanged. Here is the example from Dead Trigger 2:

16:9 (iphone 5)

Screen Shot 2015-08-17 at 11.29.34 AM.png

4:3 ( ipad)

Screen Shot 2015-08-17 at 11.28.39 AM.png

It is clear that all buttons stay in the same position, as they use the same anchor target. If you look at green button on the right, you could notice that the distance between it and screen border is the same for 16:9 resolution and 4:3 resolution. Moreover, the iphone version has relatively bigger buttons as  we want player are able to tap them easily on iphone device. And it could be achieved by using NGUI stretch to set button’s relative size.

Pros:

  • Keep UI layout unchanged
  • UI elements would be not places in a certain place of the screen

Cons:

  • It might be generated unexpected result like buttons overlap with each other

Summary

If you want to have UI elements adjust their size based on device screen resolution, especially you want player on iphone could have relatively bigger UI elements, you could use this method. But you should be careful when elements are very close to each other. Because they might overlap with each other if their size keep increasing but they stay in the same position.

Two patterns

For two patterns, it is easy to understand. The game would use two different pattern of UI based on screen resolution. Here is the example from Hearthstone:

16:9 (iphone5)

Screen Shot 2015-08-17 at 11.28.57 AM.png

4 : 3 (ipad)

Screen Shot 2015-08-17 at 11.27.57 AM.png

Pros:

  • Provide the best interaction experience by choosing the appropriate pattern according to screen resolution

Cons:

  • You need to do everything twice

Summary

If you have enough resources and time, just do it!

Error Handler

Usually in game development, we want each error to be detected immediately and processed gracefully. Recently I am working on error handler to handle all different error states in our game. Here I would like to talk about it in detailed.

To begin with, it should be clear that error detection and error process should be separated. In language like C++ and Java, they use try-catch to handle the error. But using try-catch might not be always a good idea. Joel Spolsky pointed out two drawbacks of using try-catch.

  1. They are invisible in the source code. Looking at a block of code, including functions which may or may not throw exceptions, there is no way to see which exceptions might be thrown and from where. This means that even careful code inspection doesn’t reveal potential bugs.
  2. They create too many possible exit points for a function. To write correct code, you really have to think about every possible code path through your function. Every time you call a function that can raise an exception and don’t catch it on the spot, you create opportunities for surprise bugs caused by functions that terminated abruptly, leaving data in an inconsistent state, or other code paths that you didn’t think about.

Moreover, for me, I would like to have my own error message data structure. Therefore, rather than use try-catch directly, I create my own error handler class. The general idea of this class is still similar to try-catch, it has two main functionality, which are error detection and error handle.

Here is the workflow when an error occurs:

  1. One error occurs
  2. All listeners handle this error event. If the error is solved, clear the error.
  3. Check whether error still exist
  4. If yes, bing player back to the first scene
  5. Show error message to player (UI pop up)
  6. Player could check whether error is solved by pressing error message UI button.

Back to my ErrorHandler class, it has following things:

Function

  • void ErrorStateReport();                                             //Call back function when an error is detected
  • bool CheckErrorState( ErrorType i_error);            //Check one certain type of error whether exists.
  • bool CheckError();                                                       //Check whether there is still an error
  • void ClearError();                                                        //Clear error, set errorType to None.
  • void SetError( ErrorType i_error);                         // Set errorType to a certain type of error.

Data Member

  • readonly Dictionary<ErrorType, string> ErrorMessageDictionary;                // Store display information for each error type
  • ErrorType ErrorType;                                                                                                 //Error type.
  • delegate void OnErrorState( string errorMessage, ErrorType errorType);     //Error delegate
  • event OnErrorStateEvent OnErrorState;                                                                //Error event

Here is the workflow by using ErrorHandler class:

Scene where error occurs

  • //Error occurs
  • ErrorHandler.ErrorStateReport( Error );                                                       //All listener would handle the error
  • if(ErrorClientHandler.CheckError()) ErrorClientHandler.ErrorType = // one error type
  • //Back to first scene

First scene

  • if(ErrorClientHandler.CheckError())  ShowErrorUI( ErrorClientHandler.ErrorMessageDictionary[ ErrorClientHandler.ErrorType], ErrorClientHandler.ErrorType);
  • //….we are in error state now
  • //If player press error message UI button
  • if( !ErrorClientHandler.CheckErrorState(ErrorClientHandler.ErrorType) ) //error solved
  • //Keep retrying, error still exists.

This is ErrorHandler class I create. And it could match the requirement of our game’s error handle perfectly.

 

New Update

I have to say game reset is also a very dangerous activity. It is ironic that when I tried to use error handler to reset the game, it would bring new errors and crash the whole game.

This is what resetGame function did before:

  • Uninit the game;
  • Game back to main menu scene;

But when game uninited itself, some monobehaviour scripts were still running, and they might want to have the access to game data. Unfortunately, game data just destroyed itself. Then new error (null exception) came out, and at last game crashed.

The solution to this problem is before uniniting the game data, make game first enter an empty game scene to ensure no object still has reference to game data. Then we uninit game data, and last bring player back to main menu scene. So the new workflow turns out to be:

  • Go to empty scene;
  • Uninit the game;
  • Game back to main menu scene;

Exceptions

Exceptions DoSomething()

Exceptions vs. status returns

About Unity asset memory leak issue

Loading and unloading asset could cause memory problem if we are not able to handle it carefully. Here you could find the problem we might meet, the solution and the reason why such a problem happens.

Example Problem

Texture assets remain in memory after scene changes

Description

When player is in scene A, those character textures that he has checked would be loaded, which is normal. But when he moves to scene B, those textures still remain in memory. To see memory profiler, go to Window -> Profiler, click memory panel, then chose detailed option. Run the game, click “Take Sampler:Editor”, then we could see all asset loaded in memory.

Solution:

The thing we need to do is set texture to null when dynamic card is destroyed (onDestroy()). Then call Resources.UnloadUnusedAssets() when the scene ends or changes as we use Resources.Load() to get the 2D texture.

Detailed Explanation

Usually we want to use Resources.UnloadUnusedAssets() to unload asset loaded from resource.And we have to make sure there is no reference to the asset itself before calling Resources.UnloadUnusedAssets(). In the example, the game object should be destroyed when scene changes.

However, Destroy() function does not work in a proper way. In detailed,  a lot objects in Unity, to be more precisely, all that are inherited from UnityEngine.Object, consists of two parts: a managed object and a native code c++ class in the engine itself. In the managed environment we only “see” the managed object that is “linked” with the native part. This is where two totally different worlds collide. When you call destroy on the managed object, Unity will destroy the native code part of the object. But the managed part of the class is still there! Unity will flag the class internally as “destroyed”.

In short, Unity uses a “trick” to make the managed part “pretend” to be null when the native part is destroyed. In the example, the character texture in dynamic card (2D texture derived from object) has not been destroyed correctly. There is still a reference to the texture asset loaded from Resources and Resources.UnloadUnusedAssets() could not unload it.

Therefore, in order to destroy the object properly, we have to set the asset to null, like it mentioned above. In this way, there is no reference to the texture asset loaded from Resources. And Resources.UnloadUnusedAssets() could work correctly at last.

 

Atlas Memory Problem

We probably meet the problem that atlas still remains in memory after scene changes. This is not what we want to see because usually the size of atlas is big. In order to solve that,we might need to do following things:

  • Call UIDrawCall.ReleaseInactive() or ReleaseAll()
  • Then call UnloadUnusedAssets().

Also, if the scene we work on has the static object, make sure object set to null before it is destroyed (including the texture, material and etc). Usually if we see ManagedStaticReference in “Reference by”, then we have to be careful with that.

Moreover, think twice before working on NGUI object, as NGUI object would have the reference to atlas. If the object is not destroyed correctly when scene changes, it could cause atlas remain in memory. 

Currently even Unity community could not provide available solution to atlas unloaded problem. And it is possible that NGUI or Unity itself still has some bugs. We have to rely on ourselves now.

 

Summary

To sum up, we need to check following issues if we meet similar memory problem:

1.Keep track of the asset loaded from resource, make sure it have been set to null when it is destroyed.

2.Be careful about the assignment operator. For example:

     Object obj = Resources.Load(“MyPrefab”);
GameObject instance = Instantiate(obj) as GameObject;
………
Destroy(instance);

      We also have to destroy obj if we want to call Resources.UnloadUnusedAssets() to unload the “MyPrefab” asset. This is why in 1 says “keep track of the loaded asset”. 

3.Make sure Resources.UnloadUnusedAssets() is called. Even though this function should be automatically called when scene changes ( I have tested it and it is true). It is better call Resources.UnloadUnusedAssets() explicitly just in case.

4.Make sure static object has been cleared up.

5. If the asset still could not be unloaded, try calling System.GC.Collect(). Although it might solve the problem, it is not recommended.

 

Reference

Does Destroy remove all instances?

Resources.UnloadUnusedAssets

Similar Texture Memory Problem

Unload NGUI atlas

Thesis Game – Week_7_Work: New creepy design brainstorm again

First I will still talk about the programming work I have done last week:

1. The landing script seemed not to work because the terrain is over one more at the same time and the script could work only there is one active terrain in the scene. So it should disable
other terrains that the player is not on. Also the automatically generated landingspot does not work at all. Because the landingSpots have been wrongly added to the wrong LandingController object.

2. Fix the bug of the backward flying. What I did wrong is that I changed the flock position rather than the waypoint position. So the crows tend to fly back to the previous position and then fly to the new position, leading to the backward flying behavior.

3. The audio script still could not be played because I just simply enable and disable the script. I should play the audio
function when I enable the script.

Here is the new video of our game:

Here is the brainstorm of the current creepy resources we have in our game:

Horror1

I think what we need to do are the following two things:

1. Create the concentration moment in our game to implement the bear horror, we should avoid the game play experience to turn out to be aimless walking. We need the player to focus and focus more! Just like the game scene in Silent Hill P.T. Only you focus on the door can you get be scared by the upcoming woman.

bear_horror

2. Make the sense to the horror element. We know there are so many ways to scare people, but those efficient ways must have a meaning behind. This is why I do some research about the horror culture to add more meanings in our game.

Thesis Game – Week_6_Work: Optimize the crows and fix the bug

This week I have done following things:

  1. Optimized the crows by disabling the audio script when the crow is generated. Only under the condition that the crow is close enough to the player and then the audio script will be enabled.
  2. Added the attack interval time to the crows’ attacking behavior. In the previous version, the crows would continuously attack the player. And now I added the interval  time to the attack function. Now the crows act like more real.
  3. Attached the send damage function to the crow manager script and now when crows attack the player, the player will lose the health point.
  4. Optimized the crows by changing the landAll() function. In previous project, the program would call landAll() function every frame once the player is out of the attacking range of the crows. Now I make it only call that function once.
  5. Fix the bug of the LandingManager Script. I made the default value of the crows flying variable as true.

Here is the short video of this week’s work:

Thesis Game – Week_5_Work: Landing Crows and Test Part2 Level

This week I created a new behavior of the crows, which is the landing crow. This behavior is handled by the LandingController Script. This script will grab a random crow and then make it land on the spot if the landing behavior is triggered. What I did to improve this script is adding a “LandingManager” script, which would automatically generate several landing spots at a certain amount. What we should pay attention to is now the game level is a terrain rather than a flat plane. So I need to make use of terrain

beginPosition.y = Terrain.activeTerrain.SampleHeight( beginPosition );

to get a random spot that is actually on the spot, not above nor under. This is what it looks like when you run it:

5.1

And now the crows will randomly land on these spot. When the player is closer enough to the crows, they will fly away. But if the player goes away, they will come back and land again.

Also I created the part 2 of our test level of the game, here is the design of that level:

5.2

And here is a brief video of this game level:

Thesis Game – Week_4_Work: Crows VS Player

This week I added the collision between the crows and player. Therefore, one more status of crow is also added to the crow controller, which is the dead status. After the crow is hit by the player, its status of death will be turn into true, then the crow’s movement behavior is disabled.

  1. function Dead(){
  2.             _deadTimer += Time.deltaTime;
  3.             if( gameObject.GetComponent(Rigidbody) == null ){
  4.                          var crowRigidBody:Rigidbody;
  5.                          crowRigidBody = gameObject.AddComponent (Rigidbody);
  6.                          crowRigidBody.mass = 5;
  7.              }
  8.             if( _deadTimer > 3.0f ){
  9.                          var animator:Animation;
  10.                         animator = gameObject.GetComponentInChildren(Animation);
  11.                         animator.enabled = false;
  12.             }
  13. }

From this video, you can the crows are attacking the player and the player is fighting back by using a mace ( just for a test, I know a mace for our game is weird ). If the crow gets hit, it will fall down on the ground but still flapping it wings. Then after a while, it totally dies and stops any movement. At last, the program will destroy this crow’s gameobject.

Thesis Game – Week_3_Work: Lonely Crows and Crows Attack bug fixed

1.Added the lone crow to the game

This week I am required to create new type of crow, which is called “Lonely Crow.” This certain type of crow would not surround the scarecrow and just flying around in the sky. If the player is close enough to the lonely crow, it will attack you. So I created a new variable which called “_crowLonely”  to determine the type of the crow flock. Once the crow’s type is “Lonely Crow”, first it will not surround with the scarecrow, and second it will just fly in the air ( which Flap() function will handle this ). Once the player is in the range of attacking area, the lonely crow will still attack him.

 

2.Changed the editor of the FlockController by adding the “Lone Crow” item to inspector

All the input variable we want to use for crow flock will be showed in CrowFlock inspector editor. Now I want to add one more item to that, so I need to change the code of  FlockControllerEditor. In this script, I add following code:

EditorGUILayout.Space();
EditorGUILayout.LabelField(“Bird Flock Type”, EditorStyles.boldLabel);
EditorGUILayout.LabelField(“Lone bird will just fly around an area of the map, others will fly in a circle of the scarecrow”, EditorStyles.miniLabel);
target._crowLonely = EditorGUILayout.Toggle(“Bird Lonely” , target._crowLonely);
EditorGUILayout.Space();

Here is what the new inspector editor looks like:

Capture

You can see now we have one more item here, the Bird Flock Type.

3.Fix the bug of attacking position’ height

In the previous version, the crow always attacked the feet of the player ,which was very weird. This week I fixed the problem by changing the rotation of the crow. The problem is I make the crow rotate to the player’s position, but that position is not the position of the player’s head, instead, it’s the position of the player’s body. So I added a height to the player’s position, which would be the attacking position of the crow. Then this bug is fixed.

var _rotateVector:Vector3;
_rotateVector = new Vector3( 0.0f, _spawner._attackCrowHeight, 0.0f ) + player.transform.position – transform.position;
_rotateVector.Normalize();
transform.rotation = Quaternion.Slerp( transform.rotation, Quaternion.LookRotation( _rotateVector ), 20.0f * Time.deltaTime );

 

4.Make the crows object added to the Crows object

Because the crows in the game are dynamically generated, so how to make them convenient to use would be kind of important for me. For this issue, I use an emptyObject in the scene as the parent gameObject of all the crows. So when the game begins, all the crows will be under this object, making the editor more easily to see. Just like this:

Capture

5.Add the collision between the crow and the mace

I also added the collision between the crow and mace to make the interaction of player and crow. For this issue, I will talk it in detailed in next week.