Unity supports multiple plug-in types for Android applications. This post relates to Java (non-native) plugins: JAR, AAR, source code Java. We show how to access surface view instance created by Unity player instance from inside android Java plugin. Unity version 2021.3.16f1, Android minimal API level 26 (Android 8.0 Oreo), Unity scripting backend Mono .NET Standard 2.1, Java v8.0.

Intro

Java plugin code can be called from C# scripts. The code can access Android system as a conventional android application.

Access to Java classes of the plugins is mediated by high level API available to C# scripts.

Plugins allow to extend the default Unity activity.

There is no official documentation for UnityPlayer Java API. One way to explore the unity code is to look at the decompiled code. The problem is that it’s not the latest version. Another way is to decompile dex file which comes with Unity. Just drop


~/Unity/Hub/Editor/2020.1.17f1/Editor/Data/PlaybackEngines/AndroidPlayer/Variations/mono/Release/Classes/classes.dex

onto Android Studio editor window. Here is an example:

decompiled

On the other hand, there is a source for UnityPlayerActivity class located at


~/Unity/Hub/Editor/2021.3.16f1/Editor/Data/PlaybackEngines/AndroidPlayer/Source/com/unity3d/player/UnityPlayerActivity.java

UnityPlayer surface view

We may want to access UnityPlayer surface view to grab a bitmap of the surface.

Decompiling UnityPlayer class we see that UnityPlayer

inhertits from FrameLayout
saves the created surface view in its private non-static variable mGlView

public class UnityPlayer extends FrameLayout implements IUnityPlayerLifecycleEvents {
    // ...
    private SurfaceView mGlView;
    // ...

    public UnityPlayer(Context var1, IUnityPlayerLifecycleEvents var2) {
        // ..
        if (!m.c()) {
            // ...
        } else {
            // ...
            this.mGlView = this.CreateGlView();
            this.mGlView.setContentDescription(this.GetGlViewContentDescription(var1));
            this.addView(this.mGlView);
            // ...
        }
    }

    private SurfaceView CreateGlView() {
        SurfaceView var1;
        (var1 = new SurfaceView(this.mContext)).setId(this.mContext.getResources().getIdentifier("unitySurfaceView", "id", this.mContext.getPackageName()));
        // ..
        var1.setFocusable(true);
        var1.setFocusableInTouchMode(true);
        return var1;
    }

Looking at UnityPlayerActivity, we see UnityPlayer instance is set to content view of the activity:

public class UnityPlayerActivity extends Activity implements IUnityPlayerLifecycleEvents
{
    protected UnityPlayer mUnityPlayer; // don't change the name of this variable; referenced from native code

    // ...
    // Setup activity layout
    @Override protected void onCreate(Bundle savedInstanceState)
    {
        // ...
        mUnityPlayer = new UnityPlayer(this, this);
        setContentView(mUnityPlayer);
        mUnityPlayer.requestFocus();
    }
    // ...
}

Reading mGlView of UnityPlayer

Unlike currentActivity, which is a public static field, mGlView is a private non-static field. The method proposed to read currentActivity:

AndroidJavaClass unityPlayer = new AndroidJavaClass("com.unity3d.player.UnityPlayer");
AndroidJavaObject currentActivity = unityPlayer.GetStatic<AndroidJavaObject>("currentActivity");

cannot be used to read mGlView.

We use java reflection to read mGlView. But before that, we need to have an instance of UnityPlayer. To get the instance of UnityPlayer we should read protected mUnityPlayer field of UnityPlayerActivity. Thus, we have the following algorithm:

// C# code called from Unity scripts
// First we get an instance of UnityPlayer class 
AndroidJavaClass unityPlayer = new AndroidJavaClass("com.unity3d.player.UnityPlayer");
// Then we read static field with current activity
AndroidJavaObject currentActivity = unityPlayer.GetStatic<AndroidJavaObject>("currentActivity");

// Plugin's java code
import com.unity3d.player.UnityPlayer;
import com.unity3d.player.UnityPlayerActivity;

class SurfaceViewByReflection {
    private static final String TAG = "SurfaceViewByReflection";

    private static UnityPlayer getUnityPlayer(Activity activity) {
        UnityPlayer result;
        try {
            Field field = UnityPlayerActivity.class.getDeclaredField("mUnityPlayer"); // pretending activity is instance of SubActivity
            field.setAccessible(true);
            Log.d(TAG, "getUnityPlayer: made field accessible");
            result = (UnityPlayer) field.get(activity);
            Log.d(TAG, "getUnityPlayer: result:" + result.toString());

        } catch(Exception e) {
            result = null;
            Log.d(TAG, "getUnityPlayer:" + e);
        }
        return result;
    }

    private static SurfaceView getSurfaceView(UnityPlayer unityPlayer) {
        SurfaceView result;
        if (unityPlayer == null) {
            result = null;
        }
        else {
            try {
                Field field = UnityPlayer.class.getDeclaredField("mGlView");
                field.setAccessible(true);
                result = (SurfaceView) field.get(unityPlayer);

            } catch (Exception e) {
                result = null;
                Log.d(TAG, "getSurfaceView:" + e);
            }
        }
        return result;
    }

    public static SurfaceView getUnityPlayerSurfaceView(Activity activity) {
        return getSurfaceView(getUnityPlayer(activity));
    }
}

How to import UnityPlayerActivity

UnityPlayerActivity is not a part of classes.jar of Unity framework. To import UnityPlayerActivity in your Java plugin, follow the following steps.

Add a Android library module to your application. Name it, for example, xlibrary and set package to com.unity3d.player.
Take UnityPlayerActivity.java and drop to xlibrary source folder.
Take classes.jar and drop to xlibrary libs folder.

This is how xlibrary structure looks:

xlibstructure

In your plugin build.gradle file, add dependency:
```
 compileOnly project(path: ':app:xlibrary')
```
Here :app is root project name under which xlibrary was added.
Now, you can import com.unity3d.player.UnityPlayerActivity without errors.

Note, that both UnityPlayer and UnityPlayerActivity should be instances of com.unity3d.player and com.unity3d.player.UnityPlayerActivity classes respectively. Otherwise, reflect API will throw with invalid argument type.

Following activity window view hierarchy to get to UnityPlayer surface view

We can get to UnityPlayer surface view following the android view hierarchy. Having Unity activity, we can get it’s window and root view. Then we can traverse view tree to locate UnityPlayer and its surface view.

Inside our plugin, we run the following function:

import android.app.Activity;
import android.view.SurfaceView;
import android.view.View;
import android.view.ViewGroup;
import android.view.Window;

private void show_children(View v, int level) {
    ViewGroup viewgroup=(ViewGroup)v;
    int childCount = viewgroup.getChildCount();
    String margin = "";
    for (int i=0; i<level; i++) {
        margin += " ";
    }
    Log.d(TAG, String.format("%sGroup: %s, child cnt: %d", margin, viewgroup.toString(), childCount));
    for (int i=0;i<viewgroup.getChildCount();i++) {
        View v1=viewgroup.getChildAt(i);
        if (v1 instanceof ViewGroup) {
            // Log.d(TAG, String.format("child %d is a group", i));
            show_children(v1, level + 1);
        } else {
            Log.d(TAG, String.format("%s child %d: %s", margin, i, v1.toString()));
        }
    }
}

// activity of unityPlayer sent by C# code
public entryToPlugin(Activity activity) {
    Window window = activity.getWindow();
    show_children(window.getDecorView(), 0);
}

The function show_children() prints the following results:

01-10 02:59:19.944 10173 10225 D pz    : Group: DecorView@9bdcd0b[UnityPlayerActivity], child cnt: 1
01-10 02:59:19.944 10173 10225 D pz    :  Group: android.widget.LinearLayout{9b06da2 V.E...... ......I. 0,0-810,1800}, child cnt: 2
01-10 02:59:19.946 10173 10225 D pz    :   child 0: android.view.ViewStub{ed23a33 G.E...... ......I. 0,0-0,0 #10201c7 android:id/action_mode_bar_stub}
01-10 02:59:19.946 10173 10225 D pz    :   Group: android.widget.FrameLayout{3b081f0 V.E...... ......I. 0,0-810,1800 #1020002 android:id/content}, child cnt: 1
01-10 02:59:19.946 10173 10225 D pz    :    Group: com.unity3d.player.UnityPlayer{95ea369 V.E...... ......I. 0,0-810,1800}, child cnt: 1
01-10 02:59:19.946 10173 10225 D pz    :     child 0: android.view.SurfaceView{2de493d VFE...... .F....I. 0,0-810,1800 #7f010000 app:id/unitySurfaceView aid=1073741824}

The visualized results are

hierarchy

Knowing location of the surface view we can access it using the following function:

private SurfaceView getUnityPlayerSurfaceView(Activity activity) {
    ViewGroup decorView = (ViewGroup) activity.getWindow().getDecorView();
    ViewGroup linearLayout = (ViewGroup) decorView.getChildAt(0);
    ViewGroup frameLayout = (ViewGroup) linearLayout.getChildAt(1);
    ViewGroup unityPlayer = (ViewGroup) frameLayout.getChildAt(0);
    SurfaceView surfaceView = (SurfaceView) unityPlayer.getChildAt(0);
    return surfaceView;
}

Refs

Some of the articles refrenced by the following links are outdated but stiil may be a valued source of information.

unity3d android

Surface view of Unity3D player

Table Of Contents