Problem with external file location on Android

[Ats]

It's one problem after another

So, in order to confirm that my code change regarding ReadAssetFile for Android 64 is working, I'm trying to run the FileDemo on Android. And surprise, it's looking for TestFile.txt in a folder that requires root access:
/data/user/0/com.mydomain.filedemo/files/TestFile.txt

The only way I managed to browse it and upload the TestFile.txt in it was to use:
Android Studio → View → Tool Windows → Device Explorer

This folder is for private app files (databases, preferences, cache). My savefile for Omeganaut is there and it's fine.

But for editable files, it should normally be here:
/storage/emulated/0/Android/data/com.mydomain.filedemo/files/TestFile.txt

Where the user has (more or less) permission to manage their files, depending on the Android version...
Do you happen to remember where this path is set in ZGE's code?


On the bright side, once the TestFile.txt is found, the app is working flawlessly with my modification

[Ats]

So I think it is set in zgameeditor-master\Build\android\ZgeAndroid.pas:

Code: Select all

procedure Java_org_zgameeditor_Zge_NativeInit( env : PJNIEnv; thiz : jobject; ExtPath : jstring; DataPath : jstring; LibraryPath : jstring);cdecl;
var
  P : PAnsiChar;
begin
  P := env^.GetStringUTFChars(Env,ExtPath,nil);
    GetMem(AndroidPath,ZStrLength(P)+1);
    ZStrCopy(AndroidPath,P);
    AndroidLog(AndroidPath);
  env^.ReleaseStringUTFChars(Env,ExtPath,P);

  P := env^.GetStringUTFChars(Env,DataPath,nil);
    GetMem(AndroidDataPath,ZStrLength(P)+1);
    ZStrCopy(AndroidDataPath,P);
    AndroidLog(AndroidDataPath);
  env^.ReleaseStringUTFChars(Env,DataPath,P);

  P := env^.GetStringUTFChars(Env,LibraryPath,nil);
    GetMem(AndroidLibraryPath,ZStrLength(P)+1);
    ZStrCopy(AndroidLibraryPath,P);
    AndroidLog(AndroidLibraryPath);
  env^.ReleaseStringUTFChars(Env,LibraryPath,P);
  
  AndroidThis := Env^.NewGlobalRef(Env,thiz);
end;

Coming from zgameeditor-master\Build\android\java\src\org`zgameeditor\Zge.java:

Code: Select all

        String dataPath = context.getFilesDir().getAbsolutePath() + "/";
        String libraryPath = getContext().getApplicationInfo().dataDir + "/lib/";
        String extPath = Environment.getExternalStorageDirectory().getAbsolutePath();
        NativeInit(extPath, dataPath, libraryPath);

But I fear it will break a few things if I modify all that. I don't know... We already set the library path last autumn:
viewtopic.php?p=10729#p10729

I'll check that after lunch.

[VilleK]

So I think it is set in zgameeditor-master\Build\android\ZgeAndroid.pas:

I just found out the same.

Code: Select all

String dataPath = context.getFilesDir().getAbsolutePath() + "/";

I think it is safe to change this to something else. Question is what path is recommended to use for modern Android apps?

[Ats]

Here's the usage for internal path (config file...):
https://developer.android.com/training/ ... c#internal

Code: Select all

File file = new File(context.getFilesDir(), filename);

And the same for external path (editable file...):
https://developer.android.com/training/ ... c#external

Code: Select all

File appSpecificExternalDir = new File(context.getExternalFilesDir(null), filename);

This line in Zge.java is deprecated:

Code: Select all

String extPath = Environment.getExternalStorageDirectory().getAbsolutePath();

We can still support old Android with:

Code: Select all

String extPath;
if (Build.VERSION.SDK_INT >= Build.VERSION_CODES.Q) {
    extPath = context.getExternalFilesDir(null).getAbsolutePath();
} else {
    extPath = Environment.getExternalStorageDirectory().getAbsolutePath();
}

Or completely replace it:

Code: Select all

String extPath = context.getExternalFilesDir(null).getAbsolutePath();

[VilleK]

You can make changes in the Java files without me having to build anything, right? Please try what works best. I trust your judgement

[Ats]

Hahaha, it's not that simple

After verification, this works nicely to obtain /data/user/0/com.mydomain.filedemo/files/TestFile.txt:

Code: Select all

String dataPath = context.getFilesDir().getAbsolutePath() + "/";

This is for /storage/emulated/0/Android/data/com.mydomain.filedemo/files/TestFile.txt:

Code: Select all

String extPath = context.getExternalFilesDir(null).getAbsolutePath() + "/";

And the fixed libraryPath:

Code: Select all

String libraryPath = getContext().getApplicationInfo().nativeLibraryDir + "/";

The problem is that the ZGE File component looks for files inside dataPath, not extPath. So it’s more of a design decision on ZGE’s side:

Should we look for external files in the hidden "data" folder?
The thing is that we can't easily put a file in there unless we extract the file from the apk at runtime.
If so, why not embed them instead? I don't know if there's a benefit in storing files in the data folder.
On the bright side, using the zge File to save a file, like a configuration, is nicely stored there.

Or should we store these files in the visible "ext" folder?
Same, they would need to be extracted there at some point. But the user has access to that folder.
But I'm not even sure extPath is actually used after being set. Or am I missing something?


...
After looking up again how files works in ZPlatform_Android.pas, turns out there's a very specific case in order to find files in the APK's assets folder:

Code: Select all

  if Copy(FileName,1,8)='/assets/' then
  begin
    ReadAssetFile(FileName, Memory, Size);

I only have to add /assets/ in the File component path to find the files stored in the /assets/ folder... But the exe will also need that file in an "assets" folder to be compatible.

Let's list that again:

• /assets/ folder for reading the game files (ex: our FileDemo's TestFile.txt Space Invader)
• dataPath path for reading/writing hidden files (ex: configuration, save...)
• extPath path for the user files (ex: the user modified TestFile.txt to display something else)

Maybe, if the /assets/ is not manually set in the File component path, we could search for the file in the extPath for the user files, and if it's not there, fallback to the dataPath? Would that work for reading AND saving?

[Ats]

* Little sidequest *

In order to test the read/save file behavior on Android, I started making a modification of the FileDemo to turn it into some paint program. It reads the TestFile.txt at loading, but I don't know how to save the drawing to that file once it has been modified. Can't we save plain text?

Code: Select all

<?xml version="1.0" encoding="iso-8859-1" ?>
<ZApplication Name="App" Caption="ZGameEditor: FileDemo" Camera="CameraOrtho" MouseVisible="255" FileVersion="2" AndroidPackageName="com.mydomain.filedemo">
  <OnLoaded>
    <Variable Name="Temp1"/>
    <ZExpression>
      <Expression>
<![CDATA[MousePos = -1;
MouseClick = -1;

TileCountX = round(App.ViewportWidth / 20);
TileCountY = round(App.ViewportHeight / 20);

RepeatColumn.Count = TileCountX;
RepeatLine.Count = TileCountY;

ArrayTiles.SizeDim1 = TileCountX * TileCountY;]]>
      </Expression>
    </ZExpression>
    <Repeat Name="RepeatColumn" Comment="Create all tiles" Count="40">
      <OnIteration>
        <Repeat Name="RepeatLine" Count="30">
          <OnIteration>
            <ZExpression>
              <Expression>
<![CDATA[int x = RepeatColumn.Iteration;
int y = RepeatLine.Iteration;

// Convert 2D position touchGetCount 1D index
int pos = y * TileCountX + x;

TileModel.Position.X = x - TileCountX * 0.5 + 0.5;
TileModel.Position.Y = y - TileCountY * 0.5 + 0.5;
TileModel.visible = 0;
TileModel.number = pos;
ArrayTiles[pos] = createModel(TileModel);]]>
              </Expression>
            </ZExpression>
          </OnIteration>
        </Repeat>
      </OnIteration>
    </Repeat>
    <FileAction File="LevelFile"/>
  </OnLoaded>
  <OnUpdate>
    <ZExpression Expression="MousePos = -1;"/>
    <KeyPress Comment="Mouse left click" Keys="{">
      <OnPressed>
        <ZExpression>
          <Expression>
<![CDATA[// Convert from [-1,1] range to [0,40] and [0,30] ranges
float normalizedX = (App.MousePosition.X + 1.0) * 0.5;
float normalizedY = (App.MousePosition.Y + 1.0) * 0.5;

// Then scale to array dimensions (flipping Y axis if needed)
int x = round(normalizedX * (TileCountX - 1) );
int y = round(normalizedY * (TileCountY - 1) );

MousePos = y * TileCountX + x;]]>
          </Expression>
        </ZExpression>
        <FileAction File="LevelFile" Action="1"/>
      </OnPressed>
    </KeyPress>
    <ZExpression Expression="if (MousePos == -1) MouseClick = -1;"/>
  </OnUpdate>
  <Content>
    <Model Name="TileModel" Position="19.5 14.5 0">
      <Definitions>
        <Variable Name="visible" Type="1"/>
        <Variable Name="number" Type="1"/>
      </Definitions>
      <OnUpdate>
        <ZExpression>
          <Expression>
<![CDATA[if (MousePos == CurrentModel.number) {
  if (MouseClick == -1) MouseClick = 1 - CurrentModel.visible;
  CurrentModel.visible = MouseClick;
}

if (CurrentModel.visible == 1) {
  Temp1 = noise3(abs(CurrentModel.Position.X), abs(CurrentModel.Position.Y), App.Time*0.25);
  CurrentModel.Scale.X = 2 * (1 + Temp1);
  CurrentModel.Scale.Y = 2 * (1 + Temp1);
}]]>
          </Expression>
        </ZExpression>
      </OnUpdate>
      <OnRender>
        <Condition Comment="Visible?" Expression="return CurrentModel.visible == 1;">
          <OnTrue>
            <RenderSetColor Color="0.77 0.07 0.77 1"/>
            <RenderMesh Mesh="TileMesh"/>
          </OnTrue>
        </Condition>
      </OnRender>
    </Model>
    <Mesh Name="TileMesh">
      <Producers>
        <MeshSphere Scale="0.4 0.4 0.4"/>
      </Producers>
    </Mesh>
    <Group Name="FileGroup">
      <Children>
        <File Name="LevelFile" FileName="/assets/TestFile.txt">
          <OnRead>
            <ZExpression Comment="Init CurY" Expression="TileCurY = (TileCountY/2);"/>
            <Repeat Comment="Y loop">
              <OnIteration>
                <ZExpression Comment="Init CurX" Expression="TileCurX=-(TileCountX/2);"/>
                <Repeat Comment="X loop" WhileExp="return this.Iteration&lt;TileCountX+2;">
                  <OnIteration>
                    <FileMoveData Property="TileID"/>
                    <Condition Comment="Spawn tile on &apos;A&apos; (ascii 65)" Expression="return TileID==65;">
                      <OnTrue>
                        <ZExpression>
                          <Expression>
<![CDATA[int pos = TileCurY * TileCountX + TileCurX;
ArrayTiles[pos].visible = 1;]]>
                          </Expression>
                        </ZExpression>
                      </OnTrue>
                    </Condition>
                    <ZExpression Comment="Update CurX" Expression="TileCurX+=1;"/>
                  </OnIteration>
                </Repeat>
                <ZExpression Comment="Update CurY" Expression="TileCurY-=1;"/>
              </OnIteration>
              <WhileExp>
<![CDATA[//this.Iteration=current iteration nr. Return false to end loop.
return this.Iteration<TileCountY;]]>
              </WhileExp>
            </Repeat>
          </OnRead>
          <OnWrite>
            <ZExpression Comment="Init CurY" Expression="TileCurY = (TileCountY/2);"/>
            <Repeat Comment="Y loop">
              <OnIteration>
                <ZExpression Comment="Init CurX" Expression="TileCurX=-(TileCountX/2);"/>
                <Repeat Comment="X loop" WhileExp="return this.Iteration&lt;TileCountX+2;">
                  <OnIteration>
                    <FileMoveData Property="TileID"/>
                    <ZExpression Comment="Update CurX" Expression="TileCurX+=1;"/>
                  </OnIteration>
                </Repeat>
                <ZExpression Comment="Update CurY" Expression="TileCurY-=1;"/>
              </OnIteration>
              <WhileExp>
<![CDATA[//this.Iteration=current iteration nr. Return false to end loop.
return this.Iteration<TileCountY;]]>
              </WhileExp>
            </Repeat>
          </OnWrite>
        </File>
        <Variable Name="TileID"/>
        <Variable Name="TileCurX"/>
        <Variable Name="TileCurY"/>
      </Children>
    </Group> <!-- FileGroup -->

    <Camera Name="CameraOrtho" Kind="1" Position="0 0 1" OrthoZoom="15"/>
    <Variable Name="MousePos" Type="1"/>
    <Variable Name="MouseClick" Type="1"/>
    <Array Name="ArrayTiles" Type="3"/>
    <Variable Name="TileCountX" Type="1"/>
    <Variable Name="TileCountY" Type="1"/>
  </Content>
</ZApplication>

[VilleK]

* Little sidequest *

It turns out that string array writing was never implemented. I added that now, please try it here: http://www.zgameeditor.org/files/ZGameEditor_beta.zip

Here is an example that writes an array to a file:

Code: Select all

string[] a = {"line 1", "line 2", "line 3", "line 4"};
@FileAction( File : @File(FileName: "t:\\test.txt", TargetArray : a), Action : 1 );

[Kjell]

Hi Ats,

Not sure why you'd want to use "char" encoding for a paint program ... why not just do the following?

Code: Select all

<?xml version="1.0" encoding="iso-8859-1" ?>
<ZApplication Name="App" Caption="Paint" CameraPosition="8 8 16" FOV="90" MouseVisible="255" FileVersion="2">
  <OnLoaded>
    <FileAction File="PixelsFile"/>
    <ZExpression>
      <Expression>
<![CDATA[//

Pixels.SizeDim1 = 16;
Pixels.SizeDim2 = 16;

//

for(int x=0; x<16; x++)
{
  for(int y=0; y<16; y++)
  {
    Pixel.Position.X = x+0.5;
    Pixel.Position.Y = y+0.5;

    createModel(Pixel);
  }
}]]>
      </Expression>
    </ZExpression>
  </OnLoaded>
  <OnUpdate>
    <ZExpression>
      <Expression>
<![CDATA[//

Click[1] = Click[0];
Click[0] = 0;]]>
      </Expression>
    </ZExpression>
    <KeyPress Keys="{">
      <OnPressed>
        <ZExpression>
          <Expression>
<![CDATA[//

Click[0] = 1;]]>
          </Expression>
        </ZExpression>
      </OnPressed>
    </KeyPress>
    <ZExpression>
      <Expression>
<![CDATA[//

int x = 8+App.MousePosition.X*16*App.ScreenWidth/App.ScreenHeight;
int y = 8+App.MousePosition.Y*16;

//

if(x >= 0 && x < 16 && y >= 0 && y < 16)
{
  if(Click[0] && !Click[1])
  {
    Action = !Pixels[x,y];
  }

  if(Click[0])
  {
    Pixels[x,y] = Action;
  }
}]]>
      </Expression>
    </ZExpression>
  </OnUpdate>
  <OnClose>
    <FileAction File="PixelsFile" Action="1"/>
  </OnClose>
  <Content>
    <Variable Name="Action" Type="4"/>
    <Array Name="Click" Type="4" SizeDim1="2"/>
    <Model Name="Pixel">
      <OnRender>
        <UseMaterial Material="PixelMaterial"/>
        <Condition>
          <Expression>
<![CDATA[//

int x = Pixel.Position.X;
int y = Pixel.Position.Y;

return Pixels[x,y];]]>
          </Expression>
          <OnTrue>
            <RenderSetColor Color="1 1 1 1"/>
          </OnTrue>
          <OnFalse>
            <RenderSetColor Color="1 0 0 1"/>
          </OnFalse>
        </Condition>
        <RenderMesh Mesh="PixelMesh"/>
      </OnRender>
    </Model>
    <Mesh Name="PixelMesh">
      <Producers>
        <MeshBox Scale="0.5 0.5 1" Grid2DOnly="255"/>
      </Producers>
    </Mesh>
    <Bitmap Name="PixelBitmap" Filter="2">
      <Producers>
        <BitmapExpression>
          <Expression>
<![CDATA[//

float u = 0.5-X;
float v = 0.5-Y;

float s = 8-sqrt(u*u+v*v)*16;

Pixel.R = s;
Pixel.G = s;
Pixel.B = s;
Pixel.A = 1;]]>
          </Expression>
        </BitmapExpression>
      </Producers>
    </Bitmap>
    <Material Name="PixelMaterial" Shading="1" Light="0">
      <Textures>
        <MaterialTexture Texture="PixelBitmap" TextureWrapMode="2" TexCoords="1"/>
      </Textures>
    </Material>
    <Array Name="Pixels" Type="4" Dimensions="1" SizeDim1="16" SizeDim2="16"/>
    <File Name="PixelsFile" FileName="Paint.dat" Encoding="1" TargetArray="Pixels"/>
  </Content>
</ZApplication>

K

[VilleK]

Maybe, if the /assets/ is not manually set in the File component path, we could search for the file in the extPath for the user files, and if it's not there, fallback to the dataPath? Would that work for reading AND saving?

I think originally we only considered two kinds of files. Either asset files that are readonly and part of the apk. Or settings/highscore files that are read/write and those would go to the data path. I think as long as we change the data path to a path that has read/write access then it should be fine.

[Ats]

It turns out that string array writing was never implemented. I added that now

Thanks, Ville! It works perfectly.

Not sure why you'd want to use "char" encoding for a paint program

Yeah, I started from the FileDemo and got a bit lost in the loops trying to detect the letter A
Thanks for your great and simple example!

It also confirms a bug I found while testing: the FileName path behaves differently depending on the target platform.

For Android compatibility, I put Paint.dat file in the "assets" folder, and renamed it to paint.txt so I could easily view it on the phone.
I also added a Timer to save the file every 5 seconds, since I don't think OnClose is called when the app shuts down on Android.

I tested /assets/ since ZgePlatform_Android expects that to access files inside the APK. But the Help documentation says:

FileName
The name of the file that will be opened, in a path relative to the current project. Examples: "Data.txt", "levels\Level".

So I tried both styles, with different results:

Preview mode
assets/paint.txt OK
/assets/paint.txt NOT WORKING
This is problematic for cross-platform compatibility, since Android expects /assets/.

Windows exe
assets/paint.txt OK
/assets/paint.txt OK

Linux
assets/paint.txt OK
/assets/paint.txt OK

Android
assets/paint.txt NOT WORKING
Expected, since Android uses "/assets/" to read from inside the APK
If I manually use Android Studio to create the "assets" folder in
/data/user/0/com.mydomain.paint/files/assets/paint.txt
it works for both reading and writing.

/assets/paint.txt OK but not for writing (which is expected since the file is read-only inside the APK)


I suppose the decision to require the path to start with /assets/ on Android is to avoid accidentally loading a file whose name just starts with "assets". In that case, Preview mode should also support /assets/paint.txt.

I think as long as we change the data path to a path that has read/write access then it should be fine.

Then I can simply replace dataPath with extPath. They are almost the same, but extPath points to a location the user can access, making it more suitable for placing custom user files.

[Kjell]

Hi Ats,

It also confirms a bug I found while testing: the FileName path behaves differently depending on the target platform.

Not sure if i would call this a bug ... path conventions simply differ from platform to platform.

Preview mode
/assets/paint.txt NOT WORKING

This only works when there's a "assets" folder in the root of your C drive ( assuming you're running ZGameEditor from your C drive ).

K

[Ats]

So it's not a bug, but it is a bit strange that the path behaves differently only on Preview.

If I want to produce a Windows EXE, a Linux binary, and an Android APK from the same project, it's odd to be required to use "/assets/" to embed files in the APK, while the same path searches for "C:\assets\", but only in preview mode. So if I set the file path to run on Android, I can't preview it in the editor. I would expect the path to be relative to the project folder, like with all the other targets.

This only works when there's a "assets" folder in the root of your C drive ( assuming you're running ZGameEditor from your C drive ).

I placed my assets folder in multiple locations:

Code: Select all

C:\assets\
C:\ZGameEditor\assets\
C:\ZGameEditor\Projects\FileDemo\assets\

But nothing is found in preview mode when the path is "/assets/"

[VilleK]

I would expect the path to be relative to the project folder, like with all the other targets.

I agree, we should make it so that if you specify "assets/" then this should work on Windows too and be assumed to be a subfolder to the exe-file. Windows can normally deal with paths that have a mixture of forward slashes and backslashes.

[Kjell]

Hi guys,

I agree, we should make it so that if you specify "assets/" then this should work on Windows too and be assumed to be a subfolder to the exe-file. Windows can normally deal with paths that have a mixture of forward slashes and backslashes.

Would be convenient if ZGE automagically made absolute and relative paths work cross-platform yes However, "assets/" isn't the problem, that works just fine ( i think )? It's "/assets/" that doesn't work properly / consistently.

K