fyne-cross running fixuid on everything before every command tanks performance

[squash]

Describe the bug:

Building on a lower-end system (in my case, 8gb Pi 4 with USB3 attached SSD). it seems every Docker execution is doing a fixuid first, changing ownership on a ton of files in the container every time. The fixuid process takes a minute or three before each intended command begins.

Most of the time appears to be spent resetting ownership of the Zig compiler and its assets in /usr/local/zig, which I would expect should be correct when building the container? For example, using fyne-cross to compile Go's smallest "hello world" takes almost 14 minutes.

time fyne-cross linux -arch amd64 -debug > debug.txt
...
real 13m56.435s
user 0m13.277s
sys 0m34.767s

To Reproduce:

Using fyne-cross to compile any code

Device and debug info (please complete the following information):

Device info

• OS: Linux raspberry 5.19.0-1015-raspi #\22-Ubuntu SMP PREEMPT Mon Mar 6 10:35:33 UTC 2023 aarch64 aarch64 aarch64 GNU/Linux
• Version: Ubuntu 21.10 Arm64
• Go version: go version go1.20.1 linux/arm64
• fyne-cross version: 1.4 rc1
• Fyne version: fyne.io/fyne/v2 v2.3.3

Debug info

```
[✓] "bin" dir cleaned: /app/fyne-cross/bin/linux-amd64
/usr/bin/docker run --rm -t -w /app -v /home/squash/go/src/github.com/squash/modbus-cli:/app:z -v /home/squash/.cache/fyne-cross:/go:z --platform linux/arm64 -u 1001:1001 --entrypoint fixuid -e CGO_ENABLED=1 -e GOCACHE=/go/go-build -e GOARCH=amd64 -e CC=zig cc -target x86_64-linux-gnu -isystem /usr/include -L/usr/lib/x86_64-linux-gnu -e CXX=zig c++ -target x86_64-linux-gnu -isystem /usr/include -L/usr/lib/x86_64-linux-gnu -e GOOS=linux docker.io/fyneio/fyne-cross-images:linux rm -rf /app/fyne-cross/dist/linux-amd64
fixuid: fixuid should only ever be used on development systems. DO NOT USE IN PRODUCTION
fixuid: updating user 'docker' to UID '1001'
fixuid: updating group 'docker' to GID '1001'
fixuid: recursively searching path /
fixuid: chown /home/docker
fixuid: chown /home/docker/.bash_logout
fixuid: chown /usr/local/zig/doc/langref.html
fixuid: chown /usr/local/zig/doc/std
fixuid: chown /usr/local/zig/doc/std/data.js
...
```
Make sure the output does not contain any sensitive information.

-->

[squash]

Quick note, logging with -debug in my test, we are doing the fixuid process 11 times for my hello world app being compiled for linux amd64.

[andydotxyz]

I have noticed that if you turn on logging it basically melts the CPU for the same reason.

[Bluebugs]

I am not to versed in docker, but maybe we can restrict the call to fixuid to only a subset of the image which would speed things up.

In the meantime, I will recommend using podman which doesn't need fixuid and work great on arm64. A raspberry pi will still be slow to do the compile of any large application, but still doable use case now.

[squash]

Editing fyne-cross to skip the fixuid takes that hello world app from 14 minutes to 14 seconds to compile with everything else identical. I'm looking to see if we can more narrowly use fixuid or handle the issue in a different way. The Pi isn't a i9 or anything but it's not this slow.

[squash]

I will preface this with saying I'm not clear on the entire problem the fixuid was attempting to solve, however in my testing I'm able to remove the fixuid process completely and have a successful build by setting the HOME environment variable to a writable directory (using /tmp).

The zig compiler wants to create ${HOME}/.cache/.zig but HOME is empty for users other than docker, thus it tries and fails to create /.cache/.zig

After removing this the build time for my fyne app has dropped from 14 minutes to under 1 minute.

[Bluebugs]

If I remember correctly, the reason we use fixuid is that we need to map all file created by the user to be the same as the host user or something along that line. By pointing HOME to be a temporary directory, this isn't necessary as there will not be any file from previous run and so nothing need to be mapped.

The drawback is that we loose the cache for zig between two build. Maybe we can do the above and also provide an option to mount ${HOME}/.cache/zig from the host into the container.

[squash]

OK I think I understand the problem that was being solved. I think a much easier solution would be to preface the execution step with creating a user with appropriate settings rather than doing a bunch of permissions changes.

As it stands, if you happen to be UID 1000 (should be most users on personal workstations), your username will map to the docker user inside the container which has a home directory and will have permissions to create that cache file. For any other user, I believe the cache folder will attempt to be created in / and won't be accessible between targets anyway.

Two things then:

. Am I wrong in the assumption that the user executing fyne-cross is expected to be the owner of the source tree being compiled?

. Could I suggest instead of fixuid, a small wrapper script that creates a user with the specified uid/gid and a valid HOME?

[Bluebugs]

• I think it is a correct assumption that the user of fyne-cross has the access right to the source code and it is out of scope of fyne-cross to change any host access right.
• fyne-cross has a prepare function that can execute a code before we start running any command. That here: https://github.com/fyne-io/fyne-cross/pull/176/files#diff-c61ac65440c99671df0949ba4aeffee4e31e7c10e918cd184ee1206751fb432cR179 . I am guessing you could run that script there and see if that work out when your user id is different from 1000.

[lucor]

OK I think I understand the problem that was being solved. I think a much easier solution would be to preface the execution step with creating a user with appropriate settings rather than doing a bunch of permissions changes.

In the previous versions of fyne-cross we were trying to solve the uid problem with docker using gosu and creating a user with a custom script. When added podman support we switched to fixuid to fix permission issues (#41)

It seems that fixuid allows to configure the path we want to change permissions: https://github.com/boxboat/fixuid#specify-paths-and-behavior-across-devices, so another possible way could be to limit ownership change only to:

• the output folder (i.e. /fyne-cross)
• the cache folder (i.e. $HOME/.cache/zig)

Anyway I agree that a solution that do not changes file permissions on shared volumes with docker and that works with other engines would be a nice to have.

[squash]

@lucor the issue we run into in Docker though is that unless your are uid 1000, ${HOME} isn't set at all. I don't see a way to solve this that doesn't either:

1. Use a script to create a user matching the current uid/gid with a valid home directory that exists and is owned by them and run that instead of fixuid
2. Disable the zig cache completely, which I don't think it currently allows
3. Ignore the user but set HOME to somewhere we can write such as /tmp and lose whatever zig cache might gain us
4. Ignore the user but set HOME to somewhere with a mapped .cache for zig that exists outside the container and persists across builds

I've tested 1 (10 line shell script) and 3 (HOME=/tmp) locally, 4 would be straightforward but I would want a suggestion on where that .cache would live.

I haven't used podman, i'll see about testing that out as well. It's not clear where the conflict with gosu and podman was but maybe that's more easily solved if my understanding of the problem is correct.

[Bluebugs]

I like 4. and we could map it to the host ~/.cache/zig. We can also force the HOME environment variable, I would think, from the image parameters and so point to the same directory whatever your user uid is.