[binarycaching] linux copy_file is restricted to 2GiB

[Osyotr]

Describe the bug
I'm building qt5-webkit for x64-linux-dynamic. The build is successful and vcpkg kindly creates binary cache for it. However, when I restore the library from cache, one file gets corrupted (truncated from 2.7gb to exactly 2gb, see screenshot). If I manually unpack the archive (e.g. unzip -qq /home/osyotr/.cache/vcpkg/archives/7b/7bc45507c4cc57f59860de97c1c38bdce12b35a07d5ecc3c8136b4ad44f6af75.zip -d/mnt/d/my_project/vcpkg_installed/x64-linux or just drag'n'drop from 7zip), the file doesn't get corrupted.
I suspect the real problem is not in unpacking, but in transferring files between packages/ dir and vcpkg_installed/ dir.
I also suspect that this is a regression because I was able to link against the library before updating ~2 months old vcpkg instance.
Maybe microsoft/vcpkg-tool#205? @BillyONeal

Environment

• OS: WSL2 Debian

To Reproduce

1. Install WSL2 on C:/ (this is default)
2. Put vcpkg.json on D:/
3. From inside WSL run ./vcpkg install qt5-webkit --overlay-ports=vcpkg_overlay_ports --overlay-triplets=vcpkg_overlay_triplets --debug
4. After successful build remove ~/vcpkg/packages/ and vcpkg_installed folders.
5. Repeat the command from 3.
6. Compare sizes of vcpkg_installed/lib/libQt5WebKit.so.5.212.0
   I've attached overlay ports and triplets I used to build the library, as well as vcpkg.json and vcpkg-configuration.json.
   qt5-base overlay port and x64-linux triplets are probably not needed but I attached them anyway. (both contain rpath fixes but I think the issue can be reproduced with official x64-linux triplet).
   vcpkg_overlay_ports_triplets.zip

Failure logs
No errors reported.

Additional context

[BillyONeal]

Oh dear, this is what I get for assuming the old code was correct.

As part of the indicated PR, I reused the existing code that called sendfile from rename_or_copy for the new copy_file, with the assumption that that had been in the product forever and thus had to be correct.

Well... https://man7.org/linux/man-pages/man2/sendfile.2.html#NOTES

[strega-nil]

Oh dear, this is what I get for assuming the old code was correct.
As part of the indicated PR, I reused the existing code that called sendfile from rename_or_copy for the new copy_file, with the assumption that that had been in the product forever and thus had to be correct.
Well... https://man7.org/linux/man-pages/man2/sendfile.2.html#NOTES

are you kidding me :'(

edit: oh, note this here:

The glibc sendfile() wrapper function transparently deals with the kernel differences [between sendfile() and sendfile64()].

[JackBoosY]

The glibc sendfile() wrapper function transparently deals with the kernel differences [between sendfile() and sendfile64()].

So we should use sendfile64() in x64 arch, right?

[BillyONeal]

edit: oh, note this here:

The glibc sendfile() wrapper function transparently deals with the kernel differences [between sendfile() and sendfile64()].

As far as I can see sendfile64 only changes the type and behavior of the offset member, it doesn't enable copying more than 2 GiB in a single call.

The glibc sendfile() wrapper function transparently deals with the kernel differences [between sendfile() and sendfile64()].

So we should use sendfile64() in x64 arch, right?

I don't think it's arch dependent.

[BillyONeal]

I was not able to test with this example because GCC 9.3 emits a ton of "-Wno-redundant-move" errors which causes that overlayed qt build to fail, but I still think the PR I submitted in the tool repo should fix the issue if you want to try building it yourself.

[autoantwort]

Would be fixed by #22629

[BillyONeal]

Fixed by #22629