NTFS-3G and lwext4 portlibs.

[DarkMatterCore]

Following WinterMute's suggestion, I'd like to continue the discussion from this PR here in the forums. If you're not familiar with what was discussed, please go read the comments on the PR first.

Basically, WinterMute and I were discussing the possibility of pitching both NTFS-3G and lwext4 portlibs (currently available in libusbhsfs' repository) to the pacman-packages repository, in order to make it easier for everyone else to install and use those libraries with any Switch homebrew application.

In this particular case, though, TSRBerry (the creator of that PR) and I are very interested in streamlining the building process for libusbhsfs. It is a statically linked library that acts as a userland USB Mass Storage device driver + filesystem mounter for the Switch, which I created from scratch a couple of years ago.

It is clear to me and others that providing a system-wide solution using a background process (e.g. custom sysmodule, custom FS mitm, etc.) is not feasible. As I already explained in that PR, XorTroll and I forked Atmosphère around Q3 - Q4 2019 to create a custom sysmodule for this exact purpose, which we ended up calling fsp-usb. The idea was to eventually pitch this driver to Atmosphère, so every single user could benefit from it.

While it did work, the limited amount of resources inherent to each background system process on the Switch made it too difficult to implement all the features we wanted to provide, like support for multiple partitions (using MBR/EBR/GPT/SFD partition layouts) and multiple UMS devices at once, as well as NTFS and EXT support. Running into memory issues in this kind of projects is not uncommon -- it is normal for users themselves to run into problems if they use more than two custom sysmodules at once. Atmosphère itself also has to steal memory from other processes while dealing with certain official games.

After the discussing the topic with SciresM via Discord, we decided to scrap the project altogether. I went onto writing the UMS driver from scratch as a static library and got excellent results, which brings us back to the topic at hand.

Would you please reconsider adding these two portlibs to the pacman-packages repository? I'm willing to make any modifications to the current PKGBUILD scripts I'm using if deemed necessary.

Thanks for reading.

[WinterMute]

Apologies for leaving this hanging so long. I looked and then managed to take myself down a rabbit hole followed by a whole bunch of yak shaving for something else.

This section of your ntfs-3g patch shouldn't be needed https://github.com/DarkMatterCore/libus ... ch#L16-L75. I actually added sys/mkdev.h to our toolchain headers recently while looking at building this for wii/gamecube. See https://github.com/devkitPro/pacman-pac ... pc/ntfs-3g which I updated for latest upstream recently. What does the ntfstime.h patch fix?

For liblwext4 the headers are already prefixed with ext4_ so moving them to a separate folder seems unnecessary really. Using a commit hash as a package version won't work, pacman expects increasing semantic versioning to detect new versions, it's not simply most recent. This can of course be subverted by using an actual version number for the package but using the commit hash for the source archive.

Happy enough to take PRs for these packages with the suggested changes.

[DarkMatterCore]

Apologies for leaving this hanging so long. I looked and then managed to take myself down a rabbit hole followed by a whole bunch of yak shaving for something else.

This section of your ntfs-3g patch shouldn't be needed https://github.com/DarkMatterCore/libus ... ch#L16-L75. I actually added sys/mkdev.h to our toolchain headers recently while looking at building this for wii/gamecube. See https://github.com/devkitPro/pacman-pac ... pc/ntfs-3g which I updated for latest upstream recently. What does the ntfstime.h patch fix?

For liblwext4 the headers are already prefixed with ext4_ so moving them to a separate folder seems unnecessary really. Using a commit hash as a package version won't work, pacman expects increasing semantic versioning to detect new versions, it's not simply most recent. This can of course be subverted by using an actual version number for the package but using the commit hash for the source archive.

Happy enough to take PRs for these packages with the suggested changes.

Hi there. It's alright, don't worry. I've been pretty busy myself in the last few weeks as well.

I just noticed "sys/mkdev.h" is indeed present after updating devkitA64, thanks. The ntfstime.h patch prevents the header from defining a timespec struct on its own -- we're already using the timespec struct from time.h in other parts of libusbhsfs.

Code: Select all

/*
 * assume "struct timespec" is not defined if st_mtime is not defined
 */
#if !defined(st_mtime) & !defined(__timespec_defined)
struct timespec {
	time_t tv_sec;
	long tv_nsec;
} ;
#endif

Apparently, ntfstime.h expects the "__timespec_defined" macro to be defined in time.h (or in one of the header files it may reference). But since it's not there, we had to implement that workaround.

As for lwext4, the changes sound simple enough. I'll work on it.

Thanks again!

[DarkMatterCore]

I just submitted both PRs to the pacman-packages repository:

• NTFS-3G: https://github.com/devkitPro/pacman-packages/pull/345
• lwext4: https://github.com/devkitPro/pacman-packages/pull/346

Let me know if any extra changes are needed. I'll update the build instructions from the libusbhsfs repository if the PRs get merged.

Thanks again.