kext | FUSE kernel extension | Storage library

Turns out IOPCIDevice::_CopyDeviceMemoryWithIndex was indeed the function needed to implement this (but the fact that it's private is still an inconvenient).

Bellow is some sample code showing how this could be implemented (the code uses MyDriver for the driver class name and MyDriverUserClient for the user client).

Relevant sections from MyDriver.cpp implementation:

The desired output does not look correct. Your "dictionary of dictionary" does not have a key with it. I suspect it should be something like

I have updated to latest Android Studio preview release on my MacBook Pro. It's not crashing now. You can find all android studio release and its preview release here: https://developer.android.com/studio/archive

I don't have a 100% answer as to why it's going wrong in your case. To load a dext on an unmodified system, you definitely need the following when codesigning your dext:

• Sign with a 'Developer ID Application' certificate identity and be notarised, or signed with an 'Apple Developer' certificate
• Embedded code signing entitlements including the generic DriverKit entitlement, any family-specific entitlements, and entitlements regarding user client access if necessary.
• A provisioning profile from Apple which matches:
  1. The code signing entitlements you are embedding in the dext.
  2. The type and specific instance of the code signing identity you are using to sign the dext.
  3. The Application & Bundle ID of the dext.
  4. If using an Apple Developer signing identity, the hardware IDs of the Macs on which you will be testing the dext.

For local testing, you can try the following things to temporarily work around code signing issues:

• Disabling System Integrity Protection (SIP). For purposes of DriverKit extensions, this disables some codesigning checks.
• Disabling DriverKit-specific entitlements checks. This turns off checking for a lot of family-specific entitlements. To disable the checks, set flag 0x8000 in the dk kernel boot argument, which is a bitfield. Note that unless you set flag 0x1, DriverKit will be entirely disabled. So use dk=0x8001 to disable DriverKit entitlement checks.
• Disabling AMFI checks. AMFI will normally kill your process if you try to claim entitlements which need to be enabled in a provisioning profile. You can disable AMFI with the amfi_get_out_of_my_way=1 kernel boot argument.

Obviously, you only have limited control over the provisioning profile, as the entitlements in it must be approved by Apple. (For this reason, if you're planning to ship your dext publicly, I generally recommend you try to work out what entitlements you might need before starting full development on the dext, and request them from Apple. The process can take months.) So while it would be interesting to know the exact minimum combination of workarounds required when you don't have these profiles, I haven't exhaustively tested this out yet. I realise this doesn't help much with getting your dext as close to shipping as possible while still waiting for Apple to grant missing entitlements. Hopefully one of these days I can systematically explore and document all of this.

Will Apple allow kernel extensions also in future versions of (Apple Silicon/M1 based) operating systems? Or is there at least a clear deadline?

Apple doesn't really give timelines, and they also occasionally break promises of support.

However, this sort of hard API deprecation and removal usually is done as part of a major release, so you will typically get deprecation notice for it at WWDC one year, users might start seeing deprecation notices when the .0 of the OS release ships at the earliest, and sometimes the .3 or .4 revision. Then you'll typically be told at the next WWDC that the API is blocked in the upcoming release, so by that point you should have implemented a replacement.

If not, what is Apple's officially suggested replacement for KEXT based VFS solutions?

As far as I'm aware, NSFileProviderReplicatedExtension is currently the only one.

Will the API of an NSFileProviderReplicatedExtension be improved to behave like a "real" file system so that above mentioned flaws will no longer be an issue?

Other than via beta SDKs, Apple generally doesn't pre-announce future APIs.

My advice:

• File issues for each of the file provider shortcomings you are hitting using Feedback Assistant. (Radar)
• File an "enhancement request" feedback issue with Apple for a "real" file system API replacement for the VFS KPI.
• If your vfs kext is critical to your business/product, I suggest additionally asking Apple's DTS via a TSI what they recommend for your situation. Reference the feedback IDs of the issues filed, otherwise they will recommend that you file issues.

Your hardware setup sounds fine. Note that you don't necessarily need a direct ethernet cable connection, you can use your normal office-/home-wide wired network, it's fine to have a switch (or even more than one) between debugger and target. The debugger machine can even be using wifi as long as it can reach the target's wired network connection that way, though it's not recommended. The Thunderbolt Ethernet interface need not be Apple branded, but its driver does need to include kernel debugging support. In practice, for example, I've successfully used the Ethernet port in a OWC brand Thunderbolt Dock as well. Using one of Apple's own adapters of course guarantees it will work.

If the boot-args on your target machine are the same as what you used in the VM, that's probably where your problem is coming from. You should specify the device to be used for kernel debugging, using the kdp_match_name= option. Use the ifconfig command in the Terminal to work out which of the listed devices is your Thunderbolt ethernet adapter (enX), possibly based on the IP address. en0 is typically wifi in Mac laptops, so you're usually looking for X > 0.

So you'll end up with something like kdp_match_name=en1 added to your boot-args. Make sure to reboot the machine cleanly before attempting a debugging session after updating the boot-args setting.

A few more comments:

kext-dev-mode=1

This option no longer has any effect. It only existed in OS X 10.10. You can remove it.

I didn't find the exact KDK version, but I don't think it should be the problem here

Having a KDK version that doesn't match the target's exact kernel binary UUID will not prevent a connection, but it will prevent you from doing any meaningful debugging as the memory layout of threads etc. will not be available.

My debugger is Big Sur, and my debuggee is Catalina

If you ever run Big Sur on your target, make sure to add wdt=-1 to your boot-args or the hung/crashed machine will reboot before you get a chance to connect with the debugger.

I have just checked, and the crucial detail appears to be that you are trying to access this function on arm64/aarch64. As it turns out, it's exported in the "unsupported" KPI for x86_64, but not on arm64:

• Unsupported.x86_64.exports in xnu source
• Unsupported.arm64.exports in xnu source

There's no straightforward way of accessing unexported symbols. If you know the offset of a symbol in the exact version of the running kernel, you should be able to compute the address by offsetting from a known function address; at least, this worked on x86-64. arm64 may require extra effort due to PAC (pointer authentication).

As this circumvents Apple's policies, I don't recommend using this type of technique in a shipping product.

Since macOS 11, Apple has updated documentation of the File Provider API which can be used to sync files between client and server and may serve as a replacement for file system drivers. Until this day Apple has not officially announced that functionality on macOS. The API is similar to the File Provider API that Apple has provided for iOS 11 some time ago but still it has some differences in mostly how main FileProvider class extension works and which abilities it has. There is no examples at this moment that I’ve seen but it looks like this functionality works at least on basic level. Based on what people say on the forums.

On Windows, Microsoft has introduced the Cloud Storage Provider (Cloud Sync Engine) API for Windows 10 in the year 2018 (Windows Creators update). It is used in One Drive for Windows. Here are some examples on GitHub in C++ and in C#.

So, I'm not sure if this is what is supposed to happen or not, but placing the kext in /Library/Extensions instead of /System/Library/Extensions worked.

I'm not sure if this is the correct thing to do but since it works, I'm not complaining.