video-sdk-samples | Samples demonstrating how to use various APIs | Video Utils library

my goal is to encode the main framebuffer of my Windows machine using nvenc and stream its content to my iPad using the VideoToolbox API

The code I use to encode the h264 stream is basically a copy/paste of https://github.com/NVIDIA/video-sdk-samples/tree/master/nvEncDXGIOutputDuplicationSample the only change is that instead of writing to a file, I do send the data

For the decoding I do use https://github.com/zerdzhong/SwfitH264Demo/blob/master/SwiftH264/ViewController.swift#L71

The encoding work perfectly when I write all the contents to a file, I am able to use a h264->mp4 online converter without issue, the problem is that the decoder gives me the error kVTVideoDecoderBadDataErr in the function decompressionSessionDecodeFrameCallback

So for what I tried:

• Firsly using an h264 analyzer I found that the frame order are: 7/8/5/5/5/5/1...
• I found that nvenc does encode the frames 7/8/5/5/5/5 in only one packet
• I did try to separate this packet into multiple ones using the sequence (0x00 0x00 0x00 0x01), it gave me the frames 7/8/5 separately
• As you can see I only got one 5 frame which is around 100KB, the H264 analyzer said that there are four 5 frames (which are something like 40KB, 20KB, 30KB, 10KB)
• Using a hex file viewer I saw that the sequence separating those 5 frames were (0x00 0x00 0x01) instead, tried to also separate them but I got the exact same VideoToolbox error while decompressing

here is the code I use to separate and send the frames: The protocol is simply PACKET_SIZE->PACKET_DATA The swift code is able to read the NALU types so I am confident that this is not the issue