Question - AVCaptureSessionOutputs
What is the difference between the AVCaptureVideoDataOutput
and the AVCaptureVideoDataOutputSampleBufferDelegate
?
From video Live camera feed in SwiftUI with AVCaptureVideoPreviewLayer
AVCaptureVideoDataOutput
The class that allows us to access the frames in a pixel buffer.
AVCaptureVideoDataOutputSampleBufferDelegate
We are using this protocol to declare the ViewController
which contains the AVCaptureVideoDataOutput
as a delegate. This delegate is notified to receive new frames.
Explanation
AVCaptureVideoDataOutput
and AVCaptureVideoDataOutputSampleBufferDelegate
are two different things. We use the former to write frames into the buffer. And the latter is used to notify the detector part once a new frame is available
The former is a class, the latter is a Protocol
. In the ViewController
class, we define the AVCaptureVideoDataOutput
to write frames into a pixel buffer. Whenever there is a new frame, we want the detector to run detection on it. But how does the detector know that a new frame is available? This is where the AVCaptureVideoDataOutputSampleBufferDelegate
comes in.
We make the ViewController conform to the AVCaptureVideoDataOutputSampleBufferDelegate
protocol and we also assign the ViewController
itself its own delegate (line 103). The protocol allows the ViewController
to implement a method that is called whenever there is a new frame in the pixel buffer. We can then use this to run the detector on the frame.
This is how Apple requires us to feed frames to CoreML models. For more information regarding protocols and delegation in Swift check out the Swift documentation: https://docs.swift.org/swift-book/LanguageGuide/Protocols.html
Comments
Thanks for demystifying this for me; I was finding getting Apple’s examples to work in SwiftUI challenging in terms of structuring and performance so this was a big help. Just dipping my toes into object recognition and ML generally so need all the help I can get…