Kurento Media Capabilities
Media elements and media pipelines
The main contribution of Kurento to the multimedia development community is the creation of a novel API basing on an very high level abstraction. This abstraction makes possible to create complex and interoperable multimedia applications in a simple manner and democratizes multimedia technologies making them accessible to average non-expert WWW developers. This abstraction is based on two concepts:
Media Element. Media elements are the basic building blocks of Kurento multimedia APIs. A media element can be seen as a piece of a meccano in the sense that each media element is a "black-box" holding a specific media capability. Media elemens interact among each other through a well defined interface, which hides all the low level details of the underlying technology. Hence, application developers do not need to understand such details. This interface is based on the concept of pads. A pad is the exchange point of media streams among elements. There are two types of pads: sinks, whose function is to receive media and get it into the media element, and sources, which are in charge of getting media streams out of elements.
Figure 1. A Media Element is a building block that encapsuales a media capability and is exposed to the application developer as a blackbox cabale of receiving and issuing streams. Media elements receive media streams from sink pads and generate media streams from source (src) pads.
Media Pipeline. A Media Pipeline is a graph of connected media elements. Pipelines usually take the form of chains, where the output stream generated by one element (through its source pad) is fed into one or more other elements input streams (thorugh their sink pads). Hence, the pipeline representsa sequence of processing steps over a stream. In other words, a pipeline is the "machine" that the developer builds connecting the meccano pieces that media elements represent.
Figure2. A Media Pipeline is a multimedia application built by developers connecting the desired media elements for providing a specific media capability. This picture shows an example of pipeline between a WebRtcEndpoint (left) and a RtpEndpoint (right). Both media streams are recorded into files through RecorderEndpoints (top and bottom) and the WebRTC stream is agumented by drawing a Darth Vader helmet on top of faces appearing into the stream (middle). The final functionality is an interoperable videoconference between a WebRTC and an RTP/H.264 legacy softphone, where the videoconference is recorded and augmented.
Within Kurento, we can find two main types of media elements: Endpoints and Filters:
- Endpoints: are at the end of a chain of media elemements. Endpoints are usually associated to media comming into a pipeline or getting out of a pipeline. There are endpoints devoted to exchaning media between the network and the pipeline through specific protocols (e.g. RtpEndpoint, WebRtcEndpoint, HttpEndpoint). There are also endpoints specialized in storing and recovering media into files or repositories (e.g. PlayerEndpoint, RecorderEndpoint).
- Filters: are in the middle of a chain of media elements and usually receive a input media stream and generate a processed output media stream. Kurento provides filters specialized in computer vision tasks (e.g. ZBarFilter), in augmented reality (e.g. JackVaderFilter), etc. Many filters (e.g. computer vision filters) generate events and programmers can subscribe to them through listeners.
To learn more
If you are interested in learning more, consider reading the Kurento Media Element Toolbox Section.