[RFC] wl_surface video protocol extension

Hi everybody We're having needs for a streaming_video-capable wl_surface. An extended surface that could "queue buffers up" along with presentation timestamps in the compositor so that videosink clients (like gstreamer's wayland videosink) could more effectively and precisely synchronize audio and video on specific time cues. I'd like your comments/opinion about this first draft of the protocol. Protocol and interfaces names, language, set of requests, etc... The idea for now is to let the clients queue buffers in advance and clear the queue abruptely for cases where they'd need, e.g. to "pause" the streaming.We're still having a whole bunch of implementation-wise questions, but that's a start. It'd be awesome if I had your feedback on this protocol. Thanks ! Frederic Plourde <?xml version="1.0" encoding="UTF-8"?> <protocol name="videosurface"> <copyright> Copyright © 2012-2013 Collabora, Ltd. Permission to use, copy, modify, distribute, and sell this software and its documentation for any purpose is hereby granted without fee, provided that the above copyright notice appear in all copies and that both that copyright notice and this permission notice appear in supporting documentation, and that the name of the copyright holders not be used in advertising or publicity pertaining to distribution of the software without specific, written prior permission. The copyright holders make no representations about the suitability of this software for any purpose. It is provided "as is" without express or implied warranty. THE COPYRIGHT HOLDERS DISCLAIM ALL WARRANTIES WITH REGARD TO THIS SOFTWARE, INCLUDING ALL IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS, IN NO EVENT SHALL THE COPYRIGHT HOLDERS BE LIABLE FOR ANY SPECIAL, INDIRECT OR CONSEQUENTIAL DAMAGES OR ANY DAMAGES WHATSOEVER RESULTING FROM LOSS OF USE, DATA OR PROFITS, WHETHER IN AN ACTION OF CONTRACT, NEGLIGENCE OR OTHER TORTIOUS ACTION, ARISING OUT OF OR IN CONNECTION WITH THE USE OR PERFORMANCE OF THIS SOFTWARE. </copyright> <interface name="wl_videocompositor" version="1"> <description summary="video-surface compositing"> The global interface exposing video-surface compositing capabilities. The aim of video-surfaces is to support streaming video coming from videosinks that typically need to queue video buffers with presentation timestamps in the compositor in order to accurately synchronize video and audio streams. </description> <request name="destroy" type="destructor"> <description summary="unbind from the videocompositor interface"> Tell the server that the client will not be using this protocol object anymore. This does not affect any other objects, wl_videosurface objects included. </description> </request> <enum name="error"> <entry name="video_surface_exists" value="0" summary="the surface already has a video-surface object associated"/> </enum> <request name="get_videosurface"> <description summary="promote a surface to a video-surface"> Create and attach a video-surface interface for the given surface. This effectively turns a plain wl_surface into a specialized video-surface with video-streaming capabilities via the use of buffer queues. If the given wl_surface already has a wl_video_surface object associated, the video_surface_exists protocol error is raised. </description> <arg name="id" type="new_id" interface="wl_videosurface" summary="the new videosurface object id"/> <arg name="surface" type="object" interface="wl_surface" summary="the surface to be turned into a video-surface"/> </request> </interface> <interface name="wl_videosurface" version="1"> <description summary="video-surface interface to a wl_surface"> An additional interface to a wl_surface object, which has been promoted to a video-surface. A video-surface contains a buffer queue and exposes requests that allow one to add a timestamped wl_buffer into the video-surface buffer queue. Video-surfaces do not declare any state in addition to the ones plain surfaces already use by default). Thus, once a plain sur- face has been created and promoted to a video-surface, buffer queuing happens through "queue_buffer requests". This means that queuing up buffers does *not* require any surface.attach nor surface.commit. Moreover, such an .attach or .commit will clear the buffer queue in order to exclusively attach the newly provided buffer via .attach. Hence, surface.attach and surface. commit should *not* be used when specifying streaming buffer content of a video-surface, as they are only relevant for single- buffer plain surfaces. At every repaint cycle, the compositor will look for past- timestamped buffers in the queue and if any, will render the most recent one of them. Older buffers will be removed from the list at that point and destroyed (an BUFFER_RELEASE event will be fired for every removed one). At every repaint cycle, future- timestamped (or posterior) buffers will be kept untouched. while enqueuing video-surface buffers, it's not necessary to specifying any 'damage' state because enqueued buffers will automatically be associated with fullscreen damage. </description> <request name="destroy" type="destructor"> <description summary="remove video-surface interface"> The video-surface interface is removed from the wl_surface object that was turned into a video-surface with wl_videocompositor.get_videosurface request. </description> </request> <request name="queue_buffer"> <description summary="enqueue a buffer into the surface's queue"> Queue a buffer along with its timestamp into the video-surface internal buffer queue for later/synchronized presentation to display hardware. </description> <arg name="buffer" type="object" interface="wl_buffer" summary="the buffer to queue"/> <arg name="timestamp" type="uint" summary="the buffer's presentation timestamp"/> </request> <request name="clear_buffer_queue"> <description summary="clear the surface's buffer queue"> Clear the surface's queue from all its buffers. None of the removed buffers will be presented at given times and associated timestamps are cleared from wl_buffer implementations. </description> </request> </interface> </interface> </protocol>