SlideShare a Scribd company logo

gtkgst video in your widgets!

•
•
Y
ystreet00

A look into the current ways of displaying GStreamer videos in Gtk+ applications. Presented at GNOME Asia 2015

1 of 20
Download to read offline
gtkgst: Video in Your Widgets! https://github.com/ystreet/gst-plugins-bad/tree/gtk Matthew Waters (ystreet00) GNOME Asia 2015 9 May 2015
Who Am I ● Australian ● GStreamer ● OpenGL
What is GStreamer? ● Pipeline-based media framework ● Basic building blocks: elements connected via pads ● Set of libraries with abstract API ● Plugins for specific features – Often wrapping other libraries, e.g. libav/ffmpeg, OpenCV, various codec libraries
What is GStreamer? ● Should have gone to Nirbheek's talk yesterday!
What is Gtk+? ● Cross-platform GUI widget toolkit – OS X, Windows, Linux (X11 + Wayland) – Input (Keyboard, Touch, Mouse) – Standard widgets (Buttons, Text, Containers) – Custom widgets
Computer Video ● Colour spaces/gamut ● Chroma sampling ● Scaling ● Splitting/Cutting/Editing ● Effects ● Generation
Video in GUI Toolkits ● Embed a 'window' into another window ● Watch out for threads! ● App has to know about the window system in use
GstVideoOverlay/GstNavigation ● Provide a window handle to render video into ● Notify the video sink of size changes ● Push key and mouse events into GStreamer
GstVideoOverlay GstElement *playbin = gst_element_factory_make (“playbin”, NULL); g_object_set (playbin, “uri”, “file:///path/to/file.mp4”, NULL); GtkWidget *window = gtk_window_new (GTK_WINDOW_TOPLEVEL); GtkWidget *video_window = gtk_drawing_area_new (); gtk_container_add (GTK_CONTAINER (window), video_window); gtk_widget_show_all (window); GdkWindow *video_window_xwindow = gtk_widget_get_window (video_window); gulong embed_xid = gtk_window_get_xid (video_window_xwindow); gst_video_overlay_set_window_handle (GST_VIDEO_OVERLAY (playbin), embed_xid); gst_element_set_state (playbin, GST_STATE_PLAYING);
GstVideoOverlay - demo
Alternative - gtkgst ● Renders your video into a GtkWidget provided by the GStreamer sink ● Software implementation uses cairo into a GtkDrawingArea ● Place the provided GtkWidget wherever in the widget hierarchy
gtkgst - GL ● Gtk 3.16 has new GL support ● GStreamer 1.4 has GL support ● Why don't we try integrating them?
What is OpenGL? ● Low level 3D graphics API ● Been around since the 1990's ● Uses dedicated hardware ● Cross-platform
OpenGL in GStreamer ● Contained in the libgstgl library ● GstGLDisplay, GstGLContext are your two main objects. Analogous to GdkDisplay and GdkGLContext ● Element base classes, color conversion, uploaders, GL version/extension checking ● Also contains various elements – gltestsrc, glvideomixer, glimagesink, gleffects, etc
Demo videotestsrc ! glfiltercube ! gtkglsink
Implementation - gtkgst ● ~1000 lines of code ● Relies heavily on the GStreamer GL library and the GL implementation ● Similar in style and scope to avsamplebuffersink/caopengllayersink
gtkgst - GL ● Currently a proof of concept ● Currently X11 only ● System load causes frame jitter ● Uses Gtk+'s GL context ● Wayland might not be needed due to efficient/correct subsurfaces
Uses ● Anytime you want to display video from GStreamer into a Gtk+ application – e.g. Videos/Totem
Future ● Other window systems planned (wayland, OS X, Windows) ● GSK
Thanks! ● https://github.com/ystreet/gst-plugins-bad/tree/gtk ● ystreet00 on #gstreamer on freenode

More Related Content

gtkgst video in your widgets!

  • 1. gtkgst: Video in Your Widgets! https://github.com/ystreet/gst-plugins-bad/tree/gtk Matthew Waters (ystreet00) GNOME Asia 2015 9 May 2015
  • 2. Who Am I ● Australian ● GStreamer ● OpenGL
  • 3. What is GStreamer? ● Pipeline-based media framework ● Basic building blocks: elements connected via pads ● Set of libraries with abstract API ● Plugins for specific features – Often wrapping other libraries, e.g. libav/ffmpeg, OpenCV, various codec libraries
  • 4. What is GStreamer? ● Should have gone to Nirbheek's talk yesterday!
  • 5. What is Gtk+? ● Cross-platform GUI widget toolkit – OS X, Windows, Linux (X11 + Wayland) – Input (Keyboard, Touch, Mouse) – Standard widgets (Buttons, Text, Containers) – Custom widgets
  • 6. Computer Video ● Colour spaces/gamut ● Chroma sampling ● Scaling ● Splitting/Cutting/Editing ● Effects ● Generation
  • 7. Video in GUI Toolkits ● Embed a 'window' into another window ● Watch out for threads! ● App has to know about the window system in use
  • 8. GstVideoOverlay/GstNavigation ● Provide a window handle to render video into ● Notify the video sink of size changes ● Push key and mouse events into GStreamer
  • 9. GstVideoOverlay GstElement *playbin = gst_element_factory_make (“playbin”, NULL); g_object_set (playbin, “uri”, “file:///path/to/file.mp4”, NULL); GtkWidget *window = gtk_window_new (GTK_WINDOW_TOPLEVEL); GtkWidget *video_window = gtk_drawing_area_new (); gtk_container_add (GTK_CONTAINER (window), video_window); gtk_widget_show_all (window); GdkWindow *video_window_xwindow = gtk_widget_get_window (video_window); gulong embed_xid = gtk_window_get_xid (video_window_xwindow); gst_video_overlay_set_window_handle (GST_VIDEO_OVERLAY (playbin), embed_xid); gst_element_set_state (playbin, GST_STATE_PLAYING);
  • 11. Alternative - gtkgst ● Renders your video into a GtkWidget provided by the GStreamer sink ● Software implementation uses cairo into a GtkDrawingArea ● Place the provided GtkWidget wherever in the widget hierarchy
  • 12. gtkgst - GL ● Gtk 3.16 has new GL support ● GStreamer 1.4 has GL support ● Why don't we try integrating them?
  • 13. What is OpenGL? ● Low level 3D graphics API ● Been around since the 1990's ● Uses dedicated hardware ● Cross-platform
  • 14. OpenGL in GStreamer ● Contained in the libgstgl library ● GstGLDisplay, GstGLContext are your two main objects. Analogous to GdkDisplay and GdkGLContext ● Element base classes, color conversion, uploaders, GL version/extension checking ● Also contains various elements – gltestsrc, glvideomixer, glimagesink, gleffects, etc
  • 16. Implementation - gtkgst ● ~1000 lines of code ● Relies heavily on the GStreamer GL library and the GL implementation ● Similar in style and scope to avsamplebuffersink/caopengllayersink
  • 17. gtkgst - GL ● Currently a proof of concept ● Currently X11 only ● System load causes frame jitter ● Uses Gtk+'s GL context ● Wayland might not be needed due to efficient/correct subsurfaces
  • 18. Uses ● Anytime you want to display video from GStreamer into a Gtk+ application – e.g. Videos/Totem
  • 19. Future ● Other window systems planned (wayland, OS X, Windows) ● GSK