Interface GLMediaPlayer
- All Superinterfaces:
TextureSequence
TextureSequence state machine
using a multiplexed audio/video stream as it's source.
Audio maybe supported and played back internally or via an AudioSink implementation.
Audio and video streams can be selected or muted via initStream(Uri, int, int, int)
using the appropriate stream id's.
Camera input can be selected using the CameraInputScheme Uri.
StreamWorker Decoding Thread
Most of the stream processing is performed on the decoding thread, a.k.a. StreamWorker:
- Stream initialization triggered by
initStream(..)- User gets notified whether the stream has been initialized or not viaattributesChanges(..). - Stream decoding - User gets notified of a new frame via
newFrameAvailable(...). - Caught exceptions on the decoding thread are delivered as
GLMediaPlayer.StreamExceptions.
GLContext, shared with the one passed to initGL(GL).
The shared GLContext allows the decoding thread to push the video frame data directly into
the designated TextureSequence.TextureFrame, later returned via getNextTexture(GL) and used by the user.
Caught exceptions on StreamWorker are delivered as GLMediaPlayer.StreamExceptions,
which either degrades the GLMediaPlayer.State to GLMediaPlayer.State.Uninitialized or GLMediaPlayer.State.Paused.
An occurring GLMediaPlayer.StreamException triggers a EVENT_CHANGE_ERR event,
which can be listened to via GLMediaPlayer.GLMediaEventListener.attributesChanged(GLMediaPlayer, int, long).
An occurred GLMediaPlayer.StreamException can be read via getStreamException().
GLMediaPlayer Lifecycle
Audio and video Stream IDs
| value | request | get |
|---|---|---|
STREAM_ID_NONE | mute | not available |
STREAM_ID_AUTO | auto | unspecified |
| ≥0 | specific stream | specific stream |
Current implementations (check each API doc link for details):
NullGLMediaPlayerOMXGLMediaPlayerFFMPEGMediaPlayerAndroidGLMediaPlayerAPI14
Implementations of this interface must implement:
public static final boolean isAvailable();
to be properly considered by GLMediaPlayerFactory.create(ClassLoader, String)
and GLMediaPlayerFactory.createDefault().
Timestamp Accuracy
Timestamp type and value range has been chosen to suit embedded CPUs
and characteristics of audio and video streaming. See TimeFrameI.
Audio and video synchronization
The class follows a passive A/V synchronization pattern.
Audio is being untouched, while getNextTexture(GL) delivers a new video frame
only, if its timestamp is less than MAXIMUM_VIDEO_ASYNC ahead of time.
If its timestamp is more than MAXIMUM_VIDEO_ASYNC ahead of time,
the previous frame is returned.
If its timestamp is more than MAXIMUM_VIDEO_ASYNC after time,
the frame is dropped and the next frame is being fetched.
https://en.wikipedia.org/wiki/Audio_to_video_synchronization
d_av = v_pts - a_pts;
Recommendation of audio/video pts time lead/lag at production:
- Overall: +40ms and -60ms audio ahead video / audio after video
- Each stage: +5ms and -15ms. audio ahead video / audio after video
Recommendation of av pts time lead/lag at presentation:
- TV: +15ms and -45ms. audio ahead video / audio after video.
- Film: +22ms and -22ms. audio ahead video / audio after video.
Test Streams
| Big Buck Bunny 24f 16:9 | ||||
|---|---|---|---|---|
| Big Buck Bunny | 320p | h264 | aac 48000Hz 2 chan | http://download.blender.org/peach/bigbuckbunny_movies/BigBuckBunny_320x180.mp4 |
| Big Buck Bunny | 240p | h264 | aac 48000Hz 2 chan | http://archive.org/download/BigBuckBunny_328/BigBuckBunny_512kb.mp4 |
| Big Buck Bunny | 720p | mpeg4 | ac3 48000Hz 5.1 chan | http://download.blender.org/peach/bigbuckbunny_movies/big_buck_bunny_720p_surround.avi |
| Big Buck Bunny | 720p | msmpeg4v2 | mp3 48000Hz 2 chan | http://download.blender.org/peach/bigbuckbunny_movies/big_buck_bunny_720p_stereo.avi |
| Big Buck Bunny | 720p | theora | vorbis 48000Hz 2 chan | http://download.blender.org/peach/bigbuckbunny_movies/big_buck_bunny_720p_stereo.ogg |
| Big Buck Bunny | 1080p | mpeg4 | ac3 48000Hz 5.1 chan | http://download.blender.org/peach/bigbuckbunny_movies/big_buck_bunny_1080p_surround.avi |
| WebM/Matroska (vp8/vorbis) | ||||
| Big Buck Bunny Trailer | 640p | vp8 | vorbis 44100Hz 1 chan | http://video.webmfiles.org/big-buck-bunny_trailer.webm |
| Elephants Dream | 540p | vp8 | vorbis 44100Hz 1 chan | http://video.webmfiles.org/elephants-dream.webm |
| You Tube http/rtsp | ||||
| Sintel | http://www.youtube.com/watch?v=eRsGyueVLvQ | rtsp://v3.cache1.c.youtube.com/CiILENy73wIaGQn0LpXnygYbeRMYDSANFEgGUgZ2aWRlb3MM/0/0/0/video.3gp | ||
| Audio/Video Sync | ||||
| Five-minute-sync-test1080p | https://www.youtube.com/watch?v=szoOsG9137U | rtsp://v7.cache8.c.youtube.com/CiILENy73wIaGQm133VvsA46sxMYDSANFEgGUgZ2aWRlb3MM/0/0/0/video.3gp | ||
| Audio-Video-Sync-Test-Calibration-23.98fps-24fps | https://www.youtube.com/watch?v=cGgf_dbDMsw | |||
| sound_in_sync_test | https://www.youtube.com/watch?v=O-zIZkhXNLE | |||
| title | url1 | url2 | ||
Since 2.3.0 this interface uses Uri instead of URI.
-
Nested Class Summary
Nested ClassesModifier and TypeInterfaceDescriptionstatic interfacestatic enumSee Lifecycle.static classA StreamException encapsulates a caught exception in the decoder thread, a.k.a StreamWorker, see See StreamWorker Error Handling.Nested classes/interfaces inherited from interface com.jogamp.opengl.util.texture.TextureSequence
TextureSequence.TexSeqEventListener<T extends TextureSequence>, TextureSequence.TextureFrame -
Field Summary
FieldsModifier and TypeFieldDescriptionstatic final com.jogamp.common.net.Uri.EncodedUri schemename for camera input.static final StringCamera property "height".static final StringCamera property "rate".static final StringCamera property "size", size as string, e.g.static final StringCamera property "width".static final booleanstatic final booleanstatic final intMaximum video frame async of 22 milliseconds.static final intConstant -1 for auto or unspecified.static final intConstant -2 for mute or not available.static final intDefault texture count, value 4.static final intMinimum texture count, value 1.Fields inherited from interface com.jogamp.opengl.util.texture.TextureSequence
sampler2D, samplerExternalOES -
Method Summary
Modifier and TypeMethodDescriptionvoidAdds aGLMediaPlayer.GLMediaEventListenerto this player.attachObject(String name, Object obj) Attaches the user object for the given name.Releases the GL, stream and other resources, includingattached user objects.detachObject(String name) Detaches the user object for the given name.intgetAID()Return the audio stream id, see audio and video Stream IDs.getAttachedObject(String name) Returns the attached user object for the given name.intWarning: Optional information, may not be supported by implementation.Warning: Optional information, may not be supported by implementation.intWarning: Optional information, may not be supported by implementation.intIf implementation uses aAudioSink, it's instance will be returned.floatReturns the audio volume.intintReturn allGLMediaPlayer.GLMediaEventListenerof this player.floatWarning: Optional information, may not be supported by implementation.intReturns the height of the video.Returns the last updated texture.getNextTexture(GL gl) Returns the next texture to be rendered.Returns a string represantation of this player's performance values.floatReturns the playback speed.intgetState()See Lifecycle.longWarning: Optional information, may not be supported by implementation.Returns theGLMediaPlayer.StreamExceptioncaught in the decoder thread, ornullif none occured.intcom.jogamp.common.net.UrigetUri()Return the stream location, as set byinitStream(Uri, int, int, int).intgetVID()Return the video stream id, see audio and video Stream IDs.intWarning: Optional information, may not be supported by implementation.Warning: Optional information, may not be supported by implementation.intWarning: Optional information, may not be supported by implementation.intintgetWidth()Returns the width of the video.voidInitializes OpenGL related resources.voidinitStream(com.jogamp.common.net.Uri streamLoc, int vid, int aid, int textureCount) Issues asynchronous stream initialization.booleanReturnstrueif the video frame is oriented in OpenGL's coordinate system, origin at bottom left.pause(boolean flush) Pauses the StreamWorker decoding thread.play()Starts or resumes the StreamWorker decoding thread.voidRemoves aGLMediaPlayer.GLMediaEventListenerto this player.intseek(int msec) Seeks to the new absolute position.booleansetAudioVolume(float v) Sets the audio volume, [0f..1f].booleansetPlaySpeed(float rate) Sets the playback speed.voidsetTextureMinMagFilter(int[] minMagFilter) Sets the texture min-mag filter, defaults toGL.GL_NEAREST.voidsetTextureUnit(int u) Sets the texture unit.voidsetTextureWrapST(int[] wrapST) Sets the texture min-mag filter, defaults toGL.GL_CLAMP_TO_EDGE.toString()Returns a string represantation of this player, incl.Methods inherited from interface com.jogamp.opengl.util.texture.TextureSequence
getRequiredExtensionsShaderStub, getTextureFragmentShaderHashCode, getTextureLookupFragmentShaderImpl, getTextureLookupFunctionName, getTextureMinMagFilter, getTextureSampler2DType, getTextureTarget, getTextureUnit, getTextureWrapST, isTextureAvailable
-
Field Details
-
DEBUG
static final boolean DEBUG -
DEBUG_NATIVE
static final boolean DEBUG_NATIVE -
TEXTURE_COUNT_DEFAULT
static final int TEXTURE_COUNT_DEFAULTDefault texture count, value 4.- See Also:
-
TEXTURE_COUNT_MIN
static final int TEXTURE_COUNT_MINMinimum texture count, value 1. Using the minimum texture count disables multi-threaded decoding.- See Also:
-
STREAM_ID_NONE
static final int STREAM_ID_NONEConstant -2 for mute or not available. See Audio and video Stream IDs.- See Also:
-
STREAM_ID_AUTO
static final int STREAM_ID_AUTOConstant -1 for auto or unspecified. See Audio and video Stream IDs.- See Also:
-
CameraInputScheme
static final com.jogamp.common.net.Uri.Encoded CameraInputSchemeUri schemename for camera input. E.g.camera:/0for the 1st camera device.The
Uri pathis being used to identify the camera (<id>), where the root fwd-slash is being cut-off.The <id> is usually an integer value indexing the camera ranging from [0..max-number].
The <somewhere> is usually empty, since it would imply a networking camera protocol.
The
Uri queryis used to pass options to the camera using ; as the separator. The latter avoids trouble w/ escaping.camera:/<id> camera:/<id>?width=640;height=480;rate=15 camera:/<id>?size=640x480;rate=15 camera://<somewhere>/<id> camera://<somewhere>/<id>?width=640;height=480;rate=15 camera://<somewhere>/<id>?size=640x480;rate=15 camera:///<id>?width=640;height=480;rate=15 camera:///<id>?size=640x480;rate=15Uri: [scheme:][//authority][path][?query][#fragment] w/ authority: [user-info@]host[:port] Note: 'path' starts w/ fwd slash
-
CameraPropSizeS
Camera property "size", size as string, e.g.1280x720,hd720. May not be supported on all platforms. SeeCameraInputScheme.- See Also:
-
CameraPropWidth
Camera property "width". SeeCameraInputScheme.- See Also:
-
CameraPropHeight
Camera property "height". SeeCameraInputScheme.- See Also:
-
CameraPropRate
Camera property "rate". SeeCameraInputScheme.- See Also:
-
MAXIMUM_VIDEO_ASYNC
static final int MAXIMUM_VIDEO_ASYNCMaximum video frame async of 22 milliseconds.- See Also:
-
-
Method Details
-
getTextureCount
int getTextureCount() -
setTextureUnit
void setTextureUnit(int u) Sets the texture unit. Defaults to 0. -
setTextureMinMagFilter
void setTextureMinMagFilter(int[] minMagFilter) Sets the texture min-mag filter, defaults toGL.GL_NEAREST. -
setTextureWrapST
void setTextureWrapST(int[] wrapST) Sets the texture min-mag filter, defaults toGL.GL_CLAMP_TO_EDGE. -
initStream
void initStream(com.jogamp.common.net.Uri streamLoc, int vid, int aid, int textureCount) throws IllegalStateException, IllegalArgumentException Issues asynchronous stream initialization.Lifecycle:
GLMediaPlayer.State.Uninitialized->GLMediaPlayer.State.Initialized1 orGLMediaPlayer.State.UninitializedGLMediaPlayer.State.Initializedis reached asynchronous, i.e. user gets notified viaattributesChanges(..).A possible caught asynchronous
GLMediaPlayer.StreamExceptionwhile initializing the stream off-thread will be thrown atinitGL(GL).Muted audio can be achieved by passing
STREAM_ID_NONEtoaid.Muted video can be achieved by passing
STREAM_ID_NONEtovid, in which casetextureCountis ignored as well as the passed GL object of the subsequentinitGL(GL)call.- Parameters:
streamLoc- the stream locationvid- video stream id, see audio and video Stream IDsaid- video stream id, see audio and video Stream IDstextureCount- desired number of buffered textures to be decoded off-thread, will be validated by implementation. The minimum value isTEXTURE_COUNT_DEFAULT. Ignored if video is muted.- Throws:
IllegalStateException- if not invoked inGLMediaPlayer.State.UninitializedIllegalArgumentException- if arguments are invalid- Since:
- 2.3.0
-
getStreamException
GLMediaPlayer.StreamException getStreamException()Returns theGLMediaPlayer.StreamExceptioncaught in the decoder thread, ornullif none occured.Method clears the cached
GLMediaPlayer.StreamException, hence an immediate subsequent call will returnnull. -
initGL
Initializes OpenGL related resources.Lifecycle:
ArgumentGLMediaPlayer.State.Initialized->GLMediaPlayer.State.PausedorGLMediaPlayer.State.Initializedglis ignored if video is muted, seeinitStream(Uri, int, int, int).- Parameters:
gl- current GL object. Maybenull, for audio only.- Throws:
IllegalStateException- if not invoked inGLMediaPlayer.State.Initialized.GLMediaPlayer.StreamException- forwarded from the off-thread stream initializationGLException- in case of difficulties to initialize the GL resources
-
getAudioSink
AudioSink getAudioSink()If implementation uses aAudioSink, it's instance will be returned.The
AudioSinkinstance is available afterinitStream(Uri, int, int, int), if used by implementation. -
destroy
Releases the GL, stream and other resources, includingattached user objects. -
setPlaySpeed
boolean setPlaySpeed(float rate) Sets the playback speed.To simplify test, play speed is normalized, i.e.
1.0f: ifMath.abs(1.0f - rate) < 0.01f
- Returns:
- true if successful, otherwise false, i.e. due to unsupported value range of implementation.
-
getPlaySpeed
float getPlaySpeed()Returns the playback speed. -
setAudioVolume
boolean setAudioVolume(float v) Sets the audio volume, [0f..1f].To simplify test, volume is normalized, i.e.
0.0f: ifMath.abs(v) < 0.01f1.0f: ifMath.abs(1.0f - v) < 0.01f
- Returns:
- true if successful, otherwise false, i.e. due to unsupported value range of implementation.
-
getAudioVolume
float getAudioVolume()Returns the audio volume. -
play
GLMediaPlayer.State play()Starts or resumes the StreamWorker decoding thread.Lifecycle:
GLMediaPlayer.State.Paused->GLMediaPlayer.State.Playing -
pause
Pauses the StreamWorker decoding thread.Lifecycle:
GLMediaPlayer.State.Playing->GLMediaPlayer.State.PausedIf a new frame is desired after the next
play()call, e.g. to make a snapshot of a camera input stream,flushshall be set totrue.- Parameters:
flush- iftrueflushes the video and audio buffers, otherwise keep them intact.
-
seek
int seek(int msec) Seeks to the new absolute position. The StreamWorker decoding thread is paused while doing so and the A/V buffers are flushed.Allowed in state
GLMediaPlayer.State.PlayingandGLMediaPlayer.State.Paused, otherwise ignored, see Lifecycle.- Parameters:
msec- absolute desired time position in milliseconds- Returns:
- time current position in milliseconds, after seeking to the desired position
-
getState
GLMediaPlayer.State getState()See Lifecycle.- Returns:
- the current state, either
GLMediaPlayer.State.Uninitialized,GLMediaPlayer.State.Initialized,GLMediaPlayer.State.PlayingorGLMediaPlayer.State.Paused
-
getVID
int getVID()Return the video stream id, see audio and video Stream IDs. -
getAID
int getAID()Return the audio stream id, see audio and video Stream IDs. -
getDecodedFrameCount
int getDecodedFrameCount()- Returns:
- the current decoded frame count since
play()andseek(int)as increased bygetNextTexture(GL)or the decoding thread.
-
getPresentedFrameCount
int getPresentedFrameCount()- Returns:
- the current presented frame count since
play()andseek(int)as increased bygetNextTexture(GL)for new frames.
-
getVideoPTS
int getVideoPTS()- Returns:
- current video presentation timestamp (PTS) in milliseconds of
getLastTexture()
-
getAudioPTS
int getAudioPTS()- Returns:
- current audio presentation timestamp (PTS) in milliseconds.
-
getLastTexture
Returns the last updated texture.In case the instance is just initialized, it shall return a
Not blocking.TextureFrameobject with valid attributes. The texture content may be undefined until the first call ofTextureSequence.getNextTexture(GL).
- Specified by:
getLastTexturein interfaceTextureSequence- Throws:
IllegalStateException- if not invoked inGLMediaPlayer.State.PausedorGLMediaPlayer.State.Playing
-
getNextTexture
Returns the next texture to be rendered.Implementation shall return the next frame if available, may block if a next frame may arrive soon. Otherwise implementation shall return the last frame.
Shall return
nullin case no next or last frame is available.In case the current state is not
GLMediaPlayer.State.Playing,getLastTexture()is returned.- Specified by:
getNextTexturein interfaceTextureSequence- Throws:
IllegalStateException- if not invoked inGLMediaPlayer.State.PausedorGLMediaPlayer.State.Playing- See Also:
-
addEventListener(GLMediaEventListener)GLMediaEventListener#newFrameAvailable(GLMediaPlayer, TextureFrame, long)
-
getUri
com.jogamp.common.net.Uri getUri()Return the stream location, as set byinitStream(Uri, int, int, int).- Since:
- 2.3.0
-
getVideoCodec
String getVideoCodec()Warning: Optional information, may not be supported by implementation.- Returns:
- the code of the video stream, if available
-
getAudioCodec
String getAudioCodec()Warning: Optional information, may not be supported by implementation.- Returns:
- the code of the audio stream, if available
-
getVideoFrames
int getVideoFrames()Warning: Optional information, may not be supported by implementation.- Returns:
- the total number of video frames
-
getAudioFrames
int getAudioFrames()Warning: Optional information, may not be supported by implementation.- Returns:
- the total number of audio frames
-
getDuration
int getDuration()- Returns:
- total duration of stream in msec.
-
getStreamBitrate
long getStreamBitrate()Warning: Optional information, may not be supported by implementation.- Returns:
- the overall bitrate of the stream.
-
getVideoBitrate
int getVideoBitrate()Warning: Optional information, may not be supported by implementation.- Returns:
- video bitrate
-
getAudioBitrate
int getAudioBitrate()Warning: Optional information, may not be supported by implementation.- Returns:
- the audio bitrate
-
getFramerate
float getFramerate()Warning: Optional information, may not be supported by implementation.- Returns:
- the framerate of the video
-
isGLOriented
boolean isGLOriented()Returnstrueif the video frame is oriented in OpenGL's coordinate system, origin at bottom left.Otherwise returns
false, i.e. video frame is oriented origin at top left.falseis the default assumption for videos, but user shall not rely on.falseGL orientation leads toTexture.getMustFlipVertically()==true, as reflected by allTextureSequence.TextureFrame'sTextures retrieved viagetLastTexture()orgetNextTexture(GL). -
getWidth
int getWidth()Returns the width of the video. -
getHeight
int getHeight()Returns the height of the video. -
toString
String toString()Returns a string represantation of this player, incl. state and audio/video details. -
getPerfString
String getPerfString()Returns a string represantation of this player's performance values. -
addEventListener
Adds aGLMediaPlayer.GLMediaEventListenerto this player. -
removeEventListener
Removes aGLMediaPlayer.GLMediaEventListenerto this player. -
getEventListeners
GLMediaPlayer.GLMediaEventListener[] getEventListeners()Return allGLMediaPlayer.GLMediaEventListenerof this player. -
getAttachedObject
Returns the attached user object for the given name. -
attachObject
Attaches the user object for the given name. Returns the previously set object, may be null. -
detachObject
Detaches the user object for the given name. Returns the previously set object, may be null.
-