How to display animated GIFs on Roku using SceneGraph

May 2, 2019

App Development Blog Roku Roku Development Tech Technology

For a while now, YouTube has had GIF previews for video thumbnails with the intent to improve the user experience. Users can easily see at a glance a short clip of the video without having to play the actual video, saving the user a few clicks.

Netflix also makes use of animated GIFs, albeit not as widespread as YouTube. They seem to use GIFs solely as a promotional tool, just to capture the user’s attention for specific content.

GIFs have been around forever, but lately they have been taking over OTT apps as a great way to engage with users.

Big players like Netflix and Youtube have access to Roku’s NDK, which gives them way more freedom and control over the hardware. Most Roku developers have to work with BrightScript and SceneGraph, which are very underpowered tools, specially when it comes to tackling big UI challenges.

Helpful tip:
If you’re curious to know which apps use Roku’s NDK, you can use the External Control Protocol and fetch all the installed apps on your device. It will return a xml with this structure:

<apps>
	<app id="31012" type="menu" version="1.9.28">FandangoNOW Movies & TV</app>
	<app id="12" subtype="ndka" type="appl" version="4.2.81179030">Netflix</app>
	<app id="13" subtype="ndka" type="appl" version="10.8.2019032615">Prime Video</app>
	<app id="837" subtype="ndka" type="appl" version="1.0.80000284">YouTube</app>
	<app id="46041" subtype="ndka" type="appl" version="5.13.154">Sling TV</app>
	<app id="61322" subtype="rsga" type="appl" version="4.6.227">HBO NOW</app>
	<app id="245916" subtype="rsga" type="appl" version="1.2.6">Pop Now</app>
	<app id="73386" subtype="rsga" type="appl" version="2.11.76">MTV</app>
</apps>

Apps that use the NDK will have the tag: subtype="ndka”.

The SceneGraph solution

If you look at the Poster node documentation, it states that “The Poster node class supports JPEG and PNG image files.“. That’s not a 100% accurate, because the Poster node also supports GIF images, but, it cannot animate those GIF images. So for the case of animated GIFs the Poster node will only render the first frame of that GIF.

The fact that SceneGraph does support the GIF image file format is a big step towards our goal and makes evident the solution to displaying animated GIFs.

Extract the frames of the animated GIF and store them as individual GIF files.
Create a frame-by-frame animator using a Timer node.
Display each extracted frame on a Poster node using a frame-by-frame animator.

Extracting the frames of a GIF image using plain BrightScript + SceneGraph is a bit of a challenge since Roku does not provide any image manipulation APIs with their current SDK, so the only real way to approach this problem is by going straight to the file and process its bytes and bits following the GIF file spec. The main component for this job is roByteArray.

Let’s get to the good part — the code 😎!

Prerequisites

Before we continue, there are a few things that we need to cover to be able to run and test the demo app.

Roku device (a Roku 4 or above is recommended).
Developer mode enabled on the Roku device.
Knowledge on how to side-load a Roku application. There are many ways of doing it, you could use the browser or an Atom plugin or a Sublime plugin or your own solution, it doesn’t matter.
Knowledge on how to access the BrightScript console.

Starter project

Download the starter project and run it. You should see a simple interactive grid with Poster nodes in it:

The Poster nodes are displaying only the first frame of all the GIFs available in the bundle, we are going to fix that and make sure those GIFs animate.
The GIFs are Frozen-themed so I’m sorry if you let it go already 😉.

Decoding the GIF files

As mentioned earlier, we have to extract the frames of the GIF file so that we can display each frame sequentially using our frame-by-frame animator and give the sense of motion.

GIF files are composed of 11 different types of blocks of data:

There is a lot of information out there that covers the specifications of each block type in great detail (references can be found below), so I’ll just do a quick run through of each block type, explaining how we’ll use them, covering only the essentials.

Header: Marks the beginning of the file, is always 6 bytes in length and is composed by the Signature (always “GIF”) and Version (“89a” or “87a”). Our implementation will always ensure that the header is “GIF89a”.
Logical Screen Descriptor: Defines the whole GIF displaying boundaries. Always 7 bytes in length, we only care about the 5th byte which is a packed field that helps us determine if the GIF has a global color table and the size of such table.
Global Color Table: Sequence of bytes representing red-green-blue color triplets. If present, we only need it to prepend it to the extracted frames that do not have a local color table.
Graphic Control Extension: Contains parameters used when processing the image data that follows, particularly the delay time, which we will read to calculate the average delay time between all frames. Has a fixed length of 8 bytes.
Image Descriptor: Defines the image boundaries and helps us determine if the image has a local color table and the size of such table. Is always 10 bytes in length and the table availability info is stored in 10th byte.
Local Color Table: Same structure as the global color table. If present, we just need to keep it attached to the image data.
Image Data: LZW encoded sequence of sub-blocks, of size at most 255 bytes each, containing an index into the local or global color table, for each pixel in the image.
Application Control Extension: Always 19 bytes in length. We don’t need this block in our extracted frames, we just need to skip through it.
Comment Extension: We don’t need this block in our extracted frames, we just need to determine where it starts and ends and skip through it.
Plain Text Extension: We don’t need this block in our extracted frames, we just need to determine where it starts and ends and skip through it.
Trailer: Marks the ending of the file. Is always 1 byte with the hex value: 3B.

Based on those block type descriptions here is the complete implementation of the GIF decoder component that we’ll use.

<component name="GIFDecoder" extends="Task">
  <interface>
    <!-- Callback listener -->
    <field id="delegate" type="node"/>

    <!-- Extracts the frames of the given GIF and stores them locally as individual GIF files.
      - Parameter {String} gifUrl: The url of the GIF to decode. -->
    <function name="decodeGIF"/>
  </interface>

  <script type="text/brightscript" uri="pkg:/components/GIFDecoder.brs"/>
</component>

sub decodeGIF(gifPath as String)
  m.top.functionName = "runDecoder"

  m.gifPath = gifPath
  m.gifName = CreateObject("roPath", m.gifPath).split().basename
  m.top.control = "RUN"
end sub

sub runDecoder()
  ' Load the main GIF into memory.
  ' @NOTE: For big GIFs, the bytes of the GIF should be read as needed using:
  ' `ReadFile(path as String, start_pos as Integer, length as Integer) As Boolean`.
  gifBytes = CreateObject("roByteArray")
  gifBytes.readFile(m.gifPath)

  ' Read the header block bytes. The header block always has a length of 6 bytes.
  ' Ensure the file is a supported gif file.
  headerBytes = subByteArrayFrom(gifBytes, 0, 6)
  header = headerBytes.toAsciiString()
  if header <> "GIF89a"
    ' Handle unsupported file errors.
    return
  end if

  ' Get the logical screen descriptor.
  logicalScreenDescriptorBytes = subByteArrayFrom(gifBytes, 6, 7)

  ' Get the global color table bytes (if available)
  globalColorTableBytes = invalid
  globalColorTableSize = colorTableSize(gifBytes, 6, true)
  if globalColorTableSize
    ' The global color table follows the logical screen descriptor
    globalColorTableBytes = subByteArrayFrom(gifBytes, 13, globalColorTableSize)
  end if

  ' Concatenate common bytes
  gifFrameCommonBytes = createObject("roByteArray")
  gifFrameCommonBytes.append(headerBytes)
  gifFrameCommonBytes.append(logicalScreenDescriptorBytes)

  ' The trailer byte that will be always appended to each individual GIF.
  trailerByte = createObject("roByteArray")
  trailerByte.FromHexString("3b")

  ' Capture all the frames in the GIF and store them as individual GIFs.
  frames = []
  frameNumber = 0
  totalDuration = 0.0
  byteIndex = 13 + globalColorTableSize
  while byteIndex < gifBytes.count()
    increment = 1

    hexVal = StrI(gifBytes[byteIndex], 16)
    if hexVal = "21" ' Extension block introducer
      extensionLabel = StrI(gifBytes[byteIndex + 1], 16)
      if extensionLabel = "ff" ' Application extension (will be ignored)
        ' Skip the application extension block.
        ' The application extension block has a fixed size of 19
        increment = 19
      else if extensionLabel = "fe" ' Comment extension (will be ignored)
        ' Skip the comment extension block.
        ' The comment extension block ends when a zero-value byte is found.
        commentExtensionLastByteIndex = byteIndex + 2
        while (gifBytes[commentExtensionLastByteIndex] > 0)
          commentExtensionLastByteIndex+= gifBytes[commentExtensionLastByteIndex] + 1
        end while
        increment = commentExtensionLastByteIndex - byteIndex + 1
      else if extensionLabel = "01" ' Plain text extension (will be ignored)
        ' @TODO: skip all the plain text extension bytes
        exit while
      else if extensionLabel = "f9" ' Graphic control extension
        ' The fith byte in the graphic control extension block has the delay time of the next frame.
        ' This value is represented as hundredths (1/100) of a second.
        ' @NOTE: The gif spec refers to the fith and the sixth byte but in all the examples only the fith
        ' value is taken into account, not the sixth.
        delayTime = gifBytes[byteIndex + 4] / 100.0
        totalDuration+= delayTime

        ' The graphic control extension block has a fixed size of 8
        increment = 8
      else
        ' Handle invalid extension label error.
        exit while
      end if
    else if hexVal = "2c" ' Image descriptor block
      ' Get the local color table size so that we can know where the image data starts
      localColorTableInfoByteIndex = byteIndex + 9 ' The packed field with the local table info is always in the 10th byte.
      localColorTableSize = colorTableSize(gifBytes, byteIndex)

      ' Determine the image descriptor + the image data size
      imageDataByteStartIndex = localColorTableInfoByteIndex + localColorTableSize + 1
      imageDataByteEndIndex = imageDataByteStartIndex + 1
      while (gifBytes[imageDataByteEndIndex] > 0)
        imageDataByteEndIndex+= gifBytes[imageDataByteEndIndex] + 1
      end while
      imageDescriptorAndDataSize = imageDataByteEndIndex - byteIndex + 1

      ' Create the new gif file for this frame with the common bytes
      gifFramePath = "tmp:/" + m.gifName + "_" + frameNumber.toStr() + ".gif"
      gifFrameCommonBytes.writeFile(gifFramePath)

      ' Append the global color table only if there's no local color table for this frame
      if localColorTableSize = 0 and globalColorTableBytes <> invalid
        globalColorTableBytes.appendFile(gifFramePath, 0, globalColorTableSize)
      end if

      ' Append the image data of this frame
      gifBytes.appendFile(gifFramePath, byteIndex, imageDescriptorAndDataSize)
      trailerByte.appendFile(gifFramePath, 0, 1)

      ' Save new gif url
      frames.push(gifFramePath)
      frameNumber++

      ' Go to the next block of data in the next interation
      increment = imageDataByteEndIndex - byteIndex + 1
    else if hexVal = "3b" ' Trailer (should be the last byte in a gif file)
      exit while
    end if

    byteIndex+= increment
  end while

  ' Notify delegate
  fps = totalDuration / frames.count()
  m.top.delegate.callFunc("gifDecoderDidFinish", frames, fps)
end sub

' Locates the color table based on the descriptor information and returns its size (if a color table is present).
function colorTableSize(gifBytes as Object, descriptorLocation as Integer, global = false as Boolean) as Integer
  size = 0

  ' For the case of image descriptors the packed field with the table info is always in the 10th byte
  ' and for the case of the logical screen descriptor is always in the 5th byte.
  packedFieldLocation = descriptorLocation + 9
  if global packedFieldLocation = descriptorLocation + 4

  ' The color table information is meant to be interpret in is binary (8-bit) representation.
  colorTableInfoBits = decimalTo8Bit(gifBytes[packedFieldLocation])

  ' Check if there is a color table by checking the first bit of the color table info bits.
  if colorTableInfoBits.left(1) = "1"
    ' The bits 1...3 represent the number of bits used for each color table entry minus one.
    bitsPerEntry = Val(colorTableInfoBits.right(3), 2) + 1

    ' The number of colors in the table can be calculated as: `2^(bitsPerEntry)`.
    ' Which means that the size of the table would be 3*2^(bitsPerEntry).
    size = 3 * pow(2, bitsPerEntry)
  end if

  return size
end function

' Returns a slice of the given array
function subByteArrayFrom(byteArray as Object, location as Integer, length as Integer) as Object
  newArray = CreateObject("roByteArray")
  for i = location to location + length - 1
    newArray.push(byteArray[i])
  end for
  return newArray
end function

' Converts the given decimal to its 8-bit representation
function decimalTo8Bit(decimal as Integer) as String
  return ("0000000" + StrI(decimal, 2)).right(8)
end function

' Returns a number raised to a given power.
function pow(x as Float, y as Integer) as Float
  if y = 0 then return 1

  temp = pow(x, y/2)
  if y mod 2 = 0 then return temp * temp
  if y > 0 then return x * temp * temp

  return (temp * temp) / x
end function

Please add those files to your components folder and go over all the comments in the BrightScript file for better understanding of the implementation.

Our new GIFDecoder follows a delegate pattern that we’ll use to communicate with the scene, so let’s set it up. Go to the AppScene’s init()method and create a GIFDecoder instance at the beginning of the function.

sub init()
  m.decoder = createObject("roSGNode", "GIFDecoder")
  m.decoder.delegate = m.top
  (...)
end sub

Now let’s declare and implement the GIFDecoder callback function which will be executed after the decoder finishes. To declare it, we just need to add it to the XML interface.

<component name="AppScene" extends="Scene">
  <interface>
    <!-- GIFDecoder callback -->
    <function name="gifDecoderDidFinish"/>
  </interface>

  <script type="text/brightscript" uri="pkg:/components/AppScene.brs"/>
</component>

Moving on to the callback implementation, we’re just going to print out the new frame urls for now.

sub gifDecoderDidFinish(frames as Object, fps as Float)
  ?"frames "frames
end sub

And finally, we have to actually start the decoder for the focused poster.

sub focusItem(item as Integer)
  (...)
  ' Start decoder
  m.decoder.callFunc("decodeGIF", getGIFUrl(item))
end sub

If you run the app, this time you should see all the urls for the extracted frames appear on the BrightScript console as you move through the posters.

Animation time! 🎸

We’ve reached the final and easier step of our solution, the animation 🙂. For that, we’ll simply use a Timer node to display each frame sequentially but we’ll wrap it within our custom frame-by-frame animator component to keep track of each frame more easily:

<component name="FrameAnimator" extends="Node">
  <interface>
    <function name="start"/>
    <function name="finish"/>
  </interface>

  <script type="text/brightscript" uri="pkg:/components/FrameAnimator.brs"/>
</component>


sub init()
  m.animator = CreateObject("roSGNode", "Timer")
  m.animator.ObserveField("fire", "displayNextFrame")
  m.animator.repeat = true

  m.frames = []
  m.frameIndex = -1
  m.poster = invalid
end sub

function start(frames as Object, fps as Float, poster as Object)
  m.frames = frames
  m.poster = poster

  m.animator.duration = fps
  m.animator.control = "start"
end function

function finish()
  m.animator.control = "stop"

  ' Restore first frame
  if m.frames.count() > 0 m.poster.uri = m.frames[0]

  m.frameIndex = -1
  m.frames = []
  m.poster = invalid
end function

sub displayNextFrame()
  m.frameIndex++
  if m.frameIndex >= m.frames.count()
    m.frameIndex = 0
  end if

  m.poster.uri = m.frames[m.frameIndex]
end sub

Please add FrameAnimator.xml and FrameAnimator.brs to your components folder.

Lastly, we just need to plug in the new FrameAnimator component on our scene.

Instantiate the FrameAnimator node.

sub init()
  m.animator = createObject("roSGNode", "FrameAnimator")
  (...)
end sub

2. Stop the animator when navigating to a new poster.

sub focusItem(item as Integer)
  (...)
  ' Stop previous poster animation
  m.animator.callFunc("finish")
end sub

3. Start the animation after the decoder finishes.

sub gifDecoderDidFinish(frames as Object, fps as Float)
  m.animator.callFunc("start", frames, fps, m.posterGrid.getChild(m.focusedItem))
end sub

Side-load the app one more time and navigate through the posters, the focused poster should display the animated GIF 🤞.

If you are seeing the focused poster animate, CONGRATS!, now you can have animated GIFs on your SceneGraph apps 🍾.

The complete project can be found here.

Missing features

Inter-frame coalescing: Some GIFs may have frames with empty or transparent areas that are meant to be rendered on top of the previous frame(s). Our current implementation displays each frame individually for performance reasons, GIFs that require inter-frame coalescing will not animate correctly.
Variable frame rate: Some GIFs may have different delay times between frames, our implementation ignores this and uses the average delay time as the frame rate for all frames.

Conclusion

GIF thumbnails can be a great tool for improving the user experience in your app. They can be used to promote content or to offer quick content previews. But, as we all know, Roku devices are not very powerful (and they are not trying to be), specifically with the tools available to most developers. A feature like this is obviously very expensive, so please use it responsibly 🙂.