January 23 2014 - Video playback using the Wandboard VPU, part 3

This article conlcudes the article series on gstreamer by sketching on how gstreamer acceleration is implemented on Freescale iMX6.

Gstreamer hardware acceleration on iMX6

The typical gstreamer usage on iMX6 is to use the VPU for video decoding. This is usually done using the vpudec element, and then using a Video For Linux sink (mfw_v4lsink) to display the video in a X11 window. The v4lsink has a problem though: it only supports one movie at a time. The recommended solution from Freescale is to use mfw_isink when playing back several videos -- but that does not integrate well with X11 (in fact, it changes the framebuffer to YUV mode).

Further the X11 integration is not made easier from xvimagesink being a fake in the Freescale Xorg driver (it announces XV support, but when asked to use it there are no adapters!)

Gstreamer and OpenGL

For X11 integration, glimagesink seems to be the most general.

It uses the OpenGL framework to display and resize the video. It cooperates with X11. It rescales without problems or use of additional elements (like videoscale).

While the default glimagesink does not allow placement of the movie window, or playback to a preexisting XWindow. However gstreamer allows a facility called GstXOverlay for this. It is a abstract base class (or as close you get in C) for X11 output. Using this, the handy hacker should be able to get a OpenGL and VPU accelerated X video player up and running by implementing their own imagesink (based on the code in glimagesink, and adding arguments for position and window id)


Demo image

For those who are curious about multiple simultaneous videos, there is a simple demo image here that plays for videos continously.


Talk back and discuss in the Wandboard community forums


January 20 2014 - Video playback using the Wandboard VPU, part 2


This post is the second article of three about hardware accelerated video playback on the Wandboard. Todays topic is gstreamer basics and how gstreamer framework can be used from the linux command line.

Gstreamer overview

The gstreamer framework is typically integrated in a movie player, and quite unnoticable for the user. Here, we will instead dive into the command line interface to gstreamer.

The main command is called gst-launch.

However that is not enough as such. One needs to give it arguments with:
- where to play from
- how to decode video / audio
- how to 'display' the result

The word display is in quotes, since gstreamer can read and send playback from several different sources. Here we will assume a video file is to be displayed on the screen.

An example command is:

% gst-launch filesrc location=movie.mp4 typefind=true \
 ! aiurdemux \
 ! vpudec \
 ! mfw_v4lsink

The first argument to gst-launch is typically the video source. Typically the source is a media file, but it can be a network socket or just about anything. It is from where the videostream (encoded) is to be read. In the example command, the video is to be read from the file "movie.mp4".

The sink is where to dump the decoded video, typically the screen, but can also be just about anything. Here the sink is mfw_v4lsink, which is to display it using the iMX6 Video4Linux output.

In between the source and the sink, a number of elements can be specified, acting as filters, converters etc. In the example command above, one important element is the vpudec one -- that is the VPU video decoder element. More about that in part 3.

Colourspaces

One issue one might encounter is incompatible colourspaces. The vpudec element outputs YUV format, while the framebuffer is in RGBA format. While imagesinks like ximagesink is able to convert YUV to RGBA, the conversion is very slow. The iMX6 has hardware support for colourspace conversion, and one element for colourspace conversion is the mfw_ipucsc element: gst-launch filesrc location=movie.mp4 typefind=true ! aiurdemux ! vpudec frame-plus=1 ! mfw_ipucsc ! videoscale ! ximagesink

To find out the input and output formats of elements, one can use the 'gst-inspect' command, i.e: gst-inspect vpudec


Talk back and discuss in the Wandboard community forums

January 17 2014 - Video playback using the Wandboard VPU, part 1


This is the first part of three in a series of articles about hardware accelerated video playback.
The very first part is about a common VPU driver issue on iMX6. The second and third part are about gstreamer, in general and on iMX6.

Physical memory allocation failure

Some of you will shiver seeing those words. It is a kernel error message haunting Linux video playback on the iMX6.

This blogpost discusses the reasons for this error, and proposes a workaround.

Why it happens

Primarily, the physical memory allocation failure is caused by a failure to allocate DMA memory inside the VPU driver. Contrary what some might believe, it is not because insufficient memory available or memory leals. It is due something far more sinister: DMA memory pool fragmentation.

Using the VPU for video decoding, the VPU driver will request and free blocks of varying sizes (range ~80k - ~5M), and eventually the fragmentation of the free memory will be so bad that a large continous block can no longer be allocated. Bam!

Fragmentation is a common problem when dealing with memory allocators. Many methods have been tried and tested, but thre is no silver bullet.

The current strategy employed by the kernel is a straightforward "first fit" allocator (deep inside dma_alloc_coherent()). That is, when a say 300kB block is requested, the first free block of sufficient size is used. Be it a 300kB block or a 5MB block. This way, eventually all continous, say 5MB, blocks are partially in use by far smaller blocks.

Workaround idea

While a smarter algorithm for allocating dma memory probably could improve on the situation, it would be a non-trivial task to test and implement other allocators.

It happens that the likelyhood of badly fragmented memory is only one consideration when implementing a memory allocator. Other considerations, like speed, has also to be taken into account. Let's just contend that memory allocators are a rich area of research, and focus on the problem at hand.

Instead of an ultimate memory allocator, the following workaround is proposed: how about not really freeing and allocating memory, and just caching the continous blocks when they were supposed to be freed? This is the age-old technique of using a memory pool allocator.

Results

We tested this inside the VPU driver, with promising results. Our test playback demo played four simultaneous video clips, and the DMA allocation error usually happened after 10-20 minutes.

After implementing the pool allocator, we were able to play the clips for days without any errors.

Patch

The patch can be found in the Wandboard git.

The drawback of the patch is that the VPU driver keeps a pool of memory blocks that are never freed. It binds DMA memory that can never be used by other drivers.

Future work

Also DMA memory is also requested by the PXP driver. In case of persistent issues, implementing a memory pool allocator might be beneficial in the PXP driver as well. Maybe even with combined memory pools with the VPU driver?


Talk back and discuss in the Wandboard community forums


January 13 2014 - Wandboard meets the WandCam


20140113-wandcam 

Avnet has created a camera board for Wandboard and they call it WandCam. WandCam uses an OV5460 MIPI image sensor module with autofocus from Leopard Imaging, so engineers can easily leverage this design for security/surveillance and robotics applications.

It is very affordable and available from the Avnet Express website at http://www.em.avnet.com/wandcam

January 8 2014 - Yocto 1.5 with Qt 5 for the WandBoard


20140108-yocto-qt5 


We recently released an updated version of Yocto 1.5 demo image for the Wandboard. This time it comes with Qt 5 support out of the box, including support for Qt Creator.

For those who are unfamiliar with newer Qt versions, Qt Creator is a C++ IDE for building Qt applications. Among others, it features graphical debugging and point and click GUI building tools.

One of the neat features of Qt Creator is its integration with the devkit. With one mouse click on your desktop PC, you can compile, deploy, and run your Qt application on the Wandboard!

Check it out from the downloads

Talk back and discuss in the Wandboard community forums


December 26 2013 - Introducing SPL u-boot for the Wandboard


20131226-SPL-boot 

As some of you may have noticed, the recent Ubuntu and Yocto demo images are no longer separate for different Wandboard variants (Solo, Dual, Quad).

This is due to the use of something called SPL bootloader. SPL allows you to auto-detect your CPU and memory before loading u-boot, the main bootloader.

So how does this work?

This is going to be a little technical about iMX6/ARM boot process.

As many of you already know, the iMX6 boot process has the following steps:

1. ROM code (hard coded inside the iMX6 CPU)
2. u-boot bootloader (in SD card)
3. Kernel (Linux, BSD, ... )
4. OS / Userland (Android, Ubuntu, FreeBSD etc)

In a traditional boot process memory is set up by ROM code. ROM reads a table of memory parameters inside u-boot, and sets up the memory.

So, for different memory configurations, different tables, and hence, different u-boot binaries are needed.

SPL is a very small bootloader able to fit into the very small amount of memory (128kB) available inside the CPU. SPL does autodetection and sets up memory accordingly. After memory is set up, SPL loads u-boot, and the booting proceeds as usual. So SPL is an additional booting step between "1" and "2" in the list above.

The big benefit of this is that we can have one SD card image for all configurations.

How do I get this? Is it open source?

Yes, of course it is open source. SPL  as a part of u-boot.
Download the latest Wandboard source code package (downloads section). Instructions are in the doc/ folder inside the source tarball.


December 10 2013 - Fedora 20 adds the Wandboard


20131210-wb-fedora20 

Today is a big day with the release of Fedora 20 where our beloved Wandboard is among the few ARM platforms that have been added by the Fedora ARM project team. Kudo's to them.

Download Fedora20 and give it a try.

http://fedoraproject.org/



November 12 2013 - Wandboard Java Hands-on

In today's tutorial we are showing how to quickly install Java onto the Wandboard and explorer the Java2Demo applications.

To get started you will need to prepare a Wandboard that boots from a SD card containing Ubuntu that can be found on the Wandboard Download Page.

Step 1: Download Ubuntu 12 for your Wandboard from the Wandboard Download Page and create a bootable SD card. (refer to the document that comes with the download).

Step 2: Boot your Wandboard. All instructions below should be performed on your Wandboard.

Step 3: Get the Oracle Java files

Open Firefox and go to Oracle JDK download site as below:

http://www.oracle.com/technetwork/java/javase/downloads/jdk7-downloads-1880260.html

20131111-wb-java-001
Step 3a: Click to accept the License agreement (after reading it).

20131111-wb-java-002
Step 3b: Select "jdk-7u45-linux-arm-vfp-sflt.tar.gz" (as highlighted).

Note : Select SOFT FLOAT ABI since Wandboard Ubuntu 12 currently does not support hard float.

The JDK will be saved in "~/Downloads" directory (default Firefox download location)

20131111-wb-java-003
Step 3c: On the same page, scroll down a bit further and select the demo images "jdk-7u45-linux-arm-vfp-sflt-demos.tar.gz"

The JDK demos will be saved in "~/Downloads" directory (default Firefox download location)

Step 4: Now that you have all files. Let's install them.

20131111-wb-java-004

Step 4a: Install JDK7 on Wandboard/Ubuntu by opening the terminal window

Change the current directory to ~/Downloads ($cd Downloads)
Unzip JDK 7 to /opt ($sudo tar zxvf jdk-7u45-linux-arm-vfp-sflt.tar.gz –C /opt)

20131111-wb-java-005
Step 4a: Install JDK demos on Wandboard/Ubuntu in a simalar manner as installing the JDK by using the terminal window

Change the current directory to ~/Downloads ($cd Downloads)
Unzip JDK 7 demos to /opt ($sudo tar zxvf jdk-7u45-linux-arm-vfp-sflt-demos.tar.gz –C /opt)

Step 5: Ensure that you installed Java

20131111-wb-java-006
Step 5a: Open terminal window.
Add JDK bin into PATH variable. (#export PATH=$PATH:/opt/jdk1.7.0_45/bin)
Test Java version. (#java –version)
(If you see the message as the shown above screen, it means Java VM has been installed successfully!)

Step 6: Java Demo Playtime

Now all is installed. Let's get the demo loaded on your desktop.

20131111-wb-java-007

To run the demo application:
($cd /opt/jdk1.7.0_45/demo/jfc/Java2D)
($java –jar Java2Demo.jar)

Within moments you will see the Java2Demo application appearing on your desktop. (as below).

20131111-wb-java-008


November 4 2013 - Wandboard camera says "Cheese"


20131104-wb-camera 

Over the past couple months we have seen multiple requests in the forums regarding MIPI camera's that can connect on the Wandboard FPC.

e-con Systems, a company making camera modules, has picked the Wandboard to show off their new e-CAM51IMX6 5 megapixel MIPI camera module based around the OmniVision OV5640 image sensor which can be simply connected to the Wandboard MIPI FPC expansion connector.

e-con Systems must have thought. We not only can make it. We can as well show it off on Youtube.    : https://www.youtube.com/watch?v=ixtSAx9h5Us
 
The camera modules can be purchased directly from e-con Systems webpage.

http://www.e-consystems.com/iMX6-wandboard-camera.asp


PICO-IMX6UL for Android Things

pico imx6ul kit android things

Wandboard Specifications

  WBSOLO WBDUAL WBQUAD
Processor NXP i.MX6 Solo NXP i.MX6 Duallite NXP i.MX6 Quad
Cores Cortex-A9 Single core Cortex-A9 Dual core Cortex-A9 Quad core
Graphic
engine
Vivante GC 880
+ Vivante GC 320
Vivante GC 880
+ Vivante GC 320
Vivante GC 2000
+ Vivante GC 355
+ Vivante GC 320
Memory 512 MB DDR3 1GB DDR3 2GB DDR3
       
Audio tag tag tag
Optical
S/P DIF
tag tag tag
HDMI tag tag tag
Camera
interface
tag tag tag
Micro-SD slot 2 2 2
       
Serial port tag tag tag
Expansion
Header
tag tag tag
USB tag tag tag
USB OTG tag tag tag
SATA NO NO tag
       
Gigabit LAN tag tag tag
WiFi (802.11n)   tag tag
Bluetooth   tag tag
 

79 USD

99 USD

129 USD