MinGPU: A minimum GPU library for Computer Vision

Introduction

In computer vision it is becoming popular to implement algorithms in whole or in part on a Graphics Processing Unit (GPU), due to the superior speed GPUs can offer compared to CPUs. We present GPU implementations of two well known computer vision algorithms – Lukas-Kanade optical flow and optimized normalized cross-correlation as well as homography transformation between two 3D views. We also present a MinGPU – a library which contains, as minimal as possible, all of the necessary functions to convert an existing CPU code to GPU. We provide timing charts and show, in particular, that while our MinGPU implementations of optical flow algorithm perform few hundreds times faster than CPU, our MinGPU implementation of homography transformations perform approximately 600 times faster than its C++ CPU implementation and more than 7,500 times faster than its MatLab CPU implementation.

Our paper makes two contributions to computer vision. First, we have implemented three computer vision methods on GPU which have not been done before. For each of these methods, we present timing and we provide a detailed timing comparison of GPU homography transformation to its non-GPU implementations. Second, we have created a small C++ class library, MinGPU. We intentionally designed the library and interfaces to be as minimal as possible. MinGPU provides simple interfaces which can be used to load a 2D array into the GPU and perform operations on it. All GPU and OpenGL related code is encapsulated in the library; therefore users of this library need not to know any details on how GPU works. Because GPU programming is currently not that simple for anyone outside the Computer Graphics community, this library can facilitate an introduction to GPU world to researchers who have never used the GPU before. The library works with both nVidia and ATI families of graphics cards and is configurable.

MinGPU Library Structure and Required Libraries

MinGPU library consists of four Visual Studio projects. The first project, MinGPU, contains the MinGPU library. Computer vision algorithms done on MinGPU reside in Vision project. There is also a helper project, Math, which contains math functions used in Vision project. Those three projects generate C++ libraries on compilation. The fourth project, GPUTest, serves as a testbed for those three libraries. It generates executable code on compilation. GPUTest contains examples for all computer vision algorithms in Vision project.
Before using MinGPU, three other libraries must be downloaded from internet and installed. These are Cg Toolkit, OpenGL Utility Toolkit (GLUT) and OpenGL Extensions (GLEW). OpenGL 2.0 drivers are supplied with Windows XP, it is important to install these also.

MinGPU Basic Example

MinGPU is a C++ class library. Because of incapsulation paradigm, its users do not need to know any details about its inner structure, so they do not need to know the details of how fragment processor or OpenGL drivers work. MinGPU contains just two classes: Array and Program. Class Array defines an array in the GPU memory, while class Program defines a Cg program in the GPU memory. All class methods are listed below. In a simple scenario, user prepares data array and uploads it to the GPU using methods of class Array. Cg program is then loaded and compiled. Optionally, program parameters are set using the method of class Program. Cg program is then run and the results are generated in GPU memory, which are then downloaded to CPU memory by another call to method of class Array.

Let’s convert this simple CPU code into GPU:

for (row = 0; row < MaxRow; row ++)
{
   for (col = 0; col < MaxCol; col ++)
   {
   Array[row][col] ++;
   }
}

MinGPU code which implements the same functionality on GPU will look as simple as that:

C++ code

Array Array;
Program Program;
Array.Create(fpArray, cols, rows, Luminance);
Array.CopyToGPU();
Program.Create(strProgramFile, "main");
Program.SetParameter(enTexture, "texture", (void *) Array.Id());
Program.Run(&Array);
Array.CopyFromGPU();

Cg program

float4 main (
   float2 coords : TEXCOORD0,
   samplerRECT texture) : COLOR
{
   float4 result;
   float4 val = texRECT(texture, coords);
   result = val + 1;
   return result;
}

MinGPU Class Structure

MinGPU class structure is intentionally made as simple and small as possible. MinGPU contains two classes, Array and Program. Class Array defines an array (a texture) in GPU memory, while class Program defines a program in GPU memory. The library also includes MinGPUInit() function which is implicitly called when first MinGPU class is instantiated.

1. MinGPU methods. Class Array

Class Array has three methods: Create, CopyToGPU, and CopyFromGPU.

bool Array::Create(float *pData, unsigned int dwCols, unsigned int dwRows, BYTE bMode)

This method defines a new array in GPU memory. Method Create is supplied with a pointer to an array as well as a number of columns and rows in this array. pData must either point to an allocated space of dwCols by dwRows size, or be null if array we define will never be copied to the GPU memory. If this array is copied from the GPU memory and this parameter is null, then array is created. The last parameter, bMode, specifies whether array shall be created in luminance or color mode. Note that array data from pData is not transferred to GPU in this method, it is done later, if needed, by use of CopyToGPU method.

bool Array::CopyToGPU()

This method copies an array data from computer memory to GPU.

bool Array::CopyFromGPU()

This method copies an array data from GPU to computer memory.

2. MinGPU methods. Class Program

Class Program has three methods: Create, SetParameter and Run.

bool Program::Create(char *szFilename, char *szEntryPoint)

This method creates and, if needed, compiles new Cg program. Program is stored in the external file szFilename. szEntryPoint holds a name of the entry function in the program. Programs files must be located either in working directory, or in its ‘scripts’ subfolder. Files with ‘asm’ extension are presumed to contain pre-compiled binary programs, otherwise file contains a source code which needs to be compiled. This function searches the folder for a binary program corresponding to the source code. If it is found and has a timestamp later than the source file, it is loaded instead. Otherwise, source code is compiled and stored as binary code in a file with ‘asm’ extension in the same folder. This eliminates the need to recompile a program each time it is loaded (eliminates 300-600ms delay).

bool Program::SetParameter(int nType, char *szName, void *pValue)

Most functions in Cg programs have some input parameters. For instance, we have to specify some input parameters for the entry function before we execute a program. Parameters can be values, arrays, matrices, or textures. Enumerator nType specifies the type and number format of input parameter, string szName contains its Cg program name and pValue holds the parameter value. Besides parameters we set with SetParameter method, functions may have some parameters which refer to pre-defined names. For example, parameter which refers to name TEXCOORD0 will automatically receive the row and column values for the currently processed pixel.

bool Program::Run(Array *output)

This method executes a Cg program on the GPU. Array output accepts the output of this program; it is filled as a result of program execution. The program is run separately for every cell in array output. A new value is generated for every cell in this array.

All methods in classes Array and Program return true if successful and false otherwise.

Sample Input Data

Here is the link to some images we used in library testing link.