At the forefront of Artificial Intelligence

Home » Articles » Robotics » LEGO Mindstorms

Accessing the LEGO Cam Video Stream for Motion Detection

This article will briefly look at how to access the video stream directly (using motion detection as an example). By doing this, you can analyze and modify the data as you see fit. This article assumes you've read Programming with the LEGO Cam since we will not cover initialization and basic camera commands.

Background

To easily access the data, you set a "hook" in the video stream, the camera ActiveX control then sends a notification message every frame. The notification message has 3 bits of information: an LPBITMAPHEADERINFO (pointer to bitmap information), an LPBYTE (pointer to the bitmap bytes) and a time-stamp.

Basically, that is all you need to analyze the data! The methods I will cover from here on are heavily based on the Vidbert example that is provided with the QuickCam SDK. This is easiest method for simple motion detection.

Setting Up

First things first: after initializing your camera controls, you should call "StartVideoHook".

	m_cVideo.StartVideoHook(0);

Since we'll be doing more than just detecting the motion (we'll also be displaying where it is occuring) we must hide the video control since we'll be displaying our frames manually. Next, we'll want to set up a bitmap that will allow us to draw our frames in the memory before blitting it to the screen (much quicker than drawing it to the screen pixel-by-pixel!). See the example code for the function (AllocateDIB).

After that, we'll want to set up the notification message. This is simple - in ClassWizard, select your video control and create a handler for the PortalNotification message. After ClassWizard has created your code, you can access the data through the lParam1-lParam3 parameters:

switch( lMsg ) {
case NOTIFICATIONMSG_VIDEOHOOK: {
    LPBITMAPINFOHEADER lpbi = 
	  (LPBITMAPINFOHEADER) lParam1;
    LPBYTE lpBytes = (LPBYTE) lParam2;
    unsigned long lTimeStamp = (unsigned long)lParam3;
    
    ShowMotion(lpbi, lpBytes);
  } break;

  default: break;
}

NOTIFICATIONMSG_VIDEOHOOK is defined as 10. We are now ready to process the data - but first a brief look at the bitmap structure.

Bitmaps

Bitmaps are strange in that their data is ordered from left to right, but bottom to top. Therefore, we will start processing the data from the bottom-left and work up to top-right. Each pixel consists of three bytes: a red, green and blue component. As we are accessing the bitmap data sequentially (as we will), you should remember that the bytes are reversed.

A Basic Motion Algorithm

The simplest motion algorithm simply involves subtracting one frame from another and seeing where the differences in colour are - those differences can be interpreted as motion. Below is the function, we'll break it up as we go along.

void CMotionCamDlg::ShowMotion(LPBITMAPINFOHEADER lpbi,
                               LPBYTE lpBits)
{
    int W = lpbi->biWidth;
    int H = lpbi->biHeight;
    int nPels = W * H;
    int nPadBytes = DibRowPadding(lpbi);
    int nRowStep = DibRowSize(m_pbi);

    BYTE *dst = m_pbits;
    BYTE *src, *prv;

Firstly, we'll want to set up some variables: the width and height of the bitmap, the number of pixels, the row padding and the row size. The DibRowPadding and DibRowSize were taken from the Vidbert example and simply aid calculations. m_pbits is a pointer to the bytes of the bitmap that we allocated on startup (our memory bitmap, remember?).

    for (int i = 0; i < nPels; i++) {
        src = lpBits  + (i % W) * 3 + int(i / W) * nRowStep;
        prv = m_pPrev + (i % W) * 3 + int(i / W) * nRowStep;

This perhaps takes a little explanation. We want to calculate a pointer to the byte group for the current pixel, therefore we add the X-coordinate multipled by 3 (3 bytes per pixel) and the Y-coordinate multiplied by the row-step (number of pixels per row) to the initial pointer. If you are still confused, try a few calculations manually to see how it works.

        if (abs(src[0] - prv[0] > 25) &&
            abs(src[1] - prv[1] > 25) &&
            abs(src[2] - prv[2] > 25)) {
                dst[0] = 0;
                dst[1] = 0;
                dst[2] = 255;
        } else {
            dst[0] = 0;
            dst[1] = 0;
            dst[2] = 0;
        }

Here is our basic motion algorithm! The src is pointer to the byte group in the current frame, prv is the equivalent for the previous frame (see below where we save the frame). The comparision method is fairly arbitrary - I chose to subtract the colour bytes from the current from from the previous frame (and take the absolute value). If there has been a significant enough change (all the colour values have changed by more than 25) we plot a red point, otherwise we plot a black point.

        dst += 3;

        if ((i+1) % W == 0) {
            // end of row, skip over padding
            dst += nPadBytes;
        }
    }

    memcpy(m_pPrev, lpBits, nPels*3);

We then increment the destination pointer by 3 bytes (over our pixel). If we have reached the end of our row, then we also have to compensate for any padding that occurs. After we have finished creating our memory bitmap, we copy all of our pixels to the previous frame buffer (m_pPrev) using memcpy. Note that it is the number of pixels multiplied by three to account for the RGB components of each pixel. Now we have to draw the frame...

    CDC* pDC = GetDC();

    int nOldMode = SetStretchBltMode(pDC->GetSafeHdc(), 
                                     COLORONCOLOR);

    StretchDIBits( pDC->GetSafeHdc(),
                   m_cVideoRect.left, 
                   m_cVideoRect.top,
                   m_cVideoRect.Width(), 
                   m_cVideoRect.Height(),
                   0,
                   0,
                   lpbi->biWidth, 
                   lpbi->biHeight,
                   m_pbits,
                   (BITMAPINFO*)m_pbi,
                   DIB_RGB_COLORS,
                   SRCCOPY);
 
    SetStretchBltMode(pDC->GetSafeHdc(),nOldMode);
    ReleaseDC(pDC);
}

Here we simply blit the bitmap on to our video control rectangle (that we pre-calculated earlier on, see example code). If we wanted to, we could have one area of the program displaying the real camera data, and another showing our analyzed data.

Conclusion

Once you have your camera set up to send the data to your application, it is a lot of fun messing around with various attributes to see the various effects it can have. For example, I added a few additional calculations that displayed the motion as a greyscale image denoting the amount of colour change (white for a lot, black for little). Here is a screenshot of me sitting down:

Obviously, the only problem with analyzing moving images in realtime is the speed factor. This project was done on a PIII-933 with 256Mbs RAM, therefore there was no slowdown. Slower computers might have trouble handling such large amounts of data (nearly 3.5Mbs a second at 15fps).

MotionCam.zip (20.7Kb)

Submitted: 17/03/2001

Article Toolbar

BibTeX entry