OpenCV and CameraServer

One of the coolest new features for 2017 is the addition of OpenCV and the new CameraServer to WPILib. Unlike NiVision, which was never enabled in RobotDotNet, OpenCV and CameraServer have been fully enabled.

CameraServer

CameraServer is the new library for handling USB and Ethernet cameras, along with streaming video from these devices, along with OpenCV, to other devices. All code can be found in the CSCore namespace. However in most instances, you should just be able to use the WPILib CameraServer class, which handles most of the configuration of these settings for you.

OpenCV

The OpenCV library we use is a fork of OpenCvSharp. It has been modified to work with the WPILib build system, but still handles all of the OpenCV operations you have come to expect. See the OpenCV API documentation for more information, and you should be able to look at most OpenCV documentation online for help as well.

Creating a USB Camera Stream

The CameraServer class can be used to easliy start streaming a USB Camera to either the LabVIEW dashboard, SmartDashboard, DotNetDash, or any web browser.

// Add to your existing RobotInit method
public override void RobotInit()
{
  CameraServer.Instance.StartAutomaticCapture();
}

Performing OpenCV operations on a USB Camera

OpenCV images can be grabbed directly from the USB Camera, and then the final output from OpenCV can be streamed back to the dashboard. The following example shows how to get an OpenCV image, draw an annotated rectangle on the image, and then send it back to the dashboard. Note that the sample runs this in a separate thread, as attempting to run this in a main robot loop would cause heavy control lag.

// Add additional includes
using CSCore;
using OpenCvSharp;
using System.Threading;

// Add to your existing RobotInit method
public override void RobotInit()
{
    Thread cameraThread = new Thread(() =>
    {
        // Get the USB Camera from the camera server
        UsbCamera camera = CameraServer.Instance.StartAutomaticCapture();
        camera.SetResolution(640, 480);

        // Get a CvSink. This will capture Mats from the Camera
        CvSink cvSink = CameraServer.Instance.GetVideo();
        // Setup a CvSource. This will send images back to the dashboard
        CvSource outputStream = CameraServer.Instance.PutVideo("Rectangle", 640, 480);

        // Mats are very expensive. Let's reuse this Mat.
        Mat mat = new Mat();

        while (true)
        {
            // Tell the CvSink to grab a frame from the camera and put it
            // in the source mat.  If there is an error notify the output.
            if (cvSink.GrabFrame(mat) == 0) {
                // Send the output the error.
                outputStream.NotifyError(cvSink.GetError());
                // skip the rest of the current iteration
                continue;
            }
            // Put a rectangle on the image
            Cv2.Rectangle(mat, new Point(100, 100), new Point(400, 400),
                    new Scalar(255, 255, 255), 5);
            // Give the output stream a new image to display
            outputStream.PutFrame(mat);
        }
    });
    cameraThread.IsBackground = true;
    cameraThread.Start();
}

Note the above example can also be used with an Axis ethernet camera, using the sample code below.

// Add additional includes
using CSCore;
using OpenCvSharp;
using System.Threading;

// Add to your existing RobotInit method
public override void RobotInit()
{
    Thread cameraThread = new Thread(() =>
    {
        // Get the Axis Camera from the camera server
        AxisCamera camera = CameraServer.Instance.AddAxisCamera("axis-camera.local");
        camera.SetResolution(640, 480);

        // Get a CvSink. This will capture Mats from the Camera
        CvSink cvSink = CameraServer.Instance.GetVideo();
        // Setup a CvSource. This will send images back to the dashboard
        CvSource outputStream = CameraServer.Instance.PutVideo("Rectangle", 640, 480);

        // Mats are very expensive. Let's reuse this Mat.
        Mat mat = new Mat();

        while (true)
        {
            // Tell the CvSink to grab a frame from the camera and put it
            // in the source mat.  If there is an error notify the output.
            if (cvSink.GrabFrame(mat) == 0) {
                // Send the output the error.
                outputStream.NotifyError(cvSink.GetError());
                // skip the rest of the current iteration
                continue;
            }
            // Put a rectangle on the image
            Cv2.Rectangle(mat, new Point(100, 100), new Point(400, 400),
                    new Scalar(255, 255, 255), 5);
            // Give the output stream a new image to display
            outputStream.PutFrame(mat);
        }
    });
    cameraThread.IsBackground = true;
    cameraThread.Start();
}