Documentation

Vision

This documentation refers to a feature which is only available on the physical robot kits.

The sr.robot3 library contains support for detecting fiducial markers with the provided webcam. Markers are attached to various items in the Student Robotics arena. Each marker encodes a number in a machine-readable way, which means that robots can identify these objects. For information on which markers codes are which, see the markers page.

Using knowledge of the physical size of the different markers and the characteristics of the webcam, your robot can calculate the position of markers in 3D space relative to the camera. Therefore, if the robot can see a marker that is at a fixed location in the arena, a robot can calculate its exact position in the arena.

Under the hood, the vision system is based on Zoloto, a wrapper around OpenCV’s ArUco library, using the 36H11 marker set from April

Camera

The interface to the vision system is through the camera, accessible through R.camera.

see
Take a photo through the webcam, and return a list of Marker instances, each of which describes one of the markers that were found in the image.

Here’s an example that will repeatedly print out the distance to each arena marker that the robot can see:

from sr.robot3 import *
R = Robot()

while True:
    markers = R.camera.see()
    print("I can see", len(markers), "markers:")

    for m in markers:
        print(" - Marker #{0} is {1} metres away".format(m.id, m.distance / 1000))
see_ids
Take a photo through the webcam, and return a list of marker ids (not full Marker objects). This doesn’t do the same pose estimation calculations as see, and so is much faster to run.
from sr.robot3 import *
R = Robot()

while True:
    marker_ids = R.camera.see_ids()

    if 0 in marker_ids:
        print("I can see marker 0!")
    else:
        print("I cannot see marker 0!")
save
Take a photo through the webcam, and save it to the provided location.
from sr.robot3 import *
R = Robot()

# `R.usbkey` is the path to your USB drive
marker_ids = R.camera.save(R.usbkey / "initial-view.png")
capture
Take a photo through the webcam, and return the image data as an OpenCV array.
This feature is only available on version 2022.1.0 or later of the kit.
import cv2
from sr.robot3 import *
R = Robot()

frame = R.camera.capture()

# Flip the image with OpenCV
flipped = cv2.flip(frame, 0)

Field of View

The Logitech C500 has a field of view of 72° and the C270 has a field of view of 60°.

Definition of Axes

The vision system describes the markers it can see using three coordinate systems. These are intended to be complementary to each other and contain the same information in different forms.

The axis definitions match those in common use, as follows:

x-axis
The horizontal axis running left-to-right in front of the camera. Rotation about this axis is equivalent to leaning towards or away from the camera.
y-axis
The vertical axis running top-to-bottom in front of the camera. Rotation about this axis is equivalent to turning on the spot, to the left or right.
z-axis
The axis leading away from the camera to infinity. Rotation about this axis is equivalent to being rolled sideways.
Note that the axes are all defined relative to the camera. Since we have no way to know how you've mounted your camera, you may need to account for that in your usage of the vision system's data.

Objects of the Vision System

The vision system is made up of a number of objects, the primary of which is the Marker:

Marker

A Marker object contains information about a detected marker. It has the following attributes:

id
The id of the marker.
size
The physical size of the marker, as the vision system expects it.
pixel_centre
A Coordinate describing the position of the centre of the marker.
pixel_corners
A list of 4 Coordinate instances, each representing the position of the corners of the marker.
distance
The distance between the camera and the centre of the marker, in millimetres.
orientation
An Orientation instance describing the orientation of the marker.
spherical
A Spherical instance describing the position relative to the camera.
cartesian
A ThreeDCoordinate instance describing the absolute position of the marker relative to the camera.

Coordinate

A Coordinate object contains an x and y attribute. The exact meaning and unit of these attributes depends on its source.

ThreeDCoordinate

A ThreeDCoordinate object contains an x, y and z attribute. The exact meaning and unit of these attributes depends on its source.

Orientation

An Orientation object describes the orientation of a marker.

A visual representation of how the orientation axes work. Source: SourceBots

pitch
Rotation of the marker about the cartesian x-axis, in radians.

Leaning a marker away from the camera increases the value of rot_x, while leaning it towards the camera decreases it. A value of 0 indicates that the marker is upright.

yaw
Rotation of the marker about the cartesian y-axis, in radians.

Turning a marker clockwise (as viewed from above) increases the value of rot_y, while turning it anticlockwise decreases it. A value of 0 means that the marker is perpendicular to the line of sight of the camera.

roll
Rotation of the marker about the cartesian z-axis, in radians.

Turning a marker anticlockwise (as viewed from the camera) increases the value of rot_z, while turning it clockwise decreases it. A value of 0 indicates that the marker is upright.

rot_x
An alias for pitch.
rot_y
An alias for yaw.
rot_z
An alias for roll.
rotation_matrix
The rotation matrix represented by this orientation. A list of 3 lists, each with 3 items.
quaternion
The Quarternion instance represented by this orientation.

Spherical

The spherical coordinates system has three values to specify a specific point in space.

A visual representation of Spherical coordinates. Source: SourceBots

  • r - The radial distance, the distance from the origin to the point, in millimetres.
  • θ (theta) - The angle from the azimuth to the point, in radians.
  • φ (phi) - The polar angle from the plane of the camera to the point, in radians.
rot_x
Rotation around the X-axis, in radians, corresponding to theta on the diagram.
rot_y
Rotation around the Y-axis, in radians, corresponding to phi on the diagram.
dist
Distance, in millimetres, corresponding to r on the diagram.

The camera is located at the origin, where the coordinates are (0, 0, 0).