Vision callback functions

Child scripts, or customization scripts can include a vision callback function (which is one of many system callback functions), when associated with a vision sensor. When present for a given vision sensor, then the system will call the callback function everytime a new image was acquired or applied, allowing the user to perform image processing. This is the case with following API functions: sim.handleVisionSensor, sim.checkVisionSensor, sim.checkVisionSensorEx, and sim.setVisionSensorImg.

Some conditions apply as to the location of the vision callback function: if a vision callback function is present in a child script as well as in a customization script, both attached to the vision sensor, then the child script will be called first, and the customization script second.

Following represents an empty vision callback function:

#python def sysCall_vision(inData): # We have: # inData['handle'] : the handle of the vision sensor. # inData['resolution'] : the x/y resolution of the vision sensor # inData['clippingPlanes'] : the near and far clipping planes of the vision sensor # inData['viewAngle'] : the view angle of the vision sensor (if in persp. proj. mode) # inData['orthoSize'] : the ortho size of the vision sensor (if in orth. proj. mode) # inData['perspectiveOperation'] : true if the sensor is in persp. proj. mode outData = {} outData['trigger'] = False # True if the sensor should trigger outData['packedPackets'] = [] # a list of packed packets. Can be accessed via e.g. sim.readVisionSensor return outData --lua function sysCall_vision(inData) -- We have: -- inData.handle : the handle of the vision sensor. -- inData.resolution : the x/y resolution of the vision sensor -- inData.clippingPlanes : the near and far clipping planes of the vision sensor -- inData.viewAngle : the view angle of the vision sensor (if in persp. proj. mode) -- inData.orthoSize : the ortho size of the vision sensor (if in orth. proj. mode) -- inData.perspectiveOperation : true if the sensor is in persp. proj. mode local outData = {} outData.trigger = false -- true if the sensor should trigger outData.packedPackets = {} -- a table of packed packets. Can be accessed via e.g. sim.readVisionSensor return outData end

Image processing can be performed by using various API functions. The vision plugin exports a few very simple image processing functions. Many more image processing functions are supported via the image plugin (OpenCV wrapper).

Following represents a simple edge detection vision callback function, that triggers and returns a packet of data (based on the vision plugin functions):

#python import struct def sysCall_vision(inData): simVision.sensorImgToWorkImg(inData['handle']) simVision.edgeDetectionOnWorkImg(inData['handle'], 0.1) simVision.workImgToSensorImg(inData['handle']) outData = {} outData['trigger'] = True packetData = [1.0, 42.123, 129.3] outData['packedPackets'] = [struct.pack('ff', *packetData)] return outData --lua function sysCall_vision(inData) simVision.sensorImgToWorkImg(inData.handle) simVision.edgeDetectionOnWorkImg(inData.handle, 0.1) simVision.workImgToSensorImg(inData.handle) local outData = {} outData.trigger = true local packetData = {1.0, 42.123, 129.3} outData.packedPackets = {sim.packFloatTable(packetData)} return outData end

Following represents a vision callback function, that draws a circle onto the acquired image (based on the image plugin functions):

#python def sysCall_vision(inData): imgHandle = simIM.readFromVisionSensor(inData['handle']) center = [inData['resolution'][0] / 2, inData['resolution'][1] / 2] radius = (inData['resolution'][0] + inData['resolution'][1]) / 8 simIM.circle(imgHandle, center, radius, [255, 0, 255], 4) simIM.writeToVisionSensor(imgHandle, inData['handle']) simIM.destroy(imgHandle) --lua function sysCall_vision(inData) local imgHandle = simIM.readFromVisionSensor(inData.handle) local center = {inData.resolution[1] / 2, inData.resolution[2] / 2} local radius = (inData.resolution[1] + inData.resolution[2]) / 8 simIM.circle(imgHandle, center, radius, {255, 0, 255}, 4) simIM.writeToVisionSensor(imgHandle, inData.handle) simIM.destroy(imgHandle) end