The Vision API
Currently the Vision API provides methods to find subimages or blobs on images. Results of such methods are single Match objects or arrays containing multiple Match objects. Things you might want to search for are for instance buttons on webpages or loot in games. To find non static, maybe spinning objects, one can use the more complex
Mostly for debugging purposes the Vision API provides a
.markMatches method. Those methods can be used to mark matches on a given image visually. That way you can check whether you choosed good parameters to find what you were looking for.
Table of contents
- Table of contents
- Access Vision API methods
- Vision API methods
- Vision.findMaskedMatches(image, tpl, mask, threshold, max_matches);
- Vision.findMaskedMatch(image, tpl, mask, threshold);
- Vision.findMatches(image, tpl, threshold, max_matches);
- Vision.findMatch(image, tpl, threshold);
- Vision.findBlobs(image, blob_tpl, min_distance, min_repeatability);
- Vision.markMatches(image, matches, color, thickness);
- Vision.markMatch(image, match, color, thickness);
Access Vision API methods
Every bot script has it’s own global Vision API object. That object holds all Vision API methods and can be used to access them.
var screenshot = Desktop.takeScreenshot(); var windows_icon = new Image("windows_icon.png"); var match = Vision.findMatch(screenshot, windows_icon);
Vision API methods
Vision.findMaskedMatches(image, tpl, mask, threshold, max_matches);
image(Image): The image to search the template on. (usually a screenshot).
tpl(Image): The template searched for on the
mask(Image): A mask telling the methods which parts of the template to use or ignore.
threshold(number): The resulting matches min score. Default: 0.8 (80%).
max_matches(number): The max amount of matches to look for. Default: -1 (unlimited).
Use this method to search for a masked template on the provided
max_matches matches, each of those matches has atleast the score defined by the
A template is an Image showing what you want to search for on an larger
image is usually a screenshot and the
tpl something like an image of loot in online games or buttons on websites.
Looking for non rectangular objects you might want the
.findMaskedMatches method to ignore parts of the provided
tpl. To do you you want create another image of the templates size. This images is called the
mask. Parts you want to ignore on the template are black on the
mask and parts you want to search for are white.
You can create mask manually or use the Image objects
Note: It’s actually recommended to not use the
.findMaskedMatches, but to use the
.findMatches method with a lower
threshold instead. Methods not taking a mask turned out to are more reliable.
Vision.findMaskedMatch(image, tpl, mask, threshold);
This methods does the same as the
.findMaskedMatches method, but always returns exactly one Match, instead of an array of matches.
If the method couldn’t a match, the returned match will be invalid.
Vision.findMatches(image, tpl, threshold, max_matches);
This methods works the same as the
.findMaskedMatches method, but does not take a mask as parameter.
Vision.findMatch(image, tpl, threshold);
This methods works the same as the
.findMaskedMatch method, but does not take a mask as parameter.
Vision.findBlobs(image, blob_tpl, min_distance, min_repeatability);
min_distance(number): default: 10
min_repeatability(number): default: 2
Finds light (white) blobs on the image as described by the
Note: All Match objects found using the
.findBlobs method have a score of 1.0.
If you want to search for blobs of a specific color, filter all unwanted colors using the Image objects
.isolateColorRange method in beforehand.
Vision.markMatches(image, matches, color, thickness);
image(Image): The image to draw the matches on (consider it’s size).
matches(Match Array): The matches to draw.
color(Color): The color of the rectangles marking the matches.
thickness(number): The size of the rectangles border in pixels. Default: 2
Draws a rectangle for each Match in the matches array in the provided
image. The position and size of the rectangles are provided by the matches
.getRect methods. The rectangles color will be the provided
color parameter and
thickness the rectangles thickness. Invalid matches, identified using the matches
.isValid methods, are not drawn.
image should be the image one has found the matches on or at least an image of the same size. If an match is outside of the image, it won’t be drawn.
Vision.markMatch(image, match, color, thickness);
Does the same as the
.markMatches method but only takes and draws a single match.