| name | roi-search-workflow |
| description | Procedure for searching for a region of interest in a sample using microscope image acquisition tools during general chat. |
Overview
This skill describes how to search for a region of interest (ROI) in a sample
using a microscope image acquisition workflow. The goal is to find a field of
view (FOV) that contains a requested feature and then center that feature in the
FOV.
The desired ROI either contains a specific feature of interest or has structure
that matches the user's description or reference images. Search can use various
strategies, ranging from a basic acquire-and-check approach to a more
sophisticated coarse-to-fine, multi-scale method. The image acquisition tool is
your core asset: it lets you acquire an image at a given location, optionally
with a specified sampling interval (scan step size) and FOV size. Use it to
collect images of the sample at different locations and with different
configurations to achieve the goal.
Required Tools
To perform ROI search effectively, you need:
acquire_image: Acquire an image at a specified location and with a specified
field of view.
If this tool is unavailable, notify the user that the ROI-search workflow cannot
be executed as described and explain what capability is missing.
Required Inputs
Before starting the workflow, make sure you have the following information:
- A description of the feature to search for, or a reference image of the desired
feature provided by the user.
- User-suggested search strategy (optional).
- The allowed search range in x and y.
- The initial FOV size.
- The initial step size in x and y (unless the user explicitly asks you to avoid
searching with fixed step sizes).
- The goal: does the final FOV need to be centered on the described feature or
aligned with the reference image, or is it enough for the described or shown
features to be visible in the FOV?
If any essential information is missing, ask the user for it before proceeding.
Check the tool schema as well, because the exact acquisition arguments may vary
across instruments. For example, some image acquisition tools may not take an
FOV size.
Search Strategy
The most fundamental search strategy is an acquire-check-move loop:
- Acquire an image at the corner of the search range.
- Check whether the desired feature is in the image or whether the image
matches the user-provided reference.
- If not, move to the next location and repeat the acquisition and checking.
This strategy should be used if the user did not specify any strategy.
By default, move the FOV with constant x and y strides. If the user did not
specify a stride, use the FOV size so adjacent images are tightly connected. The
default search path in the sample space should be a snake pattern with constant
x and y strides. Here is an example snake pattern:
Snake-pattern scan with constant x and y strides
(start at lower-left corner in this example)
y ^
|
| row 4 o ---> o ---> o ---> o
| |
| row 3 o <--- o <--- o <--- o
| |
| row 2 o ---> o ---> o ---> o
| |
| row 1 o <--- o <--- o <--- o
| |
| row 0 o ---> o ---> o ---> o
|
+----------------------------------------> x
x stride = constant horizontal spacing
y stride = constant vertical spacing
The user may also suggest a search strategy. For example, one possible strategy
is to run a coarse or large-spaced scan across the entire sample or a large area
within it, then perform finer searches in areas that may contain the desired
feature. Use the image acquisition tool flexibly to follow the user's
instructions. If the image acquisition tool does not support the specified
strategy, notify the user and suggest falling back to the default
acquire-check-move snake pattern.
Use your Python coding tool creatively and flexibly to perform necessary
analyses on the fly or to visualize the search state. For example, at the end
of a coarse or random search pass, you can generate a large image with local
images placed at their respective locations. Use that overview of the covered
sample area to help determine where to scan next. Some analyses may require
access to raw image data stored on disk in formats such as NPY, TIFF, or HDF5.
The acquisition tool might return paths to these raw data files, but this is
not guaranteed.
Tips
Visualization
Use your Python coding tool to create visualizations that help you understand
the search state. For example, if you have access to the raw image data,
you can generate a script that creates a large image buffer, and place the
acquired images into that buffer at their respective locations, stitching
them together into a panorama of the searched area, and update the image
after each acquisition. This can help you see which areas have been covered and
identify promising regions to focus on in subsequent search passes.
See update_and_view_panorama.py in the skill directory for an example of how
to do this.
Finalization
When the feature begins to appear in the image, refine the search more
flexibly. If the user needs the final FOV to be centered on the desired feature,
adjust the acquisition location in the direction needed to bring the feature toward
the center. For example:
- if the feature is to the left, move the FOV left;
- if the feature is toward the top, move the FOV up.
Once the feature of interest is found and, if necessary, centered, report the
final FOV coordinates clearly to the user.
Operating Guidance
- Explain the rationale for each acquisition call.
- Make only one tool call at a time.
- Respect the user-provided search bounds.
- When writing coordinates, explicitly label axes and use
(x = ..., y = ...)
format instead of listing unlabeled numbers.
- If the search cannot continue safely or productively because the feature is
not identifiable, the bounds are too restrictive, or the required tool is not
available, notify the user clearly.
- Avoid repeatedly acquiring images at the same or very similar locations. If
you find yourself sampling nearly the same location over and over, stop and
reconsider the search strategy instead of continuing blindly.
