stitch2d¶
stitch2D is a Python script that stitches a two-dimensional grid of tiles into a mosaic. It was originally developed for stitching together images collected on various microscopes in the Department of Mineral Sciences at the Smithsonian National Museum of Natural History.
When tiles are stitched together by stitch2d, they are translated, not rotated, resized, or warped. As a result, stitch2d requires all images to be the same size and orientation. Images must overlap, although they don’t necessarily need to be arranged in a grid.
Source code for this project is located on GitHub.
Contents¶
User guide¶
Install¶
Install stitch2d with pip:
pip install stitch2d
Or install from the GitHub repository using git and pip:
git clone https://github.com/adamancer/stitch2d
cd stitch2d
pip install .
Quick start¶
The following code can be used to create and save a mosaic:
from stitch2d import create_mosaic
mosaic = create_mosaic("/path/to/tiles")
try:
mosaic.load_params()
except FileNotFoundError:
mosaic.downsample(0.6)
mosaic.align()
mosaic.reset_tiles()
mosaic.save_params()
mosaic.smooth_seams()
mosaic.save("mosaic.jpg")
A simple stitching workflow is also available from the command line. To create a smoothed mosaic and save it as a JPEG, run:
stitch2d path/to/tiles --smooth -output mosaic.jpg
For more information about using this command, including available parameters, run:
stitch2d --help
Overview¶
stitch2d includes two classes that can be used to create mosaics from a list of tiles:
Mosaic
, which incorporates no information about how the tiles in the mosaic are arrangedStructuredMosaic
, which arranges the tiles into a grid based on parameters supplied by the user
You can also use create_mosaic()
, as above,
which accepts the same arguments as
StructuredMosaic
. This function returns a
StructuredMosaic
if grid parameters are
provided or can be inferred from the filenames of the tiles or a
Mosaic
if not.
Mosaic¶
Since Mosaic
doesn’t know anything about
the tile structure, it can be slow, especially for large grids where
lots of tiles need to be compared. It’s almost always faster to use
StructuredMosaic
where possible.
Initialize a Mosaic
by pointing it to the
directory where the tiles of interest live:
from stitch2d import Mosaic
mosaic = Mosaic("/path/to/tiles")
Mosaic
also includes a class attribute,
num_cores
, to specify how many cores
it should use when aligning and stitching a mosaic. By default, it uses
one core. Modify this value with:
Mosaic.num_cores = 2
Even when using multiple cores, detecting and extracting features can be time consuming. One way to speed up the process is to reduce the resolution of the tiles being analyzed:
mosaic.downsample(0.6) # downsamples all tiles larger than 0.6 mp
Alternatively you can resize the tiles without the size check:
mosaic.resize(0.6) # resizes all tiles to 0.6 mp
You can then align the smaller tiles:
mosaic.align()
In either case, you can restore the full-size images prior to stitching the mosaic together:
mosaic.reset_tiles()
Sometimes brightness and contrast can vary significantly between
adjacent tiles, producing a checkerboard effect when the mosaic is
stitched together. This can be mitigated in many cases using
smooth_seams()
, which aligns
brightness/contrast between neighboring tiles by comparing areas of
overlap:
mosaic.smooth_seams()
Once the tiles have been positioned, the mosaic can be viewed:
mosaic.show()
Or saved to a file:
mosaic.save("mosaic.tif")
Or returned as a numpy array if you need more control over the final mosaic:
arr = mosaic.stitch()
The default backend, opencv, orders color channels as BGR. You may want to reorder the color channels before working with the image in a different program. To get an RGB image from a BGR image, use:
arr = arr[...,::-1].copy()
New in 1.1: Or specify the desired channel order when stitching:
arr = mosaic.stitch("RGB")
Once the tiles are positioned, their locations are stored in the
params
attribute, which can be saved
as JSON:
mosaic.save_params("params.json")
Those parameters can then be loaded into a new mosaic if needed:
mosaic.load_params("params.json")
StructuredMosaic¶
StructuredMosaic
allows the user to specify
how the tiles in the mosaic should be arranged. For tilesets of known
structure, it is generally faster but otherwise works the same as
Mosaic
. Initialize a structured mosaic
with:
from stitch2d import StructuredMosaic
mosaic = StructuredMosaic(
"/path/to/tiles",
dim=15, # number of tiles in primary axis
origin="upper left", # position of first tile
direction="horizontal", # primary axis (i.e., the direction to traverse first)
pattern="snake" # snake or raster
)
For large tilesets where adequate-but-imperfect tile placement is
acceptable, StructuredMosaic
can use its
knowledge of the tile grid to quickly build a mosaic based on the
positions of only a handful of tiles:
# Stop aligning once 5 tiles have been successfully placed
mosaic.align(limit=5)
# Build the rest of the mosaic based on the positioned tiles. If from_placed
# is True, missing tiles are appended to the already positioned tiles. If
# False, a new mosaic is calculated from scratch.
mosaic.build_out(from_placed=True)
The build_out()
method can
also be used to ensure that all tiles (including those that could not be
placed using feature matching) appear in the final mosaic. The primary
disadvantage of this method is that the placement of those tiles is less
precise.
Beyond 8-bit images¶
New in 1.2: The Tile class now includes a
prep_imdata()
method that can be used to
tweak the image data being used to align the mosaic. When using the
default OpenCVTile class, this method creates an 8-bit copy of the image
data to use for feature detection and matching while retaining the
original data to use when building the mosaic.
The default behavior of prep_imdata()
is
simplistic. To customize it, use a subclass. For example, the default
method scales the intensities of the original data based on the maximum
intensity found in the array. For images with a small number of
extremely bright pixels, this can yield unusably dim images. A better
approach may be to use np.percentile()
:
import numpy as np
class MyTile(OpenCVTile):
def prep_imdata(self):
imdata = self.imdata - self.imdata.min()
return np.uint8(255 * imdata / np.percentile(imdata, 99))
mosaic = create_mosaic("path/to/tiles", tile_class=MyTile)
Similar tools¶
The opencv package includes powerful tools for stitching 2D and 3D images). Much of that functionality has been ported to Python as the stitching package, which streamlines the opencv API and includes a useful tutorial. I didn’t have any luck getting it to work consistently with microscope tilesets, but it includes advanced features missing from this package (lens corrections, affine transformations beyond simple translation, etc.) and can be configured to work with 2D images. It’s definitely worth a look for tilesets more complex than the simple case handled here.
Fiji also includes a 2D/3D stitching tool.
API¶
Stitches a 2D grid of images into a mosaic
stitch2d.mosaic¶
Reads and stitches images from a 2D grid into a mosaic
- class stitch2d.mosaic.Mosaic(path_or_tiles, tile_class=None)[source]¶
Bases:
object
Stitches a mosaic from a list of tiles
- grid¶
tiles arranged into a grid
- Type
list
- shape¶
shape of the mosaic as (height, width[, channels])
- Type
tuple
- size¶
number of tiles in the mosaic
- Type
tuple
- tile_class¶
class to use for tiles in the mosaic
- Type
class
- num_cores = 1¶
Number of cores to use when processing images
- Type
int
- __init__(path_or_tiles, tile_class=None)[source]¶
Initializes a mosaic from a list of tiles
- Parameters
path_or_tiles (str or list-like) – either the path to a directory containing tiles, a list of Tiles, or a list of strings or arrays that can be used to create a Tile
tile_class (class) – class to use for tiles in the mosaic. Defaults to OpenCVTile.
- property tiles¶
Gets a flattened list of all tiles in grid order
- property placed¶
Calculates number of tiles that have been placed in the mosaic
- property params¶
Summarizes parameters needed to stitch mosaic
- property detector¶
Gets a copy of the detector used to align tiles in the mosaic
- property matcher¶
Gets a copy of the matcher used to align tiles in the mosaic
Only defined if using OpenCV.
- property pool¶
Returns a shared joblib pool, creating it if needed
- placeholder(tile=None, fill_value=0)[source]¶
Creates a placeholder tile to fill in gaps in the mosaic
- bounds()[source]¶
Calculates bounds of the mosaic comprising the placed tiles
- Returns
bounds of tile as (y1, x1, y2, x2)
- Return type
tuple
- copy()[source]¶
Creates a copy of the mosaic based on the grid
Using the grid instead of the list of tiles allows the grid-building step to be skipped when the copy is initialized.
- Returns
copy of the mosaic
- Return type
- save_params(path='params.json')[source]¶
Saves coordinates for placed tiles
- Parameters
path (str) – path to the JSON file
- load_params(path_or_obj='params.json')[source]¶
Loads coordinates for placed tiles at full scale
- Parameters
path_or_obj (str or dict) – path to the JSON file or param dict from another mosaic
- Raises
FileNotFoundError – thrown if input is path and path not found
ValueError – thrown if JSON can’t be decoded or does not match this mosaic
- resize(size_or_shape, *args, **kwargs)[source]¶
Rescales all tiles in the mosaic using size or shape
- Parameters
size_or_shape (int or tuple of ints) – size in megapixels or shape of resized image
*args – any argument accepted by the resize function used by the tile class
**kwargs – any keyword argument accepted by the resize function used by the tile class
- downsample(size_or_shape, *args, **kwargs)[source]¶
Downsamples all tiles in the mosaic using the given size or shape
- Parameters
size_or_shape (int or tuple of ints) – size in megapixels or shape of resized image
*args – any argument accepted by the resize function used by the tile class
**kwargs – any keyword argument accepted by the resize function used by the tile class
- align(origin=None, limit=None, **kwargs)[source]¶
Builds a mosaic by checking each tile against all others
- Parameters
origin (Tile) – the tile around which to build the mosaic. If not given, method will select the tile with the largest number of features.
limit (int) – the number of tiles that must be successfully placed before the method finishes. If not given, the method will continue until it runs out of adjacent tiles with matching features. Setting a limit allows a decent mosaic to be created quickly.
kwargs – any keyword argument accepted by the align_to method on the Tiles comprising this mosaic
- build_out(*args, **kwargs)[source]¶
Warns user that build_out is not implemented in Mosaic class
Use StructuredMosaic instead to get this functionality.
- smooth_seams(origin=None)[source]¶
Smooths intensities at seams between tiles
- Parameters
origin (Tile) – starting tile
- stitch(channel_order=None)[source]¶
Stitches mosaic using either placed tiles or row/col of tiles
- Parameters
channel_order (str) – order of the three color channels in the stitched array, for example, RGB. Uses the backend order if not given, which can give unexpected results (for example, OpenCV uses BGR).
- Returns
- Return type
numpy.ndarray
- class stitch2d.mosaic.StructuredMosaic(path_or_tiles, tile_class=None, dim=None, origin='upper left', direction='horizontal', pattern='raster')[source]¶
Bases:
stitch2d.mosaic.Mosaic
Stitches a mosaic from a list of tiles with a known structure
- __init__(path_or_tiles, tile_class=None, dim=None, origin='upper left', direction='horizontal', pattern='raster')[source]¶
Initializes a structured mosaic from a list of tiles
- Parameters
path_or_tiles (str or list-like) – either the path to a directory containing tiles, a list of Tiles, or a list of strings or arrays that can be used to create a Tile
tile_class (class) – class to use for tiles in the mosaic. Defaults to OpenCVTile.
dim (tuple or int) – either the shape of the mosaic as (height, width) or the number of tiles in the direction traversed first, that is, the number of columns (if horizontal) or number of rows (if vertical)
origin (str) – the position of the first tile in the mosaic. One of “upper left”, “upper right”, “lower left”, or “lower right”.
direction (str) – direction to traverse first when building the mosaic. Either “horizontal” or “vertical”.
pattern (str) – whether the grid is rastered or snaked. Either “raster” or “snake”.
- align(origin=None, limit=None, **kwargs)[source]¶
Builds a mosaic outward from a single tile using feature matching
- Parameters
origin (Tile) – the tile around which to build the mosaic. If not given, method will select a tile near the center of the mosaic.
limit (int) – the number of tiles that must be successfully placed before the method finishes. If not given, the method will continue until it runs out of adjacent tiles with matching features. Setting a limit allows a decent mosaic to be created quickly.
kwargs – any keyword argument accepted by the align_to method on the Tiles comprising this mosaic
- build_out(from_placed=True, offsets=None)[source]¶
Builds out from already placed tiles using the given offset
Used to complete mosaics that include tiles that were not placed when the mosaic was built, either because the user assigned a limit or because the feature matching algorithm failed to find a home for them.
- Parameters
from_placed (bool) – if True, unplaced tiles will be tacked onto already placed tiles using the given offsets. If False, a new mosaic will be calculated from scratch using the given offsets.
offsets (tuple) – offsets between adjacent tiles as dy_row, dx_row, dy_col, dx_col. If not given, the method will estimate the offsets if any tiles have been placed or will ignore offsets if not.
- stitch2d.mosaic.build_grid(items, dim, origin='upper left', direction='horizontal', pattern='raster', fill_value=None)[source]¶
Builds a grid from a list
- Parameters
items (list) – list to convert to a grid
dim (tuple or int) – either the shape of the mosaic as (height, width) or the number of tiles in the direction traversed first, that is, the number of columns (if horizontal) or number of rows (if vertical)
origin (str) – the position of the first tile in the mosaic. One of “upper left”, “upper right”, “lower left”, or “lower right”.
direction (str) – direction to traverse first when building the mosaic. Either “horizontal” or “vertical”.
pattern (str) – whether the grid is a raster or snake
fill_value – value used to fill missing items in a ragged grid
- Returns
List of rows in the grid
- Return type
list
stitch2d.tile¶
Reads and helps place a single image from a 2D grid
- class stitch2d.tile.Tile(data, detector='sift')[source]¶
Bases:
object
An image tile in a mosaic
- source¶
the original data used to created the tile. Either the path to an image file or an array with 1, 3, or 4 channels.
- Type
str or array-like
- imdata¶
image data
- Type
numpy.ndarray
- id¶
a UUID uniquely identifying the tile
- Type
str
- row¶
the index of the row where the tile appears in the mosaic
- Type
int
- col¶
the index of the column where the tile appears in the mosaic
- Type
int
- y¶
the y coordinate of the image within the mosaic
- Type
float
- x¶
the x coordinate of the image within the mosaic
- Type
float
- channel_order¶
the order of the three color channels in the image, e.g., RGB or BGR
- Type
str
- scale¶
the current scale of the tile relative to the original image
- Type
float
- features_detected¶
whether any features were detected in this image
- Type
bool
- descriptors¶
list of descriptors found in this image
- Type
numpy.ndarray
- keypoints¶
list of coordinates of descriptors found in this image
- Type
numpy.ndarray
- detectors = {}¶
maps strings to a subclass-specific feature detector
- Type
dict
- matchers = {}¶
maps strings to a subclass-specific feature matcher
- Type
dict
- __init__(data, detector='sift')[source]¶
Initializes a mosaic from a list of tiles
- Parameters
data (str or numpy.ndarray) – path to an image file or an array of image data
detector (str) – name of the detector used to find/extract features. Currently only sift is supported.
- property detector¶
Gets the detector used to align this tile to another tile
- property matcher¶
Gets the matcher used to align this tile to another tile
- property height¶
Gets the height of the image in pixels
- property width¶
Gets the width of the image in pixels
- property channels¶
Gets the number of channels in the image
- property channel_axis¶
Gets the index where channel info is stored
- property dtype¶
Gets the dtype of the image
- property shape¶
Gets the shape of the image
- property size¶
Gets the size of the image
- property mp¶
Gets the size of the image in megapixels
- property placed¶
Whether the tile has been assigned coordinates in the mosaic
- load_imdata()[source]¶
Loads copy of source data
- Returns
copy of source data
- Return type
numpy.ndarray
- copy()[source]¶
Creates a copy of the tile
- Parameters
grid (list of lists) – grid from the mosaic containing the tile
- Returns
copy of the tile
- Return type
- bounds(as_int=False)[source]¶
Calculates the position of the tile within the mosaic
- Parameters
as_int (bool) – whether bounds are converted to integers before returning
- Returns
bounds of the image in the mosaic coordinate system as (y1, x1, y2, x2)
- Return type
tuple
- neighbors()[source]¶
Finds adjacent tiles
- Parameters
y (int) – row index
x (int) – column index
- Returns
neighboring tiles keyed to direction (top, right, bottom, left)
- Return type
dict
- convert_mosaic_coords(y1, x1, y2, x2)[source]¶
Converts mosaic coordinates to image coordinates
- Returns
mosaic coordinates translated to image coordinates
- Return type
tuple
- update(other)[source]¶
Updates attributes to match another tile
- Parameters
other (Tile) – a tile with attributes to copy over to this one
- crop(box, convert_mosaic_coords=True)[source]¶
Crops tile to the given box
- Parameters
box (tuple) – box to crop to as (y1, x1, y2, x2)
convert_mosaic_coords (bool) – whether to convert the given coordinates from mosaic to image coordinates
- Returns
image data cropped to the given box
- Return type
numpy.ndarray
- intersection(other)[source]¶
Finds the intersection between two placed tiles
- Parameters
other (Tile) – an adjacent tile that has already been placed in the mosaic
- Returns
the overlapping portion of both tiles
- Return type
tuple of Tile
- intersects(other)[source]¶
Tests if two placed tiles intersect
- Parameters
other (Tile) – an adjacent tile that has already been placed in the mosaic
- Returns
True if tiles intersect, False otherwise
- Return type
bool
- reset()[source]¶
Restores original image and resets coordinate and feature attrs
- Returns
the original tile updated to restore the original image data
- Return type
- draw(others=None)[source]¶
Creates an image from the provided tiles
- Parameters
others (list of Tiles) – a list of tiles to include in the new image. Only tiles that have been placed will be included.
- Returns
an image including all provided tiles
- Return type
numpy.ndarray
- save(path, others=None)[source]¶
Saves an image created from the provided tiles
- Parameters
path (str) – file path
others (list of Tiles) – a list of tiles to include in the new image. Only tiles that have been placed will be included.
- show(others=None)[source]¶
Shows an image created from the provided tiles
- Parameters
others (list of Tiles) – a list of tiles to include in the new image. Only tiles that have been placed will be included.
- gray()[source]¶
Returns copy of image converted to grayscale
- Returns
grayscale version of the original iamge
- Return type
numpy.ndarray
- resize(size_or_shape, *args, **kwargs)[source]¶
Resizes image to a given size or shape
- Parameters
size_or_shape (float, int, or tuple of ints) – size in megapixels or shape of resized image
*args – any argument accepted by the resize function used by the subclass
**kwargs – any keyword argument accepted by the resize function used by the subclass
- downsample(size_or_shape, *args, **kwargs)[source]¶
Downsamples image to a given size or shape if smaller than original
- Parameters
size_or_shape (float, int, or tuple of ints) – size in megapixels or shape of resized image as (height, width)
*args – any argument accepted by the resize method of the subclass
**kwargs – any keyword argument accepted by the resize method of the subclass
- Returns
the original tile downsampled to the given size or shape
- Return type
- prep_imdata()[source]¶
Returns a copy of the tile data suitable for feature detection
Users may wish to create a subclass with a custom version of this method, for example, to scale intensities, enhance contrast, or select image data from an array that include additional bands.
- Returns
copy of image data
- Return type
numpy.ndarray
- detect_and_extract(*args, **kwargs)[source]¶
Detects and extracts features within the tile
- Parameters
*args – any argument accepted by the feature detection method on the detector
**kwargs – any keyword argument accepted by the feature detection method on the detector
- align_to(other, **kwargs)[source]¶
Aligns tile to another, already placed tile
- Parameters
other (Tile) – a tile that has already been placed in the mosaic
- class stitch2d.tile.OpenCVTile(data, detector='sift', matcher='flann')[source]¶
Bases:
stitch2d.tile.Tile
An image tile in a mosaic loaded and manipulated using OpenCV
See Tile for available attributes.
- detectors = {'sift': <stitch2d.tile._DefaultInstance object>}¶
maps strings to a subclass-specific feature detector
- Type
dict
- matchers = {'bf': <stitch2d.tile._DefaultInstance object>, 'flann': <stitch2d.tile._DefaultInstance object>}¶
maps strings to a subclass-specific feature matcher
- Type
dict
- __init__(data, detector='sift', matcher='flann')[source]¶
Initializes a mosaic from a list of tiles
- Parameters
data (str or numpy.ndarray) – path to an image file or an array of image data
detector (str) – name of the detector used to find/extract features. Currently only sift is supported.
- load_imdata()[source]¶
Loads copy of source data
- Returns
copy of source data
- Return type
numpy.ndarray
- gray()[source]¶
Returns copy of image converted to grayscale
- Returns
grayscale version of the original iamge
- Return type
numpy.ndarray
- resize(size_or_shape, *args, **kwargs)[source]¶
Resizes image to a given size or shape
- Parameters
size_or_shape (float, int, or tuple of ints) – size in megapixels or shape of resized image as (height, width)
*args – any argument accepted by cv.resize
**kwargs – any keyword argument accepted by cv.resize
- Returns
the original tile resized to the given size or shape
- Return type
- prep_imdata()[source]¶
Returns a copy of the tile data suitable for feature detection
The built-in version of this method checks if the imdata attribute is an 8-bit array, returning a copy if so. Otherwise, it rescales the intensities and returns an 8-bit copy of the array. The conversion is simplistic, and users may prefer to create a subclass with a custom version of this method instead, for example, to scale intensities, enhance contrast, or select image data from an array that include additional bands.
- Returns
copy of image data as an 8-bit array
- Return type
numpy.ndarray
- detect_and_extract(*args, **kwargs)[source]¶
Detects and extracts features within the tile
- Parameters
*args – any argument accepted by the detect_and_extract method on the detector
**kwargs – any keyword argument accepted by the detect_and_extract method on the detector
- Returns
the original tile updated with features and keypoints
- Return type
- align_to(other, **kwargs)[source]¶
Aligns tile to another, already placed tile
- Parameters
other (Tile) – a tile that has already been placed in the mosaic
- Returns
the original tile updated with x and y coordinates
- Return type
- class stitch2d.tile.ScikitImageTile(data, detector='sift')[source]¶
Bases:
stitch2d.tile.Tile
An image tile in a mosaic loaded and manipulated using scikit-image
See Tile for available attributes.
- detectors = {'sift': <stitch2d.tile._DefaultInstance object>}¶
maps strings to a subclass-specific feature detector
- Type
dict
- __init__(data, detector='sift')[source]¶
Initializes a mosaic from a list of tiles
- Parameters
data (str or numpy.ndarray) – path to an image file or an array of image data
detector (str) – name of the detector used to find/extract features. Currently only sift is supported.
- load_imdata()[source]¶
Loads copy of source data
- Returns
copy of source data
- Return type
numpy.ndarray
- gray()[source]¶
Returns copy of image converted to grayscale
- Returns
grayscale version of the original iamge
- Return type
numpy.ndarray
- resize(size_or_shape, *args, **kwargs)[source]¶
Resizes image to a given size or shape
- Parameters
size_or_shape (float, int, or tuple of ints) – size in megapixels or shape of resized image
*args – any argument accepted by skimage.transform.resize
**kwargs – any keyword argument accepted by skimage.transform.resize
- Returns
the original tile resized to the given size of shape
- Return type
- detect_and_extract(*args, **kwargs)[source]¶
Detects and extracts features within the tile
- Parameters
*args – any argument accepted by the detect_and_extract method on the detector
**kwargs – any keyword argument accepted by the detect_and_extract method on the detector
- Returns
the original tile updated with features and keypoints
- Return type
- align_to(other, **kwargs)[source]¶
Aligns tile to another, already placed tile
- Parameters
other (Tile) – a tile that has already been placed in the mosaic
- Returns
the original tile updated with x and y coordinates
- Return type