NeuroPAL_ID Program

Back to overview

Core ZephIR Calls

recommend_frames

track_all

extract_traces

Native ZephIR Components

main

ZephIR: multiple object tracking via image registration

See /docs/Guide-parameters.md for detailed explanations and tips for using optional arguments.

Usage:

zephir -h | –help zephir -v | –version zephir –dataset=<dataset> [options] zephir –dataset=<dataset> [options]

Options:

-h –help show this message and exit. -v –version show version information and exit. –dataset=<dataset> path to data directory to analyze. –load_checkpoint=<load_checkpoint> resume from last checkpoint. [default: False] –load_args=<load_args> load arguments from existing args.json file. [default: False] –allow_rotation=<allow_rotation> enable rho parameter to rotate descriptors. [default: False] –channel=<channel> data channel to register. –clip_grad=<clip_grad> maximum value for gradients; use -1 to uncap gradients. [default: 1.0] –cuda=<cuda> check if a CUDA-compatible GPU is available for use. [default: True] –dimmer_ratio=<dimmer_ratio> ratio to dim out non-foveated regions. [default: 0.1] –exclude_self=<excluse_self> exclude annotations with provenance ‘NEIR’. [default: True] –exclusive_prov=<exclusive_prov> only load annotations with given provenance. –fovea_sigma=<fovea_sigma> sigma for gaussian mask for foveated regions; use -1 to disable. [default: 2.5] –gamma=<gamma> gamma correction coefficient. [default: 2] –grid_shape=<grid_shape> size of image descriptor in the xy-plane. [default: 25] –include_all=<include_all> include all existing annotations in save file. [default: True] –lambda_d=<lambda_d> coefficient for center detection loss. [default: -1.0] –lambda_n=<lambda_n> coefficient for intra-keypoint spring constant. [default: 1.0] –lambda_n_mode=<lambda_n_mode> method by which spring loss is calculated. [default: disp] –lambda_t=<lambda_t> coefficient for temporal loss. [default: -1.0] –load_nn=<load_nn> load in manually defined spring connections if available. [default: True] –lr_ceiling=<lr_ceiling> maximum value for initial learning rate. [default: 0.2] –lr_coef=<lr_coef> coefficient for initial learning rate. [default: 2.0] –lr_floor=<lr_floor> minimum value for initial learning rate. [default: 0.02] –motion_predict=<motion_predict> enable flow field motion prediction with partial annotations. [default: False] –n_chunks=<n_chunks> number of steps to divide the forward pass into. [default: 10] –n_epoch=<n_epoch> number of iterations. [default: 40] –n_epoch_d=<n_epoch_d> number of iterations for center detection. [default: 10] –n_frame=<n_frame> number of frames to analyze for temporal loss. [default: 1] –n_ref=<n_ref> override for shape_n; requires an annotated frames with exactly n_ref keypoints. –nn_max=<nn_max> maximum number of neighbors connected for spring loss. [default: 5] –save_mode=<save_mode> mode for saving results. [default: o] –sort_mode=<sort_mode> method for sorting frame order and parent-child tree. [default: similarity] –t_ignore=<t_ignore> frames to ignore for analysis. –t_ref=<t_ref> override for reference frames to search for annotations. –t_track=<t_track> frames to include for analysis; supercedes t_ignore. –wlid_ref=<wlid_ref> subset of keypoints to track by worldline_id; supercedes n_ref. –z_compensator=<z_compensator> number of additional gradient descent steps for z-axis. [default: -1.0]

zephir.main.main()

zephir.main.run_zephir(dataset: Path, args: dict)

annotator

This provides a webpage to label key-points in a given dataset.

Usage:

annotator [options]

Options:

-h –help Show this help. -v –version Show version information. –dataset=PATH Location of data for labeling. –port=PORT Port to serve on. [default: 5000]

zephir.annotator.main.downsample(v: ndarray, scale: Tuple[int, ...]) → ndarray

Downsample a volume by a factor that divides the given dimension.

zephir.annotator.main.get_annotations(t)

zephir.annotator.main.get_app_metadata()

zephir.annotator.main.get_binary_image(t, view, view_idx)

zephir.annotator.main.get_gvol(t: int, idx: int) → bytes

Just the green channel as an 8-bit array.

zephir.annotator.main.get_mip_x(t: int, idx: int) → bytes

zephir.annotator.main.get_mip_y(t: int, idx: int) → bytes

zephir.annotator.main.get_mip_z(t: int, idx: int) → bytes

zephir.annotator.main.get_rgba_volume(t: int) → ndarray

Add blank blue and alpha channels to an array with max-stride color dimension, permute the color dimension to the min-stride position, and rearrange memory to make the color dimension min-stride (last).

zephir.annotator.main.get_vol(t: int, idx: int) → bytes

zephir.annotator.main.get_volume(t: int) → ndarray

zephir.annotator.main.get_x(t: int, idx: int) → bytes

zephir.annotator.main.get_y(t: int, idx: int) → bytes

zephir.annotator.main.get_z(t: int, idx: int) → bytes

zephir.annotator.main.handle_annotation(id)

zephir.annotator.main.handle_annotations()

zephir.annotator.main.handle_load()

zephir.annotator.main.handle_rpc()

zephir.annotator.main.handle_save()

zephir.annotator.main.handle_worldline(id)

zephir.annotator.main.handle_worldlines()

zephir.annotator.main.main()

CLI entry point.

zephir.annotator.main.serve_colormaps(path)

zephir.annotator.main.serve_dir_directory_index()

zephir.annotator.main.serve_favicon()

zephir.annotator.main.serve_manifest()

zephir.annotator.main.serve_static_css(path)

zephir.annotator.main.serve_static_js(path)

models

class zephir.models.container.Container(dataset, **kwargs)

Zephir variable container.

Store and handle variables and properties that are shared across Zephir methods.

get(key: str)

Fetch variable with given name.

Parameters:

key – name of variable to fetch.

Returns:

specified variable

update(props: dict)

Update value for existing variable or add as new variable to container.props.

Parameters:

props – dictionary of names and variables to store.

This file collects all functions directly related to calculating the four major components of ZephIR’s loss function: 1. corr_loss: normalized correlation loss for image registration 2. reg_n: spring network maintaining flexible connections to nearest neighbors 3. reg_t: linear temporal smoothing across a small sequence of frames 4. reg_d: nuclei probability as detected by ZephOD network

zephir.models.losses.corr_loss(prediction, target)

Image registration loss, L_R.

Normalized correlation loss between two lists of volumes (T, C, Z, Y, X). Loss is calculated over (Z, Y, X) axes and averaged over (C) axis. (T) axis is not reduced.

Parameters:

• prediction – child descriptors

• target – target descriptors

Returns:

loss

zephir.models.losses.reg_d(k, pred, rho, n_chunks)

Feature detection loss, L_D.

Calculates feature detection probability at keypoint coordinates.

Parameters:

• k – multiplier to modulate relative contribution to loss, lambda_D

• pred – Zephod prediction of feature detection probability map

• rho – keypoint coordinates

• n_chunks – number of chunks to divide forward process into

Returns:

loss

zephir.models.losses.reg_n(k, rho, rho_p, ind, d_ref, subset=None, mode='disp')

Spatial regularization, L_N.

Spring-like cost to deformations in a network of nearest-neighbor connections, penalizing relative motion between keypoints and their neighbors, calculated according to given mode: ‘disp’: linear cost to all relative motion ‘norm’: linear cost to change in distance between keypoints, rotation is allowed ‘ljp’: Lennard-Jones cost to change in distances between keypoints, rotation is allowed, collapsing onto the same coordinates is heavily penalized

Parameters:

• k – spring stiffnesses

• rho – keypoint coordinates

• rho_p – restricted keypoint coordinates

• ind – neighbor indices

• d_ref – neighbor displacements in reference frame

• subset – indices of keypoints in restricted list

• mode – method for calculating loss

Returns:

loss

zephir.models.losses.reg_t(k, descriptors, crop_rads, npx_to_keep)

Temporal smoothing, L_T.

Extracts pixel intensities at center of descriptors and penalizes change in intensity from the center frame.

Parameters:

• k – multiplier to modulate relative contribution to loss, lambda_T

• descriptors – child descriptors (T, C, Z, Y, X)

• crop_rads – radius of center crop (Z, Y, X)

• npx_to_keep – number of pixels to keep when calculating average intensity value

Returns:

loss

class zephir.models.zephir.ZephIR(allow_rotation, dimmer_ratio, fovea_sigma, grid_shape, grid_spacing, n_chunks, n_frame, shape_n, ftr_ratio=0.6, ret_stride=2)

forward(input_tensor)

Defines the computation performed at every call.

Should be overridden by all subclasses.

Note

Although the recipe for forward pass needs to be defined within this function, one should call the Module instance afterwards instead of this since the former takes care of running the registered hooks while the latter silently ignores them.

methods

auto_annotate.py: use ZephOD’s feature detection to automatically annotate a frame in a dataset.

Usage:

auto_annotate -h | –help auto_annotate -v | –version auto_annotate –dataset=<dataset> [options]

Options:

-h –help show this message and exit. -v –version show version information and exit. –dataset=<dataset> path to data directory to analyze. –tidx=<tidx> frame number or time index to analyze. [default: 0] –channel=<channel> data channel to register. –cuda=<cuda> check if a CUDA-compatible GPU is available for use. [default: True] –min_distance=<min_distance> minimum distance between detected features in pixels. [default: 4] –min_val=<min_val> minimum intensity value for a peak. [default: 1] –model=<model> path to checkpoint for model.

zephir.methods.auto_annotate.auto_annotate(dataset: Path, tidx: int, args: dict)

zephir.methods.auto_annotate.main()

zephir.methods.build_annotations.build_annotations(container, annotation, t_ref, wlid_ref, n_ref)

Load and handle annotations from annotations.h5 file.

Annotations are loaded and sorted according to user arguments, and used to populate an empty results array. This array will be filled during tracking.

Parameters:

container – variable container; needs to contain: dataset,

exclude_self, exclusive_prov, shape_t :param annotation: override annotations to use instead of loading from file :param t_ref: override frames to use as annotations :param wlid_ref: override worldline id’s to analyze :param n_ref: override maximum number of keypoints to analyze :return: container (updated entries for: annot, shape_n, partial_annot, provenance, t_annot, worldline_id), results (pre-filled with loaded annotations)

zephir.methods.build_models.build_models(container, dimmer_ratio, grid_shape, fovea_sigma, n_chunks)

Compile PyTorch models for tracking.

Models are built according to user arguments. Zephod is only built when a compatible model is available.

Parameters:

container – variable container, needs to contain: dataset, allow_rotation,

dev, img_shape, n_frame, shape_n :param dimmer_ratio: Zephir parameter, dims descriptor edges relative to center :param grid_shape: Zephir parameter, descriptor size :param fovea_sigma: Zephir parameter, size of foveated region :param n_chunks: Zephir parameter, number of chunks to divide forward pass into :return: container (updated entry for: grid_shape), zephir, zephod

zephir.methods.build_springs.build_springs(container, load_nn, nn_max, verbose=False)

Build nearest-neighbor spring network for spatial regularization.

Build KDTree of nn_max nearest neighbors around each annotation. These connections are used to calculate spatial regularization loss, L_N. Covariances of connected pairs are compiled if multiple reference frames are available.

Parameters:

container – variable container, needs to contain: dataset, annot,

grid_shape, img_shape, shape_n, t_annot, verbose, z_compensator :param load_nn: load or save existing nn_idx.txt for spring connection indices :param nn_max: maximum number of neighbors to connect to for each keypoint :return: container (updated entries for: covar, neighbors)

zephir.methods.get_optimization_trajectory.get_optimization_trajectory(frame_to_optimize, parent, reference, container, results, zephir, zephod, clip_grad, lambda_t, lambda_d, lambda_n, lambda_n_mode, lr_ceiling, lr_coef, lr_floor, n_epoch, n_epoch_d, kernel_size=(3, 9, 9), lambda_n_decay=1.0, lr_step_size=10, lr_gamma=0.5, nb_delta=(2, 1), nb_epoch=5, sigmas=(1, 4, 4))

overwrite_checkpoint.py: overwrite existing key and value in ZephIR checkpoint.

Usage:

overwrite_checkpoint.py -h | –help overwrite_checkpoint.py -v | –version overwrite_checkpoint.py –dataset=<dataset> –key=<key> –value=<value> [options]

Options:

-h –help show this message and exit. -v –version show version information and exit. –dataset=<dataset> path to data directory to analyze. –key=<key> name of item in checkpoint to overwrite. –value=<value> new item to write into checkpoint.

zephir.methods.overwrite_checkpoint.main()

zephir.methods.overwrite_checkpoint.overwrite_checkpoint(dataset, key, value)

zephir.methods.plot_loss_maps.plot_loss_maps(keypoint_to_visualize, frame_to_visualize, reference, map_resolution, map_size, trajectory, container, dimmer_ratio, grid_shape, fovea_sigma, n_chunks, zephod, lambda_d, lambda_n, lambda_n_mode)

zephir.methods.save_annotations.save_annotations(container, results, save_mode)

Save results to file.

Handles tracking results and compiles to an annotations data frame to save to file according to save_mode: ‘o’ will overwrite existing annotations.h5, ‘w’ will write to coordinates.h5. Existing annotations will overwrite ZephIR results if include_all is True.

Parameters:

container – variable container, needs to contain: dataset, exclude_self,

exclusive_prov, include_all, p_list, provenance, shape_t, shape_n, worldline_id :param results: tracking results :param save_mode: mode for writing to file

track_frame.py: track keypoints in a single frame.

Annotator GUI macro for ZephIR to track keypoints in a single frame (and its child frames, if specified). Necessary components are loaded from checkpoint.pt to expedite the process.

zephir.methods.track_frame.track_frame(dataset, annotation, t_idx, restrict_update=True, recompile_model=False, update_children=False)

zephod

ZephOD feature detection.

Usage:

zephod -h | –help zephod -v | –version zephod –dataset=<dataset> –model=<model> [options]

Options:

-h –help show this message and exit. -v –version show version information and exit. –dataset=<dataset> path to data directory to analyze. –model=<model> path to checkpoint for model. –channel=<channel> data channel to use as input. –cuda=<cuda> check if a CUDA-compatible GPU is available for use. [default: True]

zephir.zephod.main.main()

zephir.zephod.main.run_zephod(dataset=WindowsPath('.'), dev=device(type='cpu'), channel=None, model_kwargs=None, state_dict=None)

zephir.zephod.augment.generate_synthetic_data(vol, annotations, isolates, preprocess=False, use_original=False)

zephir.zephod.augment.identify(vol, annotations, r_crop)

zephir.zephod.augment.resize_neuron(neuron, shape=None)

zephir.zephod.augment.rotate_neuron(neuron)

ZephOD network uses all functions listed below as input channels to combine/select from. This file may be edited by a user to fit their particular use case, or to add new State-of-the-Art segmentation/detection algorithms, but each edit will require retraining of the weights. Each function needs to take only the volumetric data slice (Z, Y, X) as the input, and return the processed volume with the same shape as the output. Each function will be iterated over all data channels.

zephir.zephod.channels.apply_255_lut(img: ndarray, lo=0, hi=255) → ndarray

zephir.zephod.channels.rl_deconvolution(img, sigma=5, gamma=1.0, lam=12, n_iter=10)

zephir.zephod.channels.sharpen(img: ndarray, sharp=array([[0, -3, 0], [-3, 16, -3], [0, -3, 0]])) → ndarray

zephir.zephod.channels.threshold(img: ndarray, lo=50, hi=200) → ndarray

class zephir.zephod.model.ZephOD(n_channels_in=6, n_channels_out=1, init_nodes=16, kernel=(1, 3, 3), padding=1, pool_kernel=(2, 2, 2))

forward(input_tensor)

Defines the computation performed at every call.

Should be overridden by all subclasses.

Note

Although the recipe for forward pass needs to be defined within this function, one should call the Module instance afterwards instead of this since the former takes care of running the registered hooks while the latter silently ignores them.

preprocess(vol)

zephir.zephod.model.conv(n_channels_in, n_channels_out, activation, kernel=(1, 3, 3), padding=1)

Training ZephOD feature detection network

Usage:

train_zephod -h | –help train_zephod -v | –version train_zephod –index=<index> [options]

Options:

-h –help show this message and exit. -v –version show version information and exit. –index=<index> path to json index of data directories to use as training data. –batch_size=<batch_size> batch size in volumes. [default: 1] –channel=<channel> data channel to use as input. –cuda=<cuda> check if a CUDA-compatible GPU is available for use. [default: True] –loss_func=<loss_func> loss function to use. [default: BCE] –lr_init=<lr_init> learning rate for optimizer. [default: 1.0] –model=<model> name of model being trained. –n_epoch=<n_epoch> number of epochs to train for. [default: 64] –n_trn=<n_trn> number of training samples per epoch. [default: 50] –n_val=<n_val> number of validation samples per epoch. [default: 5] –override=<override> override existing checkpoint. [default: False]

zephir.zephod.train.main()

zephir.zephod.train.train_model(index=WindowsPath('.'), dev=device(type='cpu'), channel=None, n_epoch=10, loss_func='BCE', lr_init=0.1, batch_size=1, n_trn=100, n_val=10, model_kwargs=None, state_dict=None)

Customized ZephIR Classes

build_pdists

build_tree

n_io

save_movie

Helper Scripts

vddiag

annotation_converter

converter