Features

Funtracks has a feature computation system that manages attributes (features) from tracking graphs with optional segmentation data. The architecture separates:

1. Feature metadata (what features exist and their properties) - Feature class
2. Feature computation (how to calculate feature values) - GraphAnnotator class
3. Feature storage (where feature values live on the graph) - attributes on the graph nodes/edges
4. Feature lifecycle (when to compute, activate, or update features) - computation called by Tracks, updates triggered by BasicActions

Core Components

1. Feature (TypedDict)

A Feature is a TypedDict that stores metadata about a graph feature.

Show API documentation

Bases: TypedDict

TypedDict for storing metadata associated with a graph feature.

Use factory functions like Time(), Position(), Area() etc. to create features with standard defaults.

The key is stored separately in the FeatureDict mapping (not in the Feature itself).

Attributes:

Name	Type	Description
feature_type	Literal['node', 'edge']	Specifies which graph elements the feature applies to.
value_type	ValueType	The data type of the feature values.
num_values	int	The number of values expected for this feature.
display_name	str	Optional. A display name for the feature. If not provided, the feature key is used.
value_names	Sequence[str]	Optional. Individual display names for each value when num_values > 1. Length should match num_values.
required	bool	If True, all nodes/edges in the graph are required to have this feature.
default_value	Any	If required is False, this value is returned whenever the feature value is missing on the graph.
spatial_dims	bool	Optional. If True, num_values must match the number of spatial dimensions (e.g., 2 for 2D, 3 for 3D). Used for features like Position and EllipsoidAxes.

2. FeatureDict

A FeatureDict is a dictionary (dict[str, Feature]) with special tracking for important feature keys:

Show API documentation

Bases: dict[str, Feature]

A dictionary mapping keys to Features, with special tracking for time/position.

Inherits from dict[str, Feature], so can be used directly as a dictionary. Provides convenient access to time and position features through properties.

Attributes:

Name	Type	Description
time_key	str	The key used for the time feature
position_key	str | list[str] | None	The key(s) used for position feature(s)

Parameters:

Name	Type	Description	Default
features	dict[str, Feature]	Mapping from feature keys to Features	required
time_key	str	The key for the time feature (must be in features)	required
position_key	str | list[str] | None	The key(s) for position feature(s)	required
tracklet_key	str | None	The key for the tracklet feature	required

edge_features

edge_features: dict[str, Feature]

A dict of all edge features

node_features

node_features: dict[str, Feature]

A dict of all node features

dump_json

dump_json() -> dict

Dump this FeatureDict to a json compatible dictionary

Returns:

Name	Type	Description
dict	dict	A map from the key "FeatureDict" containing features, time_key, position_key, and tracklet_key

from_json

from_json(json_dict: dict) -> FeatureDict

Generate a FeatureDict from a json dict such as one generated by dump_json

Parameters:

Name	Type	Description	Default
json_dict	dict	A dictionary with the key "FeatureDict" containing features, time_key, position_key, and tracklet_key	required

Returns:

Name	Type	Description
FeatureDict	FeatureDict	A FeatureDict object containing the features from the dictionary

register_position_feature

register_position_feature(
    key: str, feature: Feature
) -> None

Register the position feature and set the position_key.

Parameters:

Name	Type	Description	Default
key	str	The key to use for the position feature	required
feature	Feature	The Feature to register	required

register_tracklet_feature

register_tracklet_feature(
    key: str, feature: Feature
) -> None

Register the tracklet/track_id feature and set the tracklet_key.

Parameters:

Name	Type	Description	Default
key	str	The key to use for the tracklet feature	required
feature	Feature	The Feature to register	required

3. GraphAnnotator (Base Class)

An abstract base class for components that compute and update features on a graph.

Show API documentation

A base class for adding and updating graph features.

This class holds a set of features that it is responsible for. The annotator will compute these features and add them to the Tracks initially, and update them when necessary. The set of features will all be computed and updated together, although individual ones can be removed for efficiency.

Parameters:

Name	Type	Description	Default
tracks	Tracks	The tracks to manage features for.	required
features	dict[str, Feature]	A dict mapping keys to features that this annotator is capable of computing and updating.	required

Attributes:

Name	Type	Description
all_features	dict[str, tuple[Feature, bool]]	Maps feature keys to (feature, is_included) tuples. Tracks both what can be computed and what is currently being computed. Defaults to computing nothing.

features

features: dict[str, Feature]

The dict of features that this annotator currently manages.

Filtered from all_features based on inclusion flags.

activate_features

activate_features(keys: list[str]) -> None

Activate computation of the given features in the annotation process.

Filters the list to only features this annotator owns, ignoring others.

Parameters:

Name	Type	Description	Default
keys	list[str]	List of feature keys to activate. Only keys in all_features are activated.	required

can_annotate

can_annotate(tracks: Tracks) -> bool

Check if this annotator can annotate the given tracks.

Subclasses should override this method to specify their requirements (e.g., segmentation, SolutionTracks, etc.).

Parameters:

Name	Type	Description	Default
tracks	Tracks	The tracks to check compatibility with	required

Returns:

Type	Description
bool	True if the annotator can annotate these tracks, False otherwise

change_key

change_key(old_key: str, new_key: str) -> None

Rename a feature key in this annotator.

Base implementation updates the all_features dictionary. Subclasses should override this method to update any additional internal mappings they maintain.

Parameters:

Name	Type	Description	Default
old_key	str	Existing key to rename.	required
new_key	str	New key to replace it with.	required

Raises:

Type	Description
KeyError	If old_key does not exist in all_features.

compute

compute(feature_keys: list[str] | None = None) -> None

Compute a set of features and add them to the tracks.

This involves both updating the node/edge attributes on the tracks.graph and adding the features to the FeatureDict, if necessary. This is distinct from update to allow more efficient bulk computation of features.

Parameters:

Name	Type	Description	Default
feature_keys	list[str] | None	Optional list of specific feature keys to compute. If None, computes all currently active features. Any provided keys not in the currently active set are ignored.	None

Raises:

Type	Description
NotImplementedError	If not implemented in subclass.

deactivate_features

deactivate_features(keys: list[str]) -> None

Deactivate computation of the given features in the annotation process.

Filters the list to only features this annotator owns, ignoring others.

Parameters:

Name	Type	Description	Default
keys	list[str]	List of feature keys to deactivate. Only keys in all_features are deactivated.	required

update

update(action: BasicAction) -> None

Update a set of features based on the given action.

This involves both updating the node or edge attributes on the tracks.graph and adding the features to the FeatureDict, if necessary. This is distinct from compute to allow more efficient computation of features for single elements.

The action contains all necessary information about which elements to update (e.g., AddNode.node, AddEdge.edge, UpdateNodeSeg.node).

Parameters:

Name	Type	Description	Default
action	BasicAction	The action that triggered this update	required

Raises:

Type	Description
NotImplementedError	If not implemented in subclass.

4. GraphAnnotator Implementations

Annotator	Purpose	Requirements	Features Computed	API Reference
RegionpropsAnnotator	Extracts node features from segmentation using scikit-image's regionprops	segmentation must not be None	pos, area, ellipse_axis_radii, circularity, perimeter	📚 API
EdgeAnnotator	Computes edge features based on segmentation overlap between consecutive time frames	segmentation must not be None	iou (Intersection over Union)	📚 API
TrackAnnotator	Computes tracklet and lineage IDs for SolutionTracks	Must be used with SolutionTracks (binary tree structure)	tracklet_id, lineage_id	📚 API

5. AnnotatorRegistry

A registry that manages multiple GraphAnnotator instances with a unified interface. Extends list[GraphAnnotator].

Show API documentation

Bases: list[GraphAnnotator]

A list of annotators with coordinated operations.

Inherits from list[GraphAnnotator], so can be used directly as a list. Provides coordinated compute/update/enable/disable operations across all annotators.

Example

annotators = AnnotatorRegistry([ RegionpropsAnnotator(tracks, pos_key="centroid"), EdgeAnnotator(tracks), TrackAnnotator(tracks, tracklet_key="track_id"), ])

Can use as a list

annotators.append(MyCustomAnnotator(tracks))

Coordinated operations

annotators.activate_features(["area", "iou"]) annotators.compute()

Initialize with a list of annotators.

Parameters:

Name	Type	Description	Default
annotators	list[GraphAnnotator]	List of instantiated annotator objects	required

all_features

all_features: dict[str, tuple[Feature, bool]]

Dynamically aggregate all_features from all annotators.

Returns:

Type	Description
dict[str, tuple[Feature, bool]]	Dictionary mapping feature keys to (Feature, is_enabled) tuples

features

features: dict[str, Feature]

Get all currently active features from all annotators.

Returns:

Type	Description
dict[str, Feature]	Dictionary mapping feature keys to Feature definitions (only active features)

activate_features

activate_features(keys: list[str]) -> None

Activate features across all annotators.

Parameters:

Name	Type	Description	Default
keys	list[str]	List of feature keys to activate	required

Raises:

Type	Description
KeyError	If any feature keys are not available

change_key

change_key(old_key: str, new_key: str) -> None

Rename a feature key across all annotators.

Finds the annotator that owns the feature and calls its change_key method. This allows the Tracks user to rename features without needing to find the specific annotator that manages it.

Parameters:

Name	Type	Description	Default
old_key	str	Existing key to rename.	required
new_key	str	New key to replace it with.	required

Raises:

Type	Description
KeyError	If old_key does not exist in any annotator.

compute

compute(feature_keys: list[str] | None = None) -> None

Compute features across all annotators.

Parameters:

Name	Type	Description	Default
feature_keys	list[str] | None	Optional list of specific feature keys to compute. If None, computes all currently active features.	None

deactivate_features

deactivate_features(keys: list[str]) -> None

Deactivate features across all annotators.

Parameters:

Name	Type	Description	Default
keys	list[str]	List of feature keys to deactivate	required

Raises:

Type	Description
KeyError	If any feature keys are not available

update

update(action: BasicAction) -> None

Update features across all annotators based on the action.

Parameters:

Name	Type	Description	Default
action	BasicAction	The action that triggered this update	required

6. Tracks

The main class representing a set of tracks: a graph + optional segmentation + features.

Show API documentation

A set of tracks consisting of a graph and an optional segmentation.

The graph nodes represent detections and must have a time attribute and position attribute. Edges in the graph represent links across time.

Attributes:

Name	Type	Description
graph	DiGraph	A graph with nodes representing detections and and edges representing links across time.
segmentation	ndarray | None	An optional segmentation that accompanies the tracking graph. If a segmentation is provided, the node ids in the graph must match the segmentation labels.
features	FeatureDict	Dictionary of features tracked on graph nodes/edges.
annotators	AnnotatorRegistry	List of annotators that compute features.
scale	list[float] | None	How much to scale each dimension by, including time.
ndim	int	Number of dimensions (3 for 2D+time, 4 for 3D+time).

Initialize a Tracks object.

Parameters:

Name	Type	Description	Default
graph	DiGraph	NetworkX directed graph with nodes as detections and edges as links.	required
segmentation	ndarray | None	Optional segmentation array where labels match node IDs. Required for computing region properties (area, etc.).	None
time_attr	str | None	Graph attribute name for time. Defaults to "time" if None.	None
pos_attr	str | tuple[str, ...] | list[str] | None	Graph attribute name(s) for position. Can be: - Single string for one attribute containing position array - List/tuple of strings for multi-axis (one attribute per axis) Defaults to "pos" if None.	None
tracklet_attr	str | None	Graph attribute name for tracklet/track IDs. Defaults to "track_id" if None.	None
scale	list[float] | None	Scaling factors for each dimension (including time). If None, all dimensions scaled by 1.0.	None
ndim	int | None	Number of dimensions (3 for 2D+time, 4 for 3D+time). If None, inferred from segmentation or scale.	None
features	FeatureDict | None	Pre-built FeatureDict with feature definitions. If provided, time_attr/pos_attr/tracklet_attr are ignored. Assumes that all features in the dict already exist on the graph (will be activated but not recomputed). If None, core computed features (pos, area, track_id) are auto-detected by checking if they exist on the graph.	None

delete

delete(directory: Path)

Delete the tracks in the given directory. Also deletes the directory.

Parameters:

Name	Type	Description	Default
directory	Path	Directory containing tracks to be deleted	required

disable_features

disable_features(feature_keys: list[str]) -> None

Disable multiple features from computation.

Removes features from annotators and FeatureDict.

Parameters:

Name	Type	Description	Default
feature_keys	list[str]	List of feature keys to disable	required

Raises:

Type	Description
KeyError	If any feature is not available (raised by annotators)

enable_features

enable_features(
    feature_keys: list[str], recompute: bool = True
) -> None

Enable multiple features for computation efficiently.

Adds features to annotators and FeatureDict, optionally computes their values.

Parameters:

Name	Type	Description	Default
feature_keys	list[str]	List of feature keys to enable	required
recompute	bool	If True, compute feature values. If False, assume values already exist in graph and just register the feature.	True

Raises:

Type	Description
KeyError	If any feature is not available (raised by annotators)

get_area

get_area(node: Node) -> int | None

Get the area/volume of a given node. Raises a KeyError if the node is not in the graph. Returns None if the given node does not have an Area attribute.

.. deprecated:: 1.0 get_area will be removed in funtracks v2.0. Use get_node_attr(node, "area") instead.

Parameters:

Name	Type	Description	Default
node	Node	The node id to get the area/volume for	required

Returns:

Name	Type	Description
int	int | None	The area/volume of the node

get_areas

get_areas(nodes: Iterable[Node]) -> Sequence[int | None]

Get the area/volume of a given node. Raises a KeyError if the node is not in the graph. Returns None if the given node does not have an Area attribute.

.. deprecated:: 1.0 get_areas will be removed in funtracks v2.0. Use get_nodes_attr(nodes, "area") instead.

Parameters:

Name	Type	Description	Default
node	Node	The node id to get the area/volume for	required

Returns:

Name	Type	Description
int	Sequence[int | None]	The area/volume of the node

get_available_features

get_available_features() -> dict[str, Feature]

Get all features that can be computed across all annotators.

Returns:

Type	Description
dict[str, Feature]	Dictionary mapping feature keys to Feature definitions

get_iou

get_iou(edge: Edge)

Get the IoU value for the given edge.

.. deprecated:: 1.0 get_iou will be removed in funtracks v2.0. Use get_edge_attr(edge, "iou") instead.

Parameters:

Name	Type	Description	Default
edge	Edge	An edge to get the IoU value for.	required

Returns:

Type	Description
	The IoU value for the edge.

get_ious

get_ious(edges: Iterable[Edge])

Get the IoU values for the given edges.

.. deprecated:: 1.0 get_ious will be removed in funtracks v2.0. Use get_edges_attr(edges, "iou") instead.

Parameters:

Name	Type	Description	Default
edges	Iterable[Edge]	An iterable of edges to get IoU values for.	required

Returns:

Type	Description
	The IoU values for the edges.

get_pixels

get_pixels(node: Node) -> tuple[np.ndarray, ...] | None

Get the pixels corresponding to each node in the nodes list.

Parameters:

Name	Type	Description	Default
node	Node	A node to get the pixels for.	required

Returns:

Type	Description
tuple[ndarray, ...] | None	tuple[np.ndarray, ...] | None: A tuple representing the pixels for the input
tuple[ndarray, ...] | None	node, or None if the segmentation is None. The tuple will have length equal
tuple[ndarray, ...] | None	to the number of segmentation dimensions, and can be used to index the
tuple[ndarray, ...] | None	segmentation.

get_positions

get_positions(
    nodes: Iterable[Node], incl_time: bool = False
) -> np.ndarray

Get the positions of nodes in the graph. Optionally include the time frame as the first dimension. Raises an error if any of the nodes are not in the graph.

Parameters:

Name	Type	Description	Default
node	Iterable[Node]	The node ids in the graph to get the positions of	required
incl_time	bool	If true, include the time as the first element of each position array. Defaults to False.	False

Returns:

Type	Description
ndarray	np.ndarray: A N x ndim numpy array holding the positions, where N is the number of nodes passed in

get_time

get_time(node: Node) -> int

Get the time frame of a given node. Raises an error if the node is not in the graph.

Parameters:

Name	Type	Description	Default
node	Any	The node id to get the time frame for	required

Returns:

Name	Type	Description
int	int	The time frame that the node is in

load

load(
    directory: Path, seg_required=False, solution=False
) -> Tracks

Load a Tracks object from the given directory. Looks for files in the format generated by Tracks.save. Args: directory (Path): The directory containing tracks to load seg_required (bool, optional): If true, raises a FileNotFoundError if the segmentation file is not present in the directory. Defaults to False. Returns: Tracks: A tracks object loaded from the given directory

notify_annotators

notify_annotators(action: BasicAction) -> None

Notify annotators about an action so they can recompute affected features.

Delegates to the annotator registry which broadcasts to all annotators. The action contains all necessary information about which elements to update.

Parameters:

Name	Type	Description	Default
action	BasicAction	The action that triggered this notification	required

save

save(directory: Path)

Save the tracks to the given directory. Currently, saves the graph as a json file in networkx node link data format, saves the segmentation as a numpy npz file, and saves the time and position attributes and scale information in an attributes json file. Args: directory (Path): The directory to save the tracks in.

set_pixels

set_pixels(pixels: tuple[ndarray, ...], value: int) -> None

Set the given pixels in the segmentation to the given value.

Parameters:

Name	Type	Description	Default
pixels	Iterable[tuple[ndarray]]	The pixels that should be set, formatted like the output of np.nonzero (each element of the tuple represents one dimension, containing an array of indices in that dimension). Can be used to directly index the segmentation.	required
value	Iterable[int | None]	The value to set each pixel to	required

set_positions

set_positions(
    nodes: Iterable[Node],
    positions: ndarray,
    incl_time: bool = False,
)

Set the location of nodes in the graph. Optionally include the time frame as the first dimension. Raises an error if any of the nodes are not in the graph.

Parameters:

Name	Type	Description	Default
nodes	Iterable[node]	The node ids in the graph to set the location of.	required
positions	ndarray	An (ndim, num_nodes) shape array of positions to set.	required
incl_time	bool	If true, include the time as the first column of the position array. Defaults to False.	False

set_time

set_time(node: Any, time: int)

Set the time frame of a given node. Raises an error if the node is not in the graph.

Parameters:

Name	Type	Description	Default
node	Any	The node id to set the time frame for	required
time	int	The time to set	required

Architecture Diagrams

Class Diagram

classDiagram
    class Feature {
        <<TypedDict>>
        +feature_type: Literal
        +value_type: Literal
        +num_values: int
        +display_name: str|Sequence
        +required: bool
        +default_value: Any
    }

    class FeatureDict {
        +time_key: str
        +position_key: str|list|None
        +tracklet_key: str|None
        +node_features: dict
        +edge_features: dict
        +register_position_feature()
        +register_tracklet_feature()
    }

    class GraphAnnotator {
        <<abstract>>
        +tracks: Tracks
        +all_features: dict
        +features: dict
        +can_annotate()*
        +activate_features()
        +deactivate_features()
        +compute()*
        +update()*
    }

    class RegionpropsAnnotator {
        +pos_key: str
        +area_key: str
        +compute()
        +update()
    }

    class EdgeAnnotator {
        +iou_key: str
        +compute()
        +update()
    }

    class TrackAnnotator {
        +tracklet_key: str
        +lineage_key: str
        +tracklet_id_to_nodes: dict
        +lineage_id_to_nodes: dict
        +compute()
        +update()
    }

    class AnnotatorRegistry {
        +all_features: dict
        +features: dict
        +activate_features()
        +deactivate_features()
        +compute()
        +update()
    }

    class Tracks {
        +graph: nx.DiGraph
        +segmentation: ndarray|None
        +features: FeatureDict
        +annotators: AnnotatorRegistry
        +scale: list|None
        +ndim: int
        +enable_features()
        +disable_features()
        +get_available_features()
        +notify_annotators()
    }

    FeatureDict *-- Feature : stores many
    GraphAnnotator <|-- RegionpropsAnnotator : implements
    GraphAnnotator <|-- EdgeAnnotator : implements
    GraphAnnotator <|-- TrackAnnotator : implements
    AnnotatorRegistry o-- GraphAnnotator : aggregates many
    Tracks *-- FeatureDict : has one
    Tracks *-- AnnotatorRegistry : has one
    GraphAnnotator --> Tracks : references

Initialization Lifecycle

Here's what happens when you create a Tracks instance.

tracks = Tracks(graph, segmentation, ndim=3)

1. Basic Attribute Setup - save graph, segmentation, scale, etc. as instance variables
2. FeatureDict Creation - If features parameter is provided, use the provided FeatureDict and assume all features already exist on the graph. If features=None, create a FeatureDict with static features (time) and provided keys.
3. AnnotatorRegistry Creation - build an AnnotatorRegistry containing any Annotators that work on the provided tracks
4. Core Computed Features Setup - If features parameter provided, activate all computed features with keys in the features dictionary, so that updates will be computed. Does not compute any features from scratch. Otherwise, try to detect which core features are already present, activate those, and compute any missing ones from scratch.

Core Features

These features are automatically checked during initialization:

1. pos (position): Always auto-detected for RegionpropsAnnotator
2. area: Always auto-detected (for backward compatibility)
3. track_id (tracklet_id): Always auto-detected for TrackAnnotator

Example Scenarios

Scenario 1: Loading tracks from CSV with pre-computed features

# CSV has columns: id, time, y, x, area, track_id
graph = load_graph_from_csv(df)  # Nodes already have area, track_id
tracks = SolutionTracks(graph, segmentation=seg)
# Auto-detection: pos, area, track_id exist → activate without recomputing

Scenario 2: Creating tracks from raw segmentation

# Graph has no features yet
graph = nx.DiGraph()
graph.add_node(1, time=0)
tracks = Tracks(graph, segmentation=seg)
# Auto-detection: pos, area don't exist → compute them

Scenario 3: Explicit feature control with FeatureDict

# Bypass auto-detection entirely
feature_dict = FeatureDict({"t": Time(), "pos": Position(), "area": Area()})
tracks = Tracks(graph, segmentation=seg, features=feature_dict)
# All features in feature_dict are activated, none are computed

Scenario 4: Enable a new feature

tracks = Tracks(graph, segmentation)
# Initially has: time, pos, area (auto-detected or computed)

tracks.enable_features(["iou", "circularity"])
# Now has: time, pos, area, iou, circularity

# Check active features
print(tracks.features.keys())  # All features in FeatureDict (including static)
print(tracks.annotators.features.keys())  # Only active computed features

Scenario 4: Disable a feature

tracks.disable_features(["area"])
# Removes from FeatureDict, deactivates in annotators
# Note: Doesn't delete values from graph, just stops computing/updating

Extending the System

Creating a New Annotator

1. Subclass GraphAnnotator:

   from funtracks.annotators import GraphAnnotator
   
   class MyCustomAnnotator(GraphAnnotator):
       @classmethod
       def can_annotate(cls, tracks):
           # Check if this annotator can handle these tracks
           return tracks.some_condition
   
       def __init__(self, tracks, custom_key="custom"):
           super().__init__(tracks)
           self.custom_key = custom_key
   
           # Register features
           self.all_features[custom_key] = (CustomFeature(), False)
   
       def compute(self, feature_keys=None):
           # Compute feature values in bulk
           if "custom" in self.features:
               for node in self.tracks.graph.nodes():
                   value = self._compute_custom(node)
                   self.tracks.graph.nodes[node]["custom"] = value
   
       def update(self, action):
           # Incremental update when graph changes
           if "custom" in self.features:
               if isinstance(action, SomeActionType):
                   # Recompute only for affected nodes
                   pass
   

2. Register in Tracks._get_annotators():

   if MyCustomAnnotator.can_annotate(tracks):
       ann = MyCustomAnnotator(tracks)
       tracks.annotators.append(ann)
       tracks.enable_features([key for key in ann.all_features()])