Features

Funtracks has a feature computation system that manages attributes (features) from tracking graphs with optional segmentation data. The architecture separates:

Feature metadata (what features exist and their properties) - Feature class
Feature computation (how to calculate feature values) - GraphAnnotator class
Feature storage (where feature values live on the graph) - attributes on the graph nodes/edges
Feature lifecycle (when to compute, activate, or update features) - computation called by Tracks, updates triggered by BasicActions

Core Components

1. Feature (TypedDict)

A Feature is a TypedDict that stores metadata about a graph feature.

Show API documentation

Bases: TypedDict

TypedDict for storing metadata associated with a graph feature.

Use factory functions like Time(), Position(), Area() etc. to create features with standard defaults.

The key is stored separately in the FeatureDict mapping (not in the Feature itself).

Attributes:

Name	Type	Description
`feature_type`	`Literal['node', 'edge']`	Specifies which graph elements the feature applies to.
`value_type`	`ValueType`	The data type of the feature values.
`num_values`	`int`	The number of values expected for this feature.
`display_name`	`str`	Optional. A display name for the feature. If not provided, the feature key is used.
`value_names`	`Sequence[str]`	Optional. Individual display names for each value when num_values > 1. Length should match num_values.
`default_value`	`Any`	This value is returned whenever the feature value is missing on the graph.
`spatial_dims`	`bool`	Optional. If True, num_values must match the number of spatial dimensions (e.g., 2 for 2D, 3 for 3D). Used for features like Position and EllipsoidAxes.

2. FeatureDict

A FeatureDict is a dictionary (dict[str, Feature]) with special tracking for important feature keys:

Show API documentation

Bases: dict[str, Feature]

A dictionary mapping keys to Features, with special tracking for time/position.

Inherits from dict[str, Feature], so can be used directly as a dictionary. Provides convenient access to time and position features through properties.

Attributes:

Name	Type	Description
`time_key`	`str`	The key used for the time feature
`position_key`	`str \| list[str] \| None`	The key(s) used for position feature(s)

Parameters:

Name	Type	Description	Default
`features`	`dict[str, Feature]`	Mapping from feature keys to Features	required
`time_key`	`str`	The key for the time feature (must be in features)	required
`position_key`	`str \| list[str] \| None`	The key(s) for position feature(s)	required
`tracklet_key`	`str \| None`	The key for the tracklet feature	required
`lineage_key`	`str \| None`	The key for the lineage feature	`None`

edge_features

edge_features: dict[str, Feature]

A dict of all edge features

node_features

node_features: dict[str, Feature]

A dict of all node features

dump_json

dump_json() -> dict

Dump this FeatureDict to a json compatible dictionary

Returns:

Name	Type	Description
`dict`	`dict`	A map from the key "FeatureDict" containing features, time_key, position_key, tracklet_key, and lineage_key

from_json

from_json(json_dict: dict) -> FeatureDict

Generate a FeatureDict from a json dict such as one generated by dump_json

Parameters:

Name	Type	Description	Default
`json_dict`	`dict`	A dictionary with the key "FeatureDict" containing features, time_key, position_key, tracklet_key, and lineage_key	required

Returns:

Name	Type	Description
`FeatureDict`	`FeatureDict`	A FeatureDict object containing the features from the dictionary

register_lineage_feature

register_lineage_feature(
    key: str, feature: Feature
) -> None

Register the lineage feature and set the lineage_key.

Parameters:

Name	Type	Description	Default
`key`	`str`	The key to use for the lineage feature	required
`feature`	`Feature`	The Feature to register	required

register_position_feature

register_position_feature(
    key: str, feature: Feature
) -> None

Register the position feature and set the position_key.

Parameters:

Name	Type	Description	Default
`key`	`str`	The key to use for the position feature	required
`feature`	`Feature`	The Feature to register	required

register_tracklet_feature

register_tracklet_feature(
    key: str, feature: Feature
) -> None

Register the tracklet/track_id feature and set the tracklet_key.

Parameters:

Name	Type	Description	Default
`key`	`str`	The key to use for the tracklet feature	required
`feature`	`Feature`	The Feature to register	required

3. GraphAnnotator (Base Class)

An abstract base class for components that compute and update features on a graph.

Show API documentation

A base class for adding and updating graph features.

This class holds a set of features that it is responsible for. The annotator will compute these features and add them to the Tracks initially, and update them when necessary. The set of features will all be computed and updated together, although individual ones can be removed for efficiency.

Parameters:

Name	Type	Description	Default
`tracks`	`Tracks`	The tracks to manage features for.	required
`features`	`dict[str, Feature]`	A dict mapping keys to features that this annotator is capable of computing and updating.	required

Attributes:

Name	Type	Description
`all_features`	`dict[str, tuple[Feature, bool]]`	Maps feature keys to (feature, is_included) tuples. Tracks both what can be computed and what is currently being computed. Defaults to computing nothing.

features

features: dict[str, Feature]

The dict of features that this annotator currently manages.

Filtered from all_features based on inclusion flags.

activate_features

activate_features(keys: list[str]) -> None

Activate computation of the given features in the annotation process.

Filters the list to only features this annotator owns, ignoring others.

Parameters:

Name	Type	Description	Default
`keys`	`list[str]`	List of feature keys to activate. Only keys in all_features are activated.	required

can_annotate

can_annotate(tracks: Tracks) -> bool

Check if this annotator can annotate the given tracks.

Subclasses should override this method to specify their requirements (e.g., segmentation, SolutionTracks, etc.).

Parameters:

Name	Type	Description	Default
`tracks`	`Tracks`	The tracks to check compatibility with	required

Returns:

Type	Description
`bool`	True if the annotator can annotate these tracks, False otherwise

change_key

change_key(old_key: str, new_key: str) -> None

Rename a feature key in this annotator.

Base implementation updates the all_features dictionary. Subclasses should override this method to update any additional internal mappings they maintain.

Parameters:

Name	Type	Description	Default
`old_key`	`str`	Existing key to rename.	required
`new_key`	`str`	New key to replace it with.	required

Raises:

Type	Description
`KeyError`	If old_key does not exist in all_features.

compute

compute(feature_keys: list[str] | None = None) -> None

Compute a set of features and add them to the tracks.

This involves both updating the node/edge attributes on the tracks.graph and adding the features to the FeatureDict, if necessary. This is distinct from update to allow more efficient bulk computation of features.

Parameters:

Name	Type	Description	Default
`feature_keys`	`list[str] \| None`	Optional list of specific feature keys to compute. If None, computes all currently active features. Any provided keys not in the currently active set are ignored.	`None`

Raises:

Type	Description
`NotImplementedError`	If not implemented in subclass.

deactivate_features

deactivate_features(keys: list[str]) -> None

Deactivate computation of the given features in the annotation process.

Filters the list to only features this annotator owns, ignoring others.

Parameters:

Name	Type	Description	Default
`keys`	`list[str]`	List of feature keys to deactivate. Only keys in all_features are deactivated.	required

update

update(action: BasicAction) -> None

Update a set of features based on the given action.

This involves both updating the node or edge attributes on the tracks.graph and adding the features to the FeatureDict, if necessary. This is distinct from compute to allow more efficient computation of features for single elements.

The action contains all necessary information about which elements to update (e.g., AddNode.node, AddEdge.edge, UpdateNodeSeg.node).

Parameters:

Name	Type	Description	Default
`action`	`BasicAction`	The action that triggered this update	required

Raises:

Type	Description
`NotImplementedError`	If not implemented in subclass.

4. GraphAnnotator Implementations

Annotator	Purpose	Requirements	Features Computed	API Reference
RegionpropsAnnotator	Extracts node features from segmentation using scikit-image's `regionprops`	`segmentation` must not be `None`	`pos`, `area`, `ellipse_axis_radii`, `circularity`, `perimeter`	📚 API
EdgeAnnotator	Computes edge features based on segmentation overlap between consecutive time frames	`segmentation` must not be `None`	`iou` (Intersection over Union)	📚 API
TrackAnnotator	Computes tracklet and lineage IDs for SolutionTracks	Must be used with `SolutionTracks` (binary tree structure)	`tracklet_id`, `lineage_id`	📚 API

5. AnnotatorRegistry

A registry that manages multiple GraphAnnotator instances with a unified interface. Extends list[GraphAnnotator].

Show API documentation

Bases: list[GraphAnnotator]

A list of annotators with coordinated operations.

Inherits from list[GraphAnnotator], so can be used directly as a list. Provides coordinated compute/update/enable/disable operations across all annotators.

Example

annotators = AnnotatorRegistry([ RegionpropsAnnotator(tracks, pos_key="centroid"), EdgeAnnotator(tracks), TrackAnnotator(tracks, tracklet_key="track_id"), ])

Can use as a list

annotators.append(MyCustomAnnotator(tracks))

Coordinated operations

annotators.activate_features(["area", "iou"]) annotators.compute()

Initialize with a list of annotators.

Parameters:

Name	Type	Description	Default
`annotators`	`list[GraphAnnotator]`	List of instantiated annotator objects	required

all_features

all_features: dict[str, tuple[Feature, bool]]

Dynamically aggregate all_features from all annotators.

Returns:

Type	Description
`dict[str, tuple[Feature, bool]]`	Dictionary mapping feature keys to (Feature, is_enabled) tuples

features

features: dict[str, Feature]

Get all currently active features from all annotators.

Returns:

Type	Description
`dict[str, Feature]`	Dictionary mapping feature keys to Feature definitions (only active features)

activate_features

activate_features(keys: list[str]) -> None

Activate features across all annotators.

Parameters:

Name	Type	Description	Default
`keys`	`list[str]`	List of feature keys to activate	required

Raises:

Type	Description
`KeyError`	If any feature keys are not available

change_key

change_key(old_key: str, new_key: str) -> None

Rename a feature key across all annotators.

Finds the annotator that owns the feature and calls its change_key method. This allows the Tracks user to rename features without needing to find the specific annotator that manages it.

Parameters:

Name	Type	Description	Default
`old_key`	`str`	Existing key to rename.	required
`new_key`	`str`	New key to replace it with.	required

Raises:

Type	Description
`KeyError`	If old_key does not exist in any annotator.

compute

compute(feature_keys: list[str] | None = None) -> None

Compute features across all annotators.

Parameters:

Name	Type	Description	Default
`feature_keys`	`list[str] \| None`	Optional list of specific feature keys to compute. If None, computes all currently active features.	`None`

deactivate_features

deactivate_features(keys: list[str]) -> None

Deactivate features across all annotators.

Parameters:

Name	Type	Description	Default
`keys`	`list[str]`	List of feature keys to deactivate	required

Raises:

Type	Description
`KeyError`	If any feature keys are not available

update

update(action: BasicAction) -> None

Update features across all annotators based on the action.

Parameters:

Name	Type	Description	Default
`action`	`BasicAction`	The action that triggered this update	required

6. Tracks

The main class representing a set of tracks: a graph + optional segmentation + features.

Show API documentation

A set of tracks consisting of a graph and an optional segmentation.

The graph nodes represent detections and must have a time attribute and position attribute. Edges in the graph represent links across time.

Attributes:

Name	Type	Description
`graph`	`GraphView`	A graph with nodes representing detections and and edges representing links across time.
`features`	`FeatureDict`	Dictionary of features tracked on graph nodes/edges.
`annotators`	`AnnotatorRegistry`	List of annotators that compute features.
`scale`	`list[float] \| None`	How much to scale each dimension by, including time.
`ndim`	`int`	Number of dimensions (3 for 2D+time, 4 for 3D+time).

Initialize a Tracks object.

Parameters:

Name	Type	Description	Default
`graph`	`GraphView`	tracksdata directed graph with nodes as detections and edges as links.	required
`time_attr`	`str \| None`	Graph attribute name for time. Defaults to "time" if None.	`None`
`pos_attr`	`str \| tuple[str, ...] \| list[str] \| None`	Graph attribute name(s) for position. Can be: - Single string for one attribute containing position array - List/tuple of strings for multi-axis (one attribute per axis) Defaults to "pos" if None.	`None`
`tracklet_attr`	`str \| None`	Graph attribute name for tracklet/track IDs. Defaults to "track_id" if None.	`None`
`lineage_attr`	`str \| None`	Graph attribute name for lineage IDs. Defaults to "lineage_id" if None.	`None`
`scale`	`list[float] \| None`	Scaling factors for each dimension (including time). If None, all dimensions scaled by 1.0.	`None`
`ndim`	`int \| None`	Number of dimensions (3 for 2D+time, 4 for 3D+time). If None, inferred from segmentation or scale.	`None`
`features`	`FeatureDict \| None`	Pre-built FeatureDict with feature definitions. If provided, time_attr/pos_attr/tracklet_attr are ignored. Assumes that all features in the dict already exist on the graph (will be activated but not recomputed). If None, core computed features (pos, area, track_id) are auto-detected by checking if they exist on the graph.	`None`
`_segmentation`	`GraphArrayView \| None`	Internal parameter for reusing an existing GraphArrayView instance. Not intended for public use.	`None`

add_feature

add_feature(key: str, feature: Feature) -> None

Add a feature to the features dictionary and perform graph operations.

This is the preferred way to add new features as it ensures both the features dictionary is updated and any necessary graph operations are performed.

Parameters:

Name	Type	Description	Default
`key`	`str`	The key for the new feature	required
`feature`	`Feature`	The Feature object to add	required

delete_feature

delete_feature(key: str) -> None

Delete a feature from the features dictionary and perform graph operations.

This is the preferred way to delete features as it ensures both the features dictionary is updated and any necessary graph operations are performed.

Parameters:

Name	Type	Description	Default
`key`	`str`	The key for the feature to delete	required

disable_features

disable_features(feature_keys: list[str]) -> None

Disable multiple features from computation.

Removes features from annotators and FeatureDict.

Parameters:

Name	Type	Description	Default
`feature_keys`	`list[str]`	List of feature keys to disable	required

Raises:

Type	Description
`KeyError`	If any feature is not available (raised by annotators)

enable_features

enable_features(
    feature_keys: list[str], recompute: bool = True
) -> None

Enable multiple features for computation efficiently.

Adds features to annotators and FeatureDict, optionally computes their values.

Parameters:

Name	Type	Description	Default
`feature_keys`	`list[str]`	List of feature keys to enable	required
`recompute`	`bool`	If True, compute feature values. If False, assume values already exist in graph and just register the feature.	`True`

Raises:

Type	Description
`KeyError`	If any feature is not available (raised by annotators)

get_available_features

get_available_features() -> dict[str, Feature]

Get all features that can be computed across all annotators.

Returns:

Type	Description
`dict[str, Feature]`	Dictionary mapping feature keys to Feature definitions

get_mask

get_mask(node: Node) -> Mask | None

Get the segmentation mask associated with a given node.

.. deprecated:: 1.0 set_time will be removed in funtracks v2.0. Use update_node_attrs([node], {tracks.features.time_key: [time]}) instead.

Parameters:

Name	Type	Description	Default
`node`	`Node`	The node to get the mask for.	required

Returns:

Type	Description
`Mask \| None`	Mask \| None: The segmentation mask for the node, or None if no
`Mask \| None`	segmentation is available.

get_pixels

get_pixels(node: Node) -> tuple[np.ndarray, ...] | None

Get the pixels corresponding to each node in the nodes list.

Parameters:

Name	Type	Description	Default
`node`	`Node`	A node to get the pixels for.	required

Returns:

Type	Description
`tuple[ndarray, ...] \| None`	tuple[np.ndarray, ...] \| None: A tuple representing the pixels for the input
`tuple[ndarray, ...] \| None`	node, or None if the segmentation is None. The tuple will have length equal
`tuple[ndarray, ...] \| None`	to the number of segmentation dimensions, and can be used to index the
`tuple[ndarray, ...] \| None`	segmentation.

get_position

get_position(node: Node, incl_time=False) -> list

Get position of a single node. Uses a direct per-node query — do not use in a loop over many nodes; use get_positions() instead.

get_positions

get_positions(
    nodes: Iterable[Node], incl_time: bool = False
) -> np.ndarray

Get the positions of nodes in the graph. Optionally include the time frame as the first dimension. Raises an error if any of the nodes are not in the graph.

NOTE: fetches all nodes in the graph internally. Optimised for bulk use. For a single node use get_position() instead.

Parameters:

Name	Type	Description	Default
`node`	`Iterable[Node]`	The node ids in the graph to get the positions of	required
`incl_time`	`bool`	If true, include the time as the first element of each position array. Defaults to False.	`False`

Returns:

Type	Description
`ndarray`	np.ndarray: A N x ndim numpy array holding the positions, where N is the number of nodes passed in

get_time

get_time(node: Node) -> int

Get the time frame of a given node. Raises an error if the node is not in the graph.

Parameters:

Name	Type	Description	Default
`node`	`Any`	The node id to get the time frame for	required

Returns:

Name	Type	Description
`int`	`int`	The time frame that the node is in

get_times

get_times(nodes: Iterable[Node]) -> Sequence[int]

Batch fetch times for many nodes in one query. NOTE: fetches all nodes in the graph internally. Optimised for bulk use. For a single node use get_time() instead.

in_degree

in_degree(nodes: ndarray | None = None) -> np.ndarray

Get the in-degree edge_ids of the nodes in the graph.

notify_annotators

notify_annotators(action: BasicAction) -> None

Notify annotators about an action so they can recompute affected features.

Delegates to the annotator registry which broadcasts to all annotators. The action contains all necessary information about which elements to update.

Parameters:

Name	Type	Description	Default
`action`	`BasicAction`	The action that triggered this notification	required

redo

redo() -> bool

Redo the last undone action from the action history.

Returns:

Name	Type	Description
`bool`	`bool`	True if an action was redone, False if there were no actions to redo

set_positions

set_positions(nodes: Iterable[Node], positions: ndarray)

Set the location of nodes in the graph. Optionally include the time frame as the first dimension. Raises an error if any of the nodes are not in the graph.

Parameters:

Name	Type	Description	Default
`nodes`	`Iterable[node]`	The node ids in the graph to set the location of.	required
`positions`	`ndarray`	An (ndim, num_nodes) shape array of positions to set.	required
`incl_time`	`bool`	If true, time is the first column and is included in ndim. Defaults to False.	required

undo

undo() -> bool

Undo the last performed action from the action history.

Returns:

Name	Type	Description
`bool`	`bool`	True if an action was undone, False if there were no actions to undo

Architecture Diagrams

Class Diagram

classDiagram
    class Feature {
        <<TypedDict>>
        +feature_type: Literal
        +value_type: Literal
        +num_values: int
        +display_name: str|Sequence
        +default_value: Any
    }

    class FeatureDict {
        +time_key: str
        +position_key: str|list|None
        +tracklet_key: str|None
        +node_features: dict
        +edge_features: dict
        +register_position_feature()
        +register_tracklet_feature()
    }

    class GraphAnnotator {
        <<abstract>>
        +tracks: Tracks
        +all_features: dict
        +features: dict
        +can_annotate()*
        +activate_features()
        +deactivate_features()
        +compute()*
        +update()*
    }

    class RegionpropsAnnotator {
        +pos_key: str
        +area_key: str
        +compute()
        +update()
    }

    class EdgeAnnotator {
        +iou_key: str
        +compute()
        +update()
    }

    class TrackAnnotator {
        +tracklet_key: str
        +lineage_key: str
        +tracklet_id_to_nodes: dict
        +lineage_id_to_nodes: dict
        +compute()
        +update()
    }

    class AnnotatorRegistry {
        +all_features: dict
        +features: dict
        +activate_features()
        +deactivate_features()
        +compute()
        +update()
    }

    class Tracks {
        +graph: td.graph.GraphView
        +segmentation: ndarray|None
        +features: FeatureDict
        +annotators: AnnotatorRegistry
        +scale: list|None
        +ndim: int
        +enable_features()
        +disable_features()
        +get_available_features()
        +notify_annotators()
    }

    FeatureDict *-- Feature : stores many
    GraphAnnotator <|-- RegionpropsAnnotator : implements
    GraphAnnotator <|-- EdgeAnnotator : implements
    GraphAnnotator <|-- TrackAnnotator : implements
    AnnotatorRegistry o-- GraphAnnotator : aggregates many
    Tracks *-- FeatureDict : has one
    Tracks *-- AnnotatorRegistry : has one
    GraphAnnotator --> Tracks : references

Initialization Lifecycle

Here's what happens when you create a Tracks instance.

tracks = Tracks(graph, segmentation, ndim=3)

Basic Attribute Setup - save graph, segmentation, scale, etc. as instance variables
FeatureDict Creation - If features parameter is provided, use the provided FeatureDict and assume all features already exist on the graph. If features=None, create a FeatureDict with static features (time) and provided keys.
AnnotatorRegistry Creation - build an AnnotatorRegistry containing any Annotators that work on the provided tracks
Core Computed Features Setup - If features parameter provided, activate all computed features with keys in the features dictionary, so that updates will be computed. Does not compute any features from scratch. Otherwise, try to detect which core features are already present, activate those, and compute any missing ones from scratch.

Core Features

These features are automatically checked during initialization:

pos (position): Always auto-detected for RegionpropsAnnotator
area: Always auto-detected (for backward compatibility)
track_id (tracklet_id): Always auto-detected for TrackAnnotator

Example Scenarios

Scenario 1: Loading tracks from CSV with pre-computed features

from funtracks.import_export import tracks_from_df

# CSV/DataFrame has columns: id, time, y, x, area, track_id, parent_id
tracks = tracks_from_df(df, segmentation=seg)
# Auto-detection: pos, area, track_id exist → activate without recomputing

Scenario 2: Creating tracks from raw segmentation

from funtracks.utils import create_empty_graphview_graph
from funtracks.data_model import Tracks

# Create empty graph and add nodes
graph = create_empty_graphview_graph()
graph.add_node(index=1, attrs={"t": 0})
tracks = Tracks(graph, segmentation=seg)
# Auto-detection: pos, area don't exist → compute them from segmentation

Scenario 3: Explicit feature control with FeatureDict

from funtracks.features import FeatureDict, Time, Position, Area
from funtracks.data_model import Tracks

# Bypass auto-detection entirely
feature_dict = FeatureDict({"t": Time(), "pos": Position(), "area": Area()})
tracks = Tracks(graph, segmentation=seg, features=feature_dict)
# All features in feature_dict are activated, none are computed

Scenario 4: Enable a new feature

from funtracks.data_model import Tracks

tracks = Tracks(graph, segmentation)
# Initially has: time, pos, area (auto-detected or computed)

tracks.enable_features(["iou", "circularity"])
# Now has: time, pos, area, iou, circularity

# Check active features
print(tracks.features.keys())  # All features in FeatureDict (including static)
print(tracks.annotators.features.keys())  # Only active computed features

Scenario 5: Disable a feature

tracks.disable_features(["area"])
# Removes from FeatureDict, deactivates in annotators
# Note: Doesn't delete values from graph, just stops computing/updating

Extending the System

Creating a New Annotator

Subclass GraphAnnotator:

from funtracks.annotators import GraphAnnotator

class MyCustomAnnotator(GraphAnnotator):
    @classmethod
    def can_annotate(cls, tracks):
        # Check if this annotator can handle these tracks
        return tracks.some_condition

    def __init__(self, tracks, custom_key="custom"):
        super().__init__(tracks)
        self.custom_key = custom_key

        # Register features
        self.all_features[custom_key] = (CustomFeature(), False)

    def compute(self, feature_keys=None):
        # Compute feature values in bulk
        if "custom" in self.features:
            for node in self.tracks.graph.node_ids():
                value = self._compute_custom(node)
                self.tracks[node]["custom"] = value

    def update(self, action):
        # Incremental update when graph changes
        if "custom" in self.features:
            if isinstance(action, SomeActionType):
                # Recompute only for affected nodes
                pass

Register in Tracks._get_annotators():

if MyCustomAnnotator.can_annotate(tracks):
    ann = MyCustomAnnotator(tracks)
    tracks.annotators.append(ann)
    tracks.enable_features([key for key in ann.all_features()])