@layer-drone/image-matching

A TypeScript library for matching drone flight images with their provenance data. This package matches captured drone images (with EXIF metadata) to their intended flight path data, enabling validation of drone imagery.

Overview

The ImageMatcher class processes drone flight provenance data and matches it with actual captured images. It supports multiple DJI drone models and handles different filename formats and path structures.

Key Features:

• Match drone images to flight path actions using EXIF data
• Support for DJI Mini 2, 3, 3 Pro, and 4 Pro
• File size-based disambiguation for multiple matches
• Export matches as structured data

Matching Steps

1. Parse Flight Plan - Extract features (products) and their actions from FlightPathIntended
2. Match Controller Logs - Link ControllerRequestLog and ControllerResponseLog entries by tag and action index
3. Generate Paths - Convert DJI file indices to expected file paths based on drone model
4. Create Match Entries - Combine intended actions with actual controller logs
5. Associate EXIF Data - Match captured image metadata to specific response logs using filename patterns and file sizes

Installation

pnpm add @layer-drone/image-matching

Usage

Basic Example

import { createMatcher } from "@layer-drone/image-matching";

// Load your provenance data
const provenanceData = {
  FlightPathIntended: {
    type: "FeatureCollection",
    features: [...],
    metadata: {
      flightPlanType: "panorama",
      flightPlanId: 1,
      zoneHash: "abc123",
      imageCount: 17
    }
  },
  ControllerResponseLogs: [...],
  // ... other optional data
};

// Create matcher for DJI Mini 3 Pro
const matcher = createMatcher(provenanceData, "3");

// Get all matches
const matches = matcher.getMatches();

// Match images with EXIF data
const exifData = [
  {
    filename: "DJI_0851.JPG",
    fileSize: 5077938,
    location: { lat: 41.840082, lon: -71.415057, alt: 82.8 },
    timeOfCreation: "2025-10-03T14:12:54Z"
  }
];

const { unmatchedExifs } = matcher.setExifData(exifData);
console.log(`Unmatched: ${unmatchedExifs.length}`);

// Find a specific match
const result = matcher.getMatchFromExifData(exifData[0]);
if (result) {
  const { matchEntry, responseLogIndex } = result;
  console.log(`Matched to ${matchEntry.responseLog[responseLogIndex].path}`);
}

API Reference

ImageMatcher

Main class for performing image matching operations.

Constructor

new ImageMatcher(provenanceData: ProvenanceData, droneModel: "1" | "2" | "3" | "4")

Parameters:

• provenanceData - Flight provenance data containing flight path and controller logs
• droneModel - DJI drone model:
  • "1" - DJI Mini 2
  • "2" - DJI Mini 3
  • "3" - DJI Mini 3 Pro
  • "4" - DJI Mini 4 Pro

Methods

getDroneModel(): { droneModel: DroneModelType }

Returns the drone model type.

const { droneModel } = matcher.getDroneModel();
// => { droneModel: "DJI_MINI_3_PRO" }

matchProvenanceData(): Map<string, MatchEntry>

Processes provenance data and creates matches between flight actions and controller response logs. Called automatically in constructor.

getMatches(): Map<string, MatchEntry> | undefined

Returns all matched entries. Each key is a unique identifier like "pano-1.0" (product-action).

setExifData(exifData: ExifData[]): { unmatchedExifs: ExifData[] }

Matches and associates EXIF data from captured images to their corresponding response logs in provenance entries. Returns a list of EXIF entries that could not be matched.

const { unmatchedExifs } = matcher.setExifData([
  {
    filename: "DJI_0851.JPG",
    fileSize: 5077938,
    location: { lat: 41.840082, lon: -71.415057, alt: 82.8 },
    timeOfCreation: "2025-10-03T14:12:54Z",
  },
]);

if (unmatchedExifs.length > 0) {
  console.log(`Failed to match ${unmatchedExifs.length} images`);
}

getMatchFromExifData(exifData: ExifData): { matchEntry: MatchEntry; responseLogIndex: number } | undefined

Finds a specific response log match for the given EXIF data. Returns an object containing the matched entry and the index of the response log, or undefined if no match or ambiguous match.

const result = matcher.getMatchFromExifData(exifData);
if (result) {
  const { matchEntry, responseLogIndex } = result;
  const responseLog = matchEntry.responseLog[responseLogIndex];
  console.log(`Matched to: ${responseLog.path}`);
  console.log(`File size: ${responseLog.fileSize}`);
}

Matching Logic:

• DJI Mini 4 Pro: Matches pattern DJI_*_XXXX_D.JPG by extracting the 4-digit number
• Other Models: Matches DJI_XXXX.JPG pattern, handles timestamp prefixes
• Uses fileSize as tiebreaker when multiple response logs match the same pattern

getResponseLogsWithoutExif(): ControllerResponseLogWithPath[]

Returns response logs from provenance data that have no associated EXIF data.

getZoneHash(): string

Returns the zone hash from the flight path metadata.

static djiIndexToPath(index: number, droneModel: keyof typeof DroneModel): string

Converts DJI file index to expected file path format. Can be called without instantiating the class. The droneModel parameter must be one of: "1", "2", "3", or "4".

getFilePaths(): string[]

Returns all file paths from matched entries.

getActionsWithoutMatches(): Map<string, MatchEntry>

Returns actions that have no controller response logs.

getActionsWithSingleMatch(): Map<string, MatchEntry>

Returns actions that have exactly one controller response log.

getActionsWithMultipleMatches(): Map<string, MatchEntry>

Returns actions that have multiple controller response logs.

getProvenanceData(): ProvenanceData

Returns the complete provenance data used by the matcher.

createMatcher(provenanceData: ProvenanceData, droneModel: "1" | "2" | "3" | "4", exifData: ExifData[]): ImageMatcher

Factory function to create a new ImageMatcher instance.

Data Structures

Tags

Each entry has an ID composed of three parts: the intended product name, a sequential ID referring to the action's location, and (for actions requiring multiple images, like a pano) a local index number within that action.

<product-type>-<secuence-id>.<local-id>

MatchEntry

Represents a matched flight action with its associated data.

interface MatchEntry {
  intendedAction: {
    productType: string; // e.g., "pano", "map", "gridMap"
    productId: number; // Product identifier
    geom?: Geometry; // GeoJSON geometry of the action point
    action: FlightAction; // Flight action details (type, yawAngle, gimbalPitchAngle)
  };
  responseLog: ControllerResponseLogWithPath[] | undefined; // Controller responses with file paths and EXIF data
  requestLog: ControllerRequestLog | undefined; // Controller request log
}

interface ControllerResponseLogWithPath extends ControllerResponseLog {
  path: string; // Generated file path based on DJI index
  exifData?: ExifData; // Associated EXIF data from captured image (if matched)
}

Example:

{
  intendedAction: {
    productType: "pano",
    productId: 1,
    geom: {
      type: "Point",
      coordinates: [-71.415057, 41.840082, 82.8]
    },
    action: {
      type: "photo",
      yawAngle: 0,
      gimbalPitchAngle: -90
    }
  },
  responseLog: [
    {
      upLinkQuality: 100,
      timeCreated: 1696348374000,
      tag: "pano-1",
      mediaType: "photo",
      index: 6815745,
      fileSize: 5077938,
      path: "/DCIM/104MEDIA/DJI_0851.JPG",
      gimbalAttitude: { yaw: 0, roll: 0, pitch: -90 },
      aircraftLocation: { longitude: -71.415057, latitude: 41.840082, altitude: 82.8 },
      aircraftAttitude: { yaw: 0, roll: 0, pitch: 0 },
      actionIndex: 0,
      exifData: {
        filename: "DJI_0851.JPG",
        fileSize: 5077938,
        location: { lat: 41.840082, lon: -71.415057, alt: 82.8 },
        timeOfCreation: "2025-10-03T14:12:54Z"
      }
      // ... other fields
    }
  ],
  requestLog: {
    tag: "pano-1",
    actionIndex: 0,
    type: "photo",
    // ... other fields
  }
}

ProvenanceData

Flight provenance data structure containing all information about the intended flight and captured data.

interface ProvenanceData {
  FlightPathIntended: {
    type: "FeatureCollection";
    features: Array<{
      type: "Feature";
      properties: {
        actions: FlightAction[]; // Actions to perform at this point
        desiredFlyingHeightAboveGround: number;
        flyingHeightRelativeToTakeOff: number;
        takeOffOffset: number;
        flightSpeedMs: number;
        isFlyThrough: boolean;
        isFlightLineEnd: boolean;
        isFlightLineStart: boolean;
        absoluteAltitudeAtGroundLevel: number;
        takeOffCoordinates: Coordinates;
        intendedAltitudePoints?: IntendedAltitudePoint[];
      };
      geometry: {
        type: "Point";
        coordinates: [number, number, number]; // [lon, lat, alt]
      };
    }>;
    metadata: {
      flightPlanType: string; // e.g., "panorama", "map", "gridMap"
      flightPlanId: number;
      zoneHash: string;
      imageCount: number;
    };
  };
  ImageList?: ImageList; // Optional image metadata
  FlightMetadata?: FlightMetadata; // Optional flight metadata
  ControllerRequestLogs?: ControllerRequestLog[]; // Optional request logs
  ControllerResponseLogs?: ControllerResponseLog[]; // Optional response logs
}

Example:

{
  FlightPathIntended: {
    type: "FeatureCollection",
    features: [
      {
        type: "Feature",
        properties: {
          actions: [
            {
              type: "photo",
              yawAngle: 0,
              gimbalPitchAngle: -90
            }
          ],
          desiredFlyingHeightAboveGround: 50,
          flyingHeightRelativeToTakeOff: 50,
          takeOffOffset: 0,
          flightSpeedMs: 5,
          isFlyThrough: false,
          isFlightLineEnd: false,
          isFlightLineStart: false,
          absoluteAltitudeAtGroundLevel: 100,
          takeOffCoordinates: {
            longitude: -71.415057,
            latitude: 41.840082,
            altitude: 100
          }
        },
        geometry: {
          type: "Point",
          coordinates: [-71.415057, 41.840082, 150]
        }
      }
    ],
    metadata: {
      flightPlanType: "panorama",
      flightPlanId: 1,
      zoneHash: "abc123def456",
      imageCount: 17
    }
  },
  ControllerResponseLogs: [
    {
      upLinkQuality: 100,
      timeCreated: 1696348374000,
      tag: "pano-1",
      mediaType: "photo",
      index: 6815745,
      fileSize: 5077938,
      // ... other fields
    }
  ]
}

ExifData

EXIF metadata extracted from captured images.

interface ExifData {
  location: {
    lat: number; // Latitude in decimal degrees
    lon: number; // Longitude in decimal degrees
    alt: number; // Altitude in meters
  };
  timeOfCreation: string; // ISO 8601 timestamp
  fileSize: number; // File size in bytes
  filename: string; // Original filename
}

Example:

{
  location: {
    lat: 41.840082361111115,
    lon: -71.41505761111112,
    alt: 82.8
  },
  timeOfCreation: "2025-10-03T14:12:54Z",
  fileSize: 5077938,
  filename: "DJI_0851.JPG"  // or "1759531051938_DJI_0851.JPG" with timestamp prefix
}

Drone Model Differences

DJI Mini 4 Pro (Model "4")

• Path Format: /DCIM/DJI_XXX/DJI_*_YYYY_D.JPG
• Filename Format: TIMESTAMP_DJI_TIMESTAMP_YYYY_D.JPG
• Matching: Extracts 4-digit number (YYYY) from both path and filename

Other Models (DJI Mini 2, 3, 3 Pro)

• Path Format: /DCIM/XXXmedia/DJI_YYYY.JPG
• Filename Format: DJI_YYYY.JPG or TIMESTAMP_DJI_YYYY.JPG
• Matching: Extracts DJI_YYYY.JPG pattern from both

Development

Build

pnpm build

Development Mode (watch)

pnpm dev

Run Tests

pnpm test

Run Tests with Coverage

pnpm coverage

Type Check

pnpm type-check

Lint

pnpm lint

License

MIT