@techolution-ai/computer-vision

A JavaScript/TypeScript library for computer vision applications, providing tools for image processing, scanning, and MQTT-based messaging.

Table of Contents

• Features
• Usage
  • Stream the video
  • Listening to MQTT messages
  • Scanner with MQTT with UI
• API Reference
• Requirements
• License
• Contributing
• Support

Features

The library provides several modules:

• Core computer vision functionality
• Scanner module for image/video scanning
• MQTT-based messaging system for real-time communication

Usage

Stream the video

Use scanner to stream the camera's live recording to web using useScanner hook.

import { useMemo } from 'react';

import { useScanner } from '@techolution-ai/computer-vision/scanner';

function App(){

  const scanner = useScanner({ baseUrl: 'http://0.0.0.0:8501' })
  const streamUrl = useMemo(()=>scanner.getVideoStreamUrl(),[scanner]);

  // render the image with steaming url
  return <ScanImage src={streamUrl} />
}

Listening to MQTT messages

We can use useMessages to connect, subscribe to certain topics, and can receive the messages on onMessage callback.

We must use the MessagesProvider to use useMessages hook. It can be put at the root (or one level up) to the component's tree.

import {
  MessagesProvider,
  useMessages,
} from '@techolution-ai/computer-vision/messages'


function MessagesListener() {

  const handleReceiveMessage = (topic: string, message: any) => {
    // your topic and message is here
    // add your logic here
  }

  useMessages({
    topics: ['current_status', 'products_scanned'], // subscribing to topics
    onMessage: handleReceiveMessage, // message handler
  })
}

function App(){
  <MessagesProvider url="ws://0.0.0.0:9001">
    <MessagesListener />
  </MessagesProvider>
}

Scanner with MQTT with UI

We can combine above examples to do a full integration.

We also have a ready-to-use react component: ScannerStatus with default options added which can be used to show different statuses in the UI.

import { useMemo, useState } from 'react';

import {
  ScanImage,
  useScanner,
  ScannerStatus,
  defaultStatusMap,
  TStatusMap,
} from '@techolution-ai/computer-vision/scanner';

import {
  MessagesProvider,
  useMessages,
} from '@techolution-ai/computer-vision/messages';

function MessagesListener() {
  const scanner = useScanner({ baseUrl: 'http://0.0.0.0:8501' })

  const streamUrl = useMemo(() => scanner.getVideoStreamUrl(), [scanner])

  const [statusMap, setStatusMap] = useState<TStatusMap | null>(
    defaultStatusMap,
  )

  const [status, setStatus] = useState('idle')
  const [messages, setMessages] = useState<{ topic: string; message: any }[]>([])

  const handleReceiveMessage = (topic: string, message: any) => {
    try {
      const value = JSON.parse(message.toString())

      if (topic === 'current_status') {
        setStatus(value.status)
      } else if (topic === 'item_scanned') {
        setMessages((prev) => [...prev, { topic, message: value }])
      }
    } catch (e) {
      console.error('Error parsing message:', e)
    }
  }

  useMessages({
    topics: ['current_status', 'item_scanned'],
    onMessage: handleReceiveMessage,
  })

  return (
    <ScannerStatus
      style={{
        border: '4px solid #4b2f99',
        borderRadius: 16,
        overflow: 'hidden',
        position: 'relative',
        width: '100%',
        height: '100%',
      }}
      status={status}
      statusMap={statusMap}
    >
      <ScanImage
        src={streamUrl}
        style={{
          width: '100%',
          height: '100%',
          objectFit: 'cover',
          position: 'absolute',
          top: 0,
          left: 0,
          bottom: 0,
          right: 0,
        }}
      />
    </ScannerStatus>
  )
}

function App() {
  return (
    <MessagesProvider url="ws://0.0.0.0:9001">
      <MessagesListener />
    </MessagesProvider>
  )
}

API Reference

The library exposes three main modules:

1. Scanner Module

   • Specialized scanning capabilities
   • Import via @techolution-ai/computer-vision/scanner

2. Messages Module

   • MQTT-based messaging system
   • Import via @techolution-ai/computer-vision/messages

MessagesProvider (component)

Provider for MQTT connection.

Props

Prop | Type | Default Value | Remarks | ----- | ----- | ----- | ----- | | url | string | -- | Required | | enableDebugging | boolean | false | Optional, to enable logging |

ScannerStatus (component)

Take all HTMLDivElement props as root

Props

Prop | Type | Default Value | Remarks | ----- | ----- | ----- | ----- | | status | string | -- | Required | | statusMap | TStatusMap (Object) | null | Optional, uses default statusMap internally if not passed |

useMessages (react hook)

Props

| Prop | Type | Default Value | Remarks | | ----- | ----- | ----- | ----- | | topics | string[] | -- | Required | | onMessage | (topic:string,message:Buffer)=>void | callback to receive messages | Required | | onError | function | Callback function when an error occurs. (not implemented) | | onConnect | function | Callback function when the connection is established. (not implemented) | | onClose | function |Callback function when the connection is closed. (not implemented) |

useScanner (react hook)

Props

| Prop | Type | Default Value | Remarks | | ----- | ----- | ----- | ----- | | baseUrl | string | -- | Required | The base URL for the scanner API. | endpoints | object | Custom endpoints for the scanner API. (Optional)

APIs (returned functions)

| API | Remarks | | ----- | ----- | | getVideoStreamUrl() | Returns the URL for the video stream. | | startInference() | Starts the inference process. | | stopInference() | Stops the inference process. | | captureFrame() | Captures the current frame. |

Requirements

• React ≥ 16.8.0
• Node.js ≥ 14.0.0
• TypeScript ≥ 4.0.0 (for TypeScript users)
• MQTT server running along with AI backend service

License

This project is licensed under the MIT License - see the LICENSE file for details.

Contributing

Please read our contributing guidelines and code of conduct before submitting pull requests.

Support

For issues and feature requests, please use the GitHub issue tracker.