capacitor-card-scanner

Recognize a pokemon card with camera open in background

Install

To use npm

npm install capacitor-card-scanner

To use yarn

yarn add capacitor-card-scanner

Sync native files

npx cap sync

How it works

The native camera preview is placed behind the WebView and the WebView is made transparent. Your HTML UI (buttons, overlays, the card frame) is drawn on top of the live camera feed.

For the camera to be visible, every element between the page and the camera must have a transparent background (html, body and the layout containers).

iOS

Add to the app's Info.plist:

<key>NSCameraUsageDescription</key>
<string>The camera is used to scan cards</string>

OutSystems

1. Add a CSS class to the Theme / screen:

   html:has(body.camera-active),
   body.camera-active,
   body.camera-active #reactContainer,
   body.camera-active .screen-container,
   body.camera-active .active-screen,
   body.camera-active .screen,
   body.camera-active .layout,
   body.camera-active .main,
   body.camera-active .main-content,
   body.camera-active .content {
     background: transparent !important;
   }

2. On the screen OnReady (JavaScript node):

   const { CardScanner } = Capacitor.Plugins;
   document.body.classList.add('camera-active');
   await CardScanner.start({ position: 'rear' });

3. On OnDestroy:

   document.body.classList.remove('camera-active');
   await Capacitor.Plugins.CardScanner.stop();

4. Button to scan:

   const photo = await Capacitor.Plugins.CardScanner.capture({ quality: 80 });
   $parameters.ImageBase64 = photo.base64;

Card detection (Android)

After start(), call startDetection(). A red rectangle is drawn around the card (between the camera and the WebView) and its text is read on-device with ML Kit. cardDetected fires once per card, when the same collector number (or name) was read on stableFrames consecutive runs. Remove the card from view for ~1.5s to scan the same card again.

const { CardScanner } = Capacitor.Plugins;
await CardScanner.addListener('cardDetected', (card) => {
  // card.name, card.collectorNumber ('025/165'), card.text, card.lines, card.cardBox, card.image
});
await CardScanner.startDetection({ intervalMs: 300, stableFrames: 2, includeImage: false });
// ...
await CardScanner.stopDetection();
await CardScanner.removeAllListeners();

API

• start(...)
• stop()
• capture(...)
• setTorch(...)
• setZoom(...)
• flip()
• startDetection(...)
• stopDetection()
• retryCard()
• setCardState(...)
• focus(...)
• addListener('cardDetected', ...)
• removeAllListeners()
• checkPermissions()
• requestPermissions()
• Interfaces
• Type Aliases

start(...)

start(options?: StartOptions | undefined) => Promise<void>

Opens the camera behind the WebView. The WebView background is made transparent, so the HTML page must also use transparent backgrounds where the camera should be visible.

| Param | Type | | ------------- | ----------------------------------------------------- | | options | StartOptions |

stop()

stop() => Promise<void>

Stops the camera and removes the preview.

capture(...)

capture(options?: CaptureOptions | undefined) => Promise<CaptureResult>

Takes a photo from the running camera.

| Param | Type | | ------------- | --------------------------------------------------------- | | options | CaptureOptions |

Returns: Promise<CaptureResult>

setTorch(...)

setTorch(options: TorchOptions) => Promise<void>

Turns the flashlight on/off.

| Param | Type | | ------------- | ----------------------------------------------------- | | options | TorchOptions |

setZoom(...)

setZoom(options: ZoomOptions) => Promise<void>

Sets the zoom ratio.

| Param | Type | | ------------- | --------------------------------------------------- | | options | ZoomOptions |

flip()

flip() => Promise<void>

Switches between the rear and front camera.

startDetection(...)

startDetection(options?: DetectionOptions | undefined) => Promise<void>

Starts looking for cards: draws a red rectangle around the card and reads its text, emitting cardDetected. Requires start() first. Android only.

| Param | Type | | ------------- | ------------------------------------------------------------- | | options | DetectionOptions |

stopDetection()

stopDetection() => Promise<void>

Stops card detection and hides the rectangle.

retryCard()

retryCard() => Promise<void>

Forget the last reported card, so the card in view is reported again on the next reading. Call it when the app could not identify the card (e.g. no match in the catalog); this makes stableFrames: 1 safe to use.

setCardState(...)

setCardState(options: SetCardStateOptions) => Promise<void>

Shows the app's verdict on the card in view by colouring its outline (and vibrating, see haptics). Lasts until another card appears, a card is reported or retryCard() is called, when the outline goes back to red (reading) by itself.

| Param | Type | | ------------- | ------------------------------------------------------------------- | | options | SetCardStateOptions |

focus(...)

focus(options: FocusOptions) => Promise<void>

Focuses the camera at a point of the screen (tap to focus) and shows a focus ring there. The WebView sits on top of the camera, so the app forwards its own taps. Card focusing (focusOnCard) pauses for 3 s. By default the tapped card is then read and reported at once (see read).

| Param | Type | | ------------- | ----------------------------------------------------- | | options | FocusOptions |

addListener('cardDetected', ...)

addListener(eventName: 'cardDetected', listener: (event: CardDetectedEvent) => void) => Promise<PluginListenerHandle>

Fired once per card, when its text has been read consistently.

| Param | Type | | --------------- | ----------------------------------------------------------------------------------- | | eventName | 'cardDetected' | | listener | (event: CardDetectedEvent) => void |

Returns: Promise<PluginListenerHandle>

removeAllListeners()

removeAllListeners() => Promise<void>

checkPermissions()

checkPermissions() => Promise<PermissionStatus>

Returns: Promise<PermissionStatus>

requestPermissions()

requestPermissions() => Promise<PermissionStatus>

Returns: Promise<PermissionStatus>

Interfaces

StartOptions

| Prop | Type | Description | Default | | -------------- | --------------------------------------------------------- | -------------------- | ------------------- | | position | CameraPosition | Which camera to use. | 'rear' |

CaptureResult

| Prop | Type | Description | | ------------ | ------------------- | ------------------------------------------------- | | base64 | string | Base64 encoded JPEG (without the data: prefix). | | width | number | | | height | number | |

CaptureOptions

| Prop | Type | Description | Default | | ------------- | ------------------- | --------------------- | --------------- | | quality | number | JPEG quality (0-100). | 85 |

TorchOptions

| Prop | Type | | ------------- | -------------------- | | enabled | boolean |

ZoomOptions

| Prop | Type | Description | | ----------- | ------------------- | ----------------------------------------------------- | | ratio | number | Zoom ratio, clamped between the camera's min and max. |

DetectionOptions

| Prop | Type | Description | Default | | ------------------------- | ------------------------------------------------------- | --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | | intervalMs | number | Minimum time between two OCR runs, in milliseconds. The red rectangle is updated on every frame regardless. | 300 | | stableFrames | number | How many consecutive OCR runs must read the same card before cardDetected fires. | 2 | | includeImage | boolean | Include a JPEG of the detected card in the event. | false | | cardAspectRatio | number | Expected width / height of an upright card. Only upright cards are detected; a card lying sideways falls outside the tolerance. | 0.716 (63 x 88 mm) | | aspectTolerance | number | Allowed relative deviation from cardAspectRatio (0.15 = ±15%). Increase it if tilted cards are not detected. | 0.15 | | minCardArea | number | Smallest card size, as a fraction of the camera frame (0-1). | 0.08 | | maxCardArea | number | Largest card size, as a fraction of the camera frame (0-1). | 0.9 | | requireCard | boolean | Only read text when a card-shaped object is visible. When false, the whole frame is read if no card is found. | true | | debug | boolean | Normally the card is marked with four rounded corner brackets in its exact proportions, gliding and fading smoothly. With debug, the raw detector box is drawn instead, labelled with its area (a) and width/height ratio (r), plus the rejected objects (see showRejected) and the OCR strips (ocrMode: 'strips'). Use it to tune the filters above. | false | | showRejected | boolean | With debug, also outline (in grey) the objects that were not accepted as a card, with the filter that rejected them. Never shown without debug. | true | | repeatAfterMs | number | How long (ms) the card must be out of view before the same card can be reported again. While a card stays in view it is reported only once. | 1500 | | cardGapMs | number | While the card outline stays in view, at most one card is reported, even if later readings give a different name. The outline must disappear for longer than this (ms) for the next card to be read, e.g. when swapping cards. | 500 | | sameCardMaxDistance | number | | | | sameCardThreshold | number | Two pictures whose colour signatures (8x8 luma and colour averages of the artwork) differ by at most this (0-1) are the same card. Decides whether the card in view changed, so it is read again, and stops the same card being reported twice. Lower = more readings, fewer look-alikes missed. | 0.2 | | maxRetries | number | How many times retryCard() can re-report the same card. Stops a card missing from the catalog, or a card-shaped object, from firing forever. The budget resets when the card leaves the view or another card appears. | 5 | | nameLines | number | The card name is looked for only in the first N lines of text on the card, top to bottom. | 3 | | reservedWords | string[] | Words/phrases that are never the card name (card types, subtypes...). A line made only of these words is skipped, so "Trainer" or "Stage 1" are ignored but "Energy Retrieval" is still a name. Replaces the defaults. Also matched without spaces, so "STAGE1" counts as "Stage 1". | ['Basic', 'Stage', 'Stage 1', 'Stage 2', 'Trainer', 'Item', 'Supporter', 'Stadium', 'Energy', 'Special Energy', 'Basic Energy', 'Pokémon', 'Pokémon Tool', 'Tera'] | | reservedPrefixes | string[] | Lines starting with any of these are never the card name. Replaces the defaults. | ['Evolves from', 'HP'] | | ocrMode | 'full' | 'strips' | Where OCR runs. 'full': the whole card. 'strips': only a strip at the top of the card (name) and one at the bottom (collector number), which is faster and reads the small number better, as the bottom strip is enlarged. With debug, the strips are shaded on the card. | 'full' | | nameStripHeight | number | 'strips' mode: height of the name strip, from the top of the card down, as a fraction of the card height. | 0.25 | | numberStripHeight | number | 'strips' mode: height of the number strip, from the bottom of the card up, as a fraction of the card height. | 0.15 | | numberStripScale | number | 'strips' mode: how much the number strip is enlarged before OCR (1-4). | 2 | | fastAccept | boolean | Report a reading at once when it has both the name and the collector number with its set total ("045/198"), without waiting for stableFrames. Incomplete readings still need stableFrames. | true | | recheckMs | number | While a reported card stays in view with the same picture, OCR pauses; it can still re-read it this often. 0 = never: the colour signature already tells look-alike cards apart. | 0 | | requireFullyVisible | boolean | Only accept cards fully inside the part of the camera image shown on screen. The preview fills the screen by cropping the camera image (usually the sides), so without this a card can be read before the user sees it. With debug, such cards are outlined in grey as offscreen. | true | | focusOnCard | boolean | Point the camera's focus and exposure at the card when it appears or moves clearly, and go back to automatic focus when it leaves. | true | | haptics | boolean | Vibrate when setCardState() is called: a click for identified, a light tick for choose, a double click for notFound. | true | | appearAnimation | 'pokeball' | 'open' | 'fade' | How the corner brackets appear on a new card: 'pokeball' (a Pokéball pops in and wobbles while the card is read, then splits into the four brackets when setCardState() gives the verdict, or after 3 s at the latest), 'open' (they spread from the middle of the card) or 'fade'. | 'pokeball' | | overlayColors | OverlayColors | Colours of the corner brackets per state, as #RRGGBB. Missing ones keep the default: detected grey #C8C8C8, reading (OCR started) red #E3350D, and the app's verdicts set with setCardState(): identified green #2ECC71, choose yellow #FFCB05, notFound orange #FF8C00. | |

OverlayColors

| Prop | Type | | ---------------- | ------------------- | | detected | string | | reading | string | | identified | string | | choose | string | | notFound | string |

SetCardStateOptions

| Prop | Type | Description | | ----------- | ---------------------------------------------------------------- | --------------------------------------------------------------------------------------------------------------------------------- | | state | 'identified' | 'choose' | 'notFound' | 'reading' | 'identified' (green, with a pulse), 'choose' (yellow), 'notFound' (orange, with a shake), or 'reading' to go back to red. |

FocusOptions

| Prop | Type | Description | Default | | ---------- | -------------------- | -------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | ----------------- | | x | number | Horizontal position, as a fraction (0-1) of the screen width (e.g. event.clientX / window.innerWidth). | | | y | number | Vertical position, as a fraction (0-1) of the screen height (e.g. event.clientY / window.innerHeight). | | | read | boolean | Also read the card there as soon as focus settles (at most 0.7 s), without waiting for intervalMs or stableFrames (the event has manual: true). A card already reported is not reported again; use retryCard() for that. | true |

PluginListenerHandle

| Prop | Type | | ------------ | ----------------------------------------- | | remove | () => Promise<void> |

CardDetectedEvent

| Prop | Type | Description | | --------------------- | ----------------------------------- | --------------------------------------------------------------------------------------------------------------------------------- | | name | string | Best guess for the card name (topmost line of text on the card). | | collectorNumber | string | Collector number, e.g. 025/165. | | text | string | All text read on the card. | | lines | TextLine[] | | | cardBox | Box | Where the card is in the camera frame. Missing if no card outline was found. | | image | string | Base64 JPEG of the card, when includeImage is true. | | imageHash | string | 64-bit perceptual hash of the card image (16 hex characters). Photos of the same card give hashes that differ in only a few bits. | | detectionMs | number | Time (ms) from the card appearing in view to this event. | | ocrMs | number | Duration (ms) of the text recognition that produced this event. | | attempt | number | 1 for the first report of this card, 2+ for reports after retryCard(). | | lastAttempt | boolean | True when maxRetries is used up: retryCard() will not report this card again. | | manual | boolean | True when this reading was asked for by a tap (focus() with read). |

TextLine

| Prop | Type | Description | | ---------- | ----------------------------------- | ------------------------------- | | text | string | | | box | Box | Position inside the card image. |

Box

Normalized rectangle (0-1).

| Prop | Type | | ------------ | ------------------- | | x | number | | y | number | | width | number | | height | number |

PermissionStatus

| Prop | Type | | ------------ | ----------------------------------------------------------- | | camera | PermissionState |

Type Aliases

CameraPosition

'rear' | 'front'

PermissionState

'prompt' | 'prompt-with-rationale' | 'granted' | 'denied'