[!IMPORTANT] :warning: Status: Public Preview - The Windows AI Addon is experimental and in active development. We'd love your feedback! Share your thoughts by creating an issue.

The Windows AI Addon for Electron is a Node.js native addon that provides access to the Windows AI APIs, enabling Electron applications to leverage Windows AI capabilities directly from JavaScript.

Prerequisites

• Copilot+PC
• Windows 11 (using Windows Insider Preview build 26120.3073 (Dev and Beta Channels) or later)
• Node.js 18.x or later
• Visual Studio 2022 with C++ build tools
• Python 3.x
• Limited Access Feature Token for Phi Silica - Request a token using the LAF Access Token Request Form. Only needed if you wish to use the GenerateResponseAsync API.

See Windows AI API's Dependencies for more information on system requirements to run Windows AI API's and scripts to install required prerequisites.

[!IMPORTANT] Verify your device is able to access Windows AI models by downloading the AI Dev Gallery app. Navigate to the "AI APIs" samples and ensure they can run on your device. If the samples are blocked, the AI models may be missing from your machine. You can manually request a download by selecting the "Request Model" button and following the directions within Windows Update settings.

Dependencies

This package depends on the @microsoft/winappcli package. Any electron app which uses this package must also take a dependency on @microsoft/winappcli and follow its installation and setup steps.

Get Started Using @microsoft/winapp-windows-ai in an Electron App

Let's walk through a simple tutorial of how to get started using @microsoft/winapp-windows-ai. In this tutorial, we will create a simple Electron app that summarizes a block of text using Windows AI's Text Summarizer.

1. Create an Electron App

Create an electron app by following the getting started directions at Electron: Building you First App.

The instructions below follow the steps for adding @microsoft/winapp-windows-ai to a standard Electron app. This module can also be added to Electron apps built using Electron Forge and other templates.

2. Add @microsoft/winapp-windows-ai as a Dependency

cd <your-electron-app>
npm i @microsoft/winapp-windows-ai

3. Add @microsoft/winappcli as a dev Dependency

npm i @microsoft/winappcli -D

4. Install and Setup Dependencies

Initialize your project for calling Windows APIs. This will create an appxmanifest.xml, required assets, and make the Windows App SDK available for usage:

npx winapp init

[!IMPORTANT] When you run npx winapp init, it generates a winapp.yaml file for managing sdk versions. Make sure the version of the Microsoft.WindowsAppSDK package is 1.8.xxxxx. If it's not, simly set the version to 1.8.251106002 and run npx winapp restore to ensure the proper dependencies are avialable for the project.

5. Add systemAIModels Capability

Add systemAIModels Capability in appxmanifest.xml for app to gain access to local models:

<Capabilities>
    <rescap:Capability Name="systemAIModels" />
</Capabilities>

6. Add Debug Identity

Before your app can call the AI APIs, we need to make sure the electron process has identity as defined in you appxmanifest.xml. Add app identity to the Electron process for debugging:

npx winapp node add-electron-debug-identity

[!IMPORTANT] There is a known issue with sparse packaging Electron applications which causes the app to crash on start or not render the web content. The issue has been fixed in Windows but has not yet fully propagated to all Windows devices. If you are seeing this issue after calling add-electron-debug-identity, you can disable sandboxing in your Electron app for debug purposes with the --no-sandbox flag. This issue does not affect full MSIX packaging.

7. Use @microsoft/winapp-windows-ai

In main.js or index.js:

const {
  LanguageModel,
  AIFeatureReadyResultState,
  TextSummarizer,
  LanguageModelResponseStatus,
} = require("@microsoft/winapp-windows-ai");

const summarizeText = async () => {
  try {
    const readyResult = await LanguageModel.EnsureReadyAsync();
    if (readyResult.Status !== AIFeatureReadyResultState.Success) {
      console.log("AI not ready:", readyResult.ErrorDisplayText);
    }

    const model = await LanguageModel.CreateAsync();
    const textSummarizer = new TextSummarizer(model);

    const longText = `
      Artificial intelligence (AI) is intelligence demonstrated by machines, 
      in contrast to the natural intelligence displayed by humans and animals. 
      Leading AI textbooks define the field as the study of "intelligent agents": 
      any device that perceives its environment and takes actions that maximize 
      its chance of successfully achieving its goals.
    `;

    const summarizationPromise = textSummarizer.SummarizeAsync(longText);

    if (summarizationPromise.progress) {
      summarizationPromise.progress((error, progressText) => {
        console.log(progressText);
      });
    }

    const result = await summarizationPromise;

    if (result.Status === LanguageModelResponseStatus.Complete) {
      console.log(result.Text);
    } else {
      console.log("Summarization failed with status:", result.Status);
    }
  } catch (error) {
    console.error("Error:", error);
  }
}

Call summarizeText somewhere in your main.js or index.js.

Here's an example:

const createWindow = () => {
  const win = new BrowserWindow({
    width: 800,
    height: 600,
    autoHideMenuBar: true,
    webPreferences: {
      preload: __dirname + '/preload.js',
      sandbox: false,
    }
  })

  win.loadFile('index.html')

  win.webContents.openDevTools();

+  summarizeText();
}

8. Run Your App

npx electron .

You should see the local model streaming the response to the console.

Supported APIs

This package supports many of the API's within

• Microsoft.Windows.AI.Text
• Microsoft.Windows.AI.Imaging
• Microsoft.Windows.AI

For the full list of supported API's, see Supported API's.

If you have a request for additional API support, please file an issue.

Usage Examples

For more examples on how to use @microsoft/winapp-windows-ai within your Electron application, see the Usage Guide.

Development

Project Structure

winapp-windows-ai/
├── winapp-windows-ai/
│   ├── winapp-windows-ai.cc         # Main addon entry point
│   ├── LanguageModelProjections.h   # Language model & text skills APIs
│   ├── LanguageModelProjections.cpp
│   ├── ImagingProjections.h         # Imaging APIs
│   ├── ImagingProjections.cpp
│   ├── ContentSeverity.h            # Content safety APIs
│   ├── ContentSeverity.cpp
│   ├── LimitedAccessFeature.h       # Limited Access Feature token APIs
│   ├── LimitedAccessFeature.cpp
│   ├── ProjectionHelper.h           # Utility functions
│   ├── ProjectionHelper.cpp
│   └── binding.gyp                  # Build configuration
├── test-app/                        # Sample Electron application
│   ├── main.js                      # Electron main process
│   ├── preload.js                   # Preload script for @microsoft/winapp-windows-ai integration
│   ├── index.html                   # Sample UI for testing APIs
│   └── package.json                 # Sample app package configuration and dependencies
├── docs/                            # Documentation files
├── scripts/                         # Build scripts
│   ├── build-pkg.ps1                # PowerShell script for package building
│   └── get-build-number.ps1         # Script to retrieve build numbers
├── index.d.ts                       # TypeScript type definitions
├── index.js                         # Main module entry point and exports
└── package.json                     # Package configuration and dependencies

Building the Package Locally

1. Clone the Repository

git clone https://github.com/microsoft/winapp-windows-ai.git

2. Install Dependencies

cd <path to @microsoft/winapp-windows-ai repo>
npm install
npx winapp restore

3. Build the Native Addon

npm run build-all

Building test-app Locally

1. Build Package Locally

Complete Building the Package Locally steps above.

2. Setup Dependencies

Install dependencies:

npm install

Initialize Windows App Development CLI (select N to preserve dependency versions):

npx winapp init
npm run setup-debug

3. Run the App

npm run start

If you make changes to the @microsoft/winapp-windows-ai package and want to see your changes loaded into the sample app, make sure to re-build the addon before re-running test-app.

License

This project is licensed under the MIT License - see the LICENSE for details.

Troubleshooting

Common Issues

1. "AI not ready" or EnsureReadyAsync/GetReadyState return status AIFeatureReadyResultState::Failure(2): Ensure Windows 11 25H2+ (Windows Insider Preview) and Copilot+ PC requirements are met. Validate your device has Windows AI models installed and available by downloading the AI Dev Gallery app. Then navigate to the "AI APIs" samples and ensure they can run on your device. (see Prerequisites)

2. EnsureReadyAsync cancelled or failed: Ensure appxmanifest has the systemAIModels capability added (see Add Debug Identity + Capabilities to App)

3. "Class Not Registered": Make sure you have correctly setup the @microsoft/winappcli package (see Add @microsoft/winappcli as a Dependency).

   • If the issue is still persisting:
     1. Delete node_modules, yarn.lock, and .winapp
     2. Run npm cache clean --force
     3. Run npm install
     4. Run npx winapp restore
     5. Run npx winapp node add-electron-debug-identity

4. App renders blank: You may need to disable sandboxing when running your Electron app on Windows. Then re-run npx winapp node add-electron-debug-identity (see Add Debug Identity + Capabilities to App)

5. "Can't find module: @microsoft/winapp-windows-ai": Verify you have added the @microsoft/winapp-windows-ai package as a dependency to your application.

6. Image file not found: You must use absolute file paths with proper Windows path separators.

7. Content moderation blocks: Adjust ContentFilterOptions severity levels as appropriate.

8. Memory issues: Always call Close() or Dispose() methods to clean up resources.

9. GenerateResponseAsync calls are failing: GenerateResponseAsync uses a native API that is currently a Limited Access Feature. To request an unlock token, please use the LAF Access Token Request Form. To use this method, you must first call LimitedAccessFeature.TryUnlockToken. See Usage Guide for usage examples.

10. See Windows AI API's Troubleshooting for more information on troubleshooting the native Windows AI API's.