Desktop app for video generation

Internet Services

USA

nda

A desktop application designed to capture interactions with screens or separate windows, generating a playbook with the help of existing API.

Web app

Electron.js

TypeScript

Swift

overview

[01]

About the client

The solution is a generative AI platform that helps teams deliver knowledge faster, making it easy to share with customers and employees. Its video documentation addresses customer inquiries, onboards, trains staff, and promotes self-service, ultimately boosting team performance.

[02]

Business challenges

The primary challenge was to develop a high-performance MVP desktop app for macOS and Windows using a single codebase. The app needed to enable seamless video documentation by capturing user actions like mouse or keyboard events. All data and a screenshot of this interaction should be saved.

Solution

The MVP desktop app streamlines video documentation creation. It allows users to capture video and interaction data (screenshots and mouse/keyboard events) to create a playbook with the help of the core API.

[01]

Screen Capture and Recording

The application's main functionality centers around screen capture, which is accomplished through Electron’s desktop-capturer API. Users can capture their interactions with specific application windows or entire screen content. Once captured, the app allows users to create a playbook in the Web App.

[02]

Cross-Platform Support: macOS & Windows

The application was developed using Electron.js, which allows it to operate seamlessly across macOS and Windows platforms. Electron.js leverages web technologies like HTML, CSS, and JavaScript, utilizing the Chromium engine to render the interface and Node.js to access system-level resources. This allows Axon to maintain a single codebase while supporting multiple operating systems.

[03]

Dynamic User Interface

The UI was built using React, providing a responsive and dynamic interface which was further replaced by its design system.

[04]

Video Processing

The integration of FFmpeg enables precise video cropping based on the selected application window by identifying its coordinates and dimensions, and ensuring the video is saved in the required format.

Custom native libraries, including electron-edge-js, koffi, and MUI, were developed to capture detailed user interaction data such as mouse and keyboard activity, the currently active application, window dimensions, and other relevant metrics. These libraries ensure accurate data recording and seamless integration with the application.

[05]

API Integration

The application was fully integrated with the client's existing API to enable the efficient creation of playbooks, a key app feature. This ensures a smooth connection between recorded content and the generated instructional guides, making it a critical tool for companies aiming to automate documentation processes.

[06]

User Authentication

The app uses Firebase-based authentication with deep linking to provide a secure and seamless user login experience. This allows for easy management of user accounts, session persistence, and secure access to the application’s features.