In line with Lightform’s core mission, to democratize projection mapping for the masses, we wanted to create an easy to use experience that would allow people to easily create at-home projection mapping experiences. To help bring this into the home we designed and developed a user friendly set up experience to guide our users to get them creating awesome environments quickly. In addition we created a new app experience for users to engage with their content and experiences outside of our prosumer (think artists and design agencies) projection mapping software. Here is it’s story.
Technologies
StencilJs | Typescript | Ionic | Stencil Store | SCSS | P5.js
PWA | JSX | Jest | Storybooks | Rollup | Workbox
REST APIs | RPC APIs | NPM | Gulp | AWS
FTUX – The First Time User Experience
A Stencil Js web application running through an Android Web View instance
Part of the development process for creating the FTUX web app and the Web Controller was to first decide on a tech stack. In order to limit the dependency on framework trends, I decided to look for a web component based solution that would create standards-compliant components. Here I used Stencil Js as hierarchy of components and used the Stencil State Tunnel as a lightweight state management tool that works similar to React’s Redux.
This page makes calls to the Lightform A/R Projector to get and display device and network settings before moving the network pairing. Here I made a highly customized virtual keyboard using NPM package. This allowed users to access a full QWERTY keyboard via just the D-pad on the Lightform remote control. All remote control behavior was handled and controlled via the web app.
After a device has connected to the internet, the web app makes calls to the backend via the device’s connection to verify if it has the most up to date firmware. If it did not have it, then it would download and install the firmware on the spot before moving on to the next step.
Using the Lightform device’s remote control, a user can enter their one time password that is provided to them through the mobile web app simply called the Web Controller.
Web Controller – Customized Projection Mapping App
A Stencil Js and Ionic Progressive Web Application
In order to make projection mapping a breeze for people, we wanted to empower users to create custom experiences right at home. Through this web app, we provided them the tools and content to explore and create right at their fingertips. The Web Controller works similarly to your music app where you can select different music, or in our case visuals, to play on demand. But we also added a UI to allow people to better fine tune their projection mapping experience incase the AI behind the alignment had a different idea than you did.
In order to create a frictionless and seamless process for projection mapping the app was developed to help guide users through set up their custom art pieces for projection mapping.
Leveraging the popular Processing library, P5.js, I was able to create a corner pinning UI that adjust the image rendered in the UI and then send the updated coordinates to the projector to show the change in projection.
In order to give users an on demand visual experience we created a UI that allows you to change and adjust the projected mood the same way that you would set the mood for a party. In order to be able to seamlessly update the application, we went with a PWA approach. This not only made the application device agnostic but it also meant that the web page could be installed as a desktop app in the toolbar or a a mobile app on users home screen.