Skip to main content

Prototype Windows 10 app eliminates the napkin by turning sketches into code

Ink to Code, a Microsoft Garage Project

Quickly capturing an idea on a paper napkin can lead to lucrative results, but what if those doodles could turn into lines of code? That is what a group of Microsoft Garage Project interns set out to create: A Windows 10 app that enables app designers to draw wireframe sketches and export them into Visual Studio. Simply called Ink to Code, Xamarin funded the project and joined by five additional interns from the Massachusetts Institute of Technology to join and help develop the first prototype app. 

The sponsoring team and interns were both motivated by a desire to modernize the brainstorming and prototyping process from using napkin and whiteboard sketches, to an experience that is more automated and cohesive with the Visual Studio suite,” Microsoft ‎Product Marketing Manager Lainie Huston said. 

The app is based on Windows 10’s Smart Ink feature injected into the Universal Windows Platform for Microsoft’s Creators Update, which relies on artificial intelligence to improve ink recognition. Of course, that means you need a Windows 10 device that supports Windows Ink and touch-based input, but with this setup, you won’t need to sift through 30 sketches to compile your final code-based design. 

But Ink to Code isn’t meant to completely generate your app from scratch. It merely accelerates the initial layout process by recognizing common design symbols to get you started. According to Huston, the app currently supports buttons, images, labels, text fields, and text paragraphs. It’s a digital canvas where designers and programmers can merge to hammer out ideas and speed up the overall development process. 

Windows 10 Ink to Code app
Image used with permission by copyright holder

“Ink to Code can be used as a more productive canvas in brainstorm meetings, or even more significantly, as a tool that can bridge the gap between collaborators with different levels of design or technical knowledge,” Huston adds. 

As stated, Ink to Code is just a prototype for now, and presumably only visible to Windows 10 devices with touch-based screens; the Microsoft Store didn’t display the linked app on our non-touch laptop. But as the demo video shows, you can use a supported stylus to roughly draw specific elements of the app, such as its rectangular border, an image placement box, a text input field, and so on. The app turns all hand-drawn shapes into perfect lines and circles for a clean look without any manual coding involved. 

Microsoft first introduced Smart Ink in May 2017. Part of the platform is Ink Analysis that recognizes shapes and understands composition, such as text residing inside a roughly drawn rectangle. Ink Presenter collects all the strokes, and Direct Ink deals with rendering the strokes with digital ink. 

“Our goal is to understand user intent and empower developers to turn it into rich digital constructs, as well as to leverage understanding from all parts of the system,” the company says. 

Microsoft introduced similar technology to Sticky Notes with the release of Anniversary Update (Redstone 1) in August 2016. 

Editors' Recommendations

Kevin Parrish
Former Digital Trends Contributor
Kevin started taking PCs apart in the 90s when Quake was on the way and his PC lacked the required components. Since then…
How to use Windows Sandbox in Windows 10
Try sketchy applications without harming your computer using Windows Sandbox
how to use windows sandbox 20190730 124914

With the release of the Windows 10 May 2019 Update, several features were added to Windows 10. One of the more interesting was a new application known as Windows Sandbox.

Windows Sandbox is an integrated element and is primarily designed to allow you to run untrusted and sketchy apps separate from the rest of the operating system.

Read more
How to turn off notifications in Windows 10
Windows 10 Notifications

Today, it seems like you get notifications for almost everything. Whether it's a new text message or an email about your friends’ latest social media posts, the notifications never stop. But if you need to get work done on Windows 10, you need to know how to turn those distracting notifications off.

Keep reading to learn how to turn off notifications so that you can buckle down and focus. You can always reply to those messages and notifications during your next break.
Turn off notifications
Step 1: Open the Windows Settings menu by pressing the Windows Key + I, or search for "settings" in the Windows Search Bar and click the corresponding result. Make sure you choose System in the Settings menu.

Read more
Microsoft confirms Windows 10 21H1, its first major update for this year

Just one day after rumors indicated a potential announcement, Microsoft has officially revealed the first major update to Windows 10 for this year. Appropriately named Windows 10 21H1, the update is now available for "seekers" in early preview testing in the Beta branch of the Windows Insider program. For everyone else who isn't a beta tester, Microsoft is targeting a May 2021 rollout via Windows Update.

"We are introducing the next feature update to Windows 10, version 21H1. As people continue to rely on Windows more than ever to work, learn, and have fun, we understand the importance of providing the best possible update experience to help people and organizations stay protected and productive. It is a responsibility we do not take lightly," said Microsoft.

Read more