StaffPad Piano Capture: Uses AI to translate performances in notes

27 Sep 2023 by Robin Vincent | 4,5 / 5,0 | Approximate reading time: 3 Minutes

StaffPad Piano Capture  ·  Source: StaffPad

StaffPad Piano Capture  ·  Source: StaffPad

StaffPad MIDI Capture  ·  Source: StaffPad

StaffPad Video Staff  ·  Source: StaffPad

• 
• 
• 
• 

Previous Next

StaffPad, the iPad and Surface scoring app, can use the AI-powered Piano Capture to listen to a real performance and transcribe it directly into a readable piece of sheet music.

StaffPad

It has always been a uniquely elegant piece of scorewriting software. It uses touch and the pen/pencil of the iPad and hybrid laptops to write sheet music like you would on paper. Its ability to recognise your hand-written scrawl as music is quite remarkable. Over the years, it expanded from the Microsoft Surface to the Apple iPad Pro and gathered a high-quality orchestral library along the way.

This latest update adds some innovative pitch detection tools and the ability to handle a MIDI input and run video on its own staff.

Piano Capture

Pitch detection and sound-to-MIDI technology have been around for decades, so what makes Piano Capture innovative? Software such as Melodyne can extract MIDI and notes from recorded audio. But what’s been more difficult is the ability to interpret that MIDI data in a way that makes sense on a piece of sheet music. It would generally require so much editing to turn it into a decent piano score that it would probably be easier to write it out freehand directly into StaffPad.

So StaffPad used on-device machine learning to analyse thousands of hours of music from a special training pipeline called Pianola. It generated the content through digitally-controller acoustic pianos and sample libraries and simulated all sorts of real scenarios and environments. The system then goes through the recordings and works out how to recreate the audio input. Over time it learns to recognise what note pitches and durations look like and so can better transcribe the performance. The AI then tidies it all up a bit, and the result is fabulous.

What’s great about it is how elegantly it works. You just plonk your iPad Pro or Surface on your music stand, set it recording and play. It all works on the device, with the built-in mics, so there’s no need for setting up an audio interface or external microphones. StaffPad has always been the least “computery” notation software out there, and now it has a very untechnical way of capturing live performances. It doesn’t have to be live either because you can do the same with recorded material.

MIDI Capture

It’s boggling really, that up until now StaffPad has exclusively used a hand-written method of scoring. The ability to notate from a MIDI controller keyboard is long overdue, but MIDI Capture now makes that possible. So, while Piano Capture makes it very easy to transcribe your piano, MIDI Capture will help you use a MIDI keyboard to notate everything else directly in.

StaffPad MIDI Capture

Cinematic Scoring

The other new feature in this chunky update is the support for e a video staff. You get a strip of thumbnails running across the top of the score and can have a window showing the video that follows the playback. Fantastic for people writing for film.

StaffPad Piano Capture

Update

Piano Capture, MIDI Capture and the Video Staff are all contained in the latest StaffPad update, which goes out free to all current users. I’m a long-time user, but I haven’t seen the update myself as yet, but I’m told it can be a bit slow through the Microsoft Store. Otherwise, it’s available for $89.99 in the Apple App Store or Microsoft Store.

• StaffPad website.

How do you like this post?

Rating: Yours: | ø:

This might also interest you

StaffPad makes it to iPad with a huge 2020 update

Sibelius First 8: “Lite” composing software from Avid

Piano Producer records MIDI files with the piano of your dreams

Rode Capture is a free mobile app for content creators