I’ve automagically recorded my digital piano musings starting about 28 years ago. Initially, I used a simple C code on a Linux server using select to dump from dev MIDI to a timestamped file per session (a variation of an answering machine I had previously hacked together with a friend). Later, I used pianoteq recording on an iMac, which also detects gaps and records separate timestamped files. The experience has been very positive and helpful for improving technique and composition, so I do recommend this hardware if you dont already record everything. One piece of advice, however: you want to let people who touch your piano know they get recorded, or ask them to wait until you turn off the recording. I tend to forget that I record everything after a while, and an unsuspecting friend might incorrectly assume that their improv does not lead to a permanent record. I had several people be surprised when they first learned about the automatic recording after the fact, and had to remove one or two files at their request (though the last time I needed to remove a file was around 1998; more recently, people didn’t seem to mind).
In my opinion, you shouldn't add an off switch. It subverts the main feature, always on recording. As you mentioned you can unplug it. Maybe on the app side make it simple to edit out a recording.
I ordered one to use secretly. My dad’s 90 and was a working musician since he was 15 or so. He’s always wanted to make recordings of his playing but gets distracted by the technology, even something relatively plug and play like Garage Band. I’m looking forward to plugging this in and just capturing his playing without distraction.
I'm definitely not the target market for this, but I do want to say that this really scratches an itch for something that should exist, and I really want to compliment you on how fantastic the execution is! Very clean look in both the physical and digital realms, and I really appreciate just how fully featured it is! It really seems like the ideal version of what this product would be. Also love the idea of the bookmark pattern. Out of curiosity, is that customizable too?
+1 for this! Some considerations I thought of: 1. smaller keyboards don't easily have access to upper octaves. 2. User might frequently change the transpose on their keyboard.
Could there be a Create Bookmark Now button in the app as a fallback?
Likewise. I also get more "in my head" if I'm actively recording, and it just isn't the same relaxing experience as improvising and noodling. Really excited for this.
-Laptop on battery balancing on the corner of the digital piano
-Launching a custom script piano.sh (I remember that the initial install process on ubuntu was painful)
(The script (3 lines) :
qsynth&
aconnect 24 129
rosegarden&
)
-Then hit record on rosegarden
-It works I can get the music out of my head
-Then I fight with the UI to select the time position to be able to playback
-Then I select another track to record another layer on top while the playback is running, I misconfigured the recording and it recorded over my previous track.
-Then I have lost my music.
-Then I test what happens when I change the instruments on the piano : rosegarden doesn't record the instrument change and the soundfounts are different than what my piano have so it doesn't sound exactly the same
-Then I close rosegarden, remember why I hate it, plug a jack in the line-in start audacity, hit record, play music to calm me down.
-Save wav file in a lossless compression format. Store it in a file somewhere where it most likely won't ever be played again, telling myself in a few years I'll just run a script that will transcribe it and organise it directly.
**
My setup is awful from a cold start, but gets better with practice when I remember how to use the UI.
The ability to playback an existing track while I record a new track is what I used it for. And the metronome.
The main problem with midi is that the sound fonts are not exactly the same.
I probably won't buy it though, I don't want yet another app. I am not sure if it will work in 10 years. The sound is not exactly what I played. It something that should have been an option in the piano and not a $99 option.
This was a big concern of mine too. That's why as much of Jamcorder as possible is open standards.
There is also a local web interface, http://jamcorder.local, that is fully extensible by the user. And the BLE and Wifi APIs are not locked down. You can implement your own app.
The biggest limiting factor IMO, are Wifi and BLE still going to be in common use in 30 years? That's hard to predict. But the SD card will still be there. Its a tough problem.
I think that's about as good as you can do, short of putting a screen on it.
----
There's even a feature identical UART interface for the device. Surely that will still work in 30 years.
It seems the biggest future proofing thing is that all recordings are just written to an SD card. So even if the app doesn't work at all, you can always pull the card and read data off there.
It'd be so easy to do a version of the "infamous Dropbox comment" on this ("you can already build such a system yourself quite trivially by getting a MIDI cable, an audio interface, and a raspberry PI...") but of course what you have is exactly a sort of Dropbox Of MIDI here where it Just Works™ and backs up all your music automatically with no hassle.
This is a really cool example of ambient technology. Typically when people talk about ambient technology they're talking about something like an e-ink display that is pushing information to you, but in a way that doesn't require interactivity and isn't screaming for your attention. This is a little different in that it's always _receiving_ information from you without any need for interaction or maintenance, except on your terms.
The interaction model is pretty clever too. Since it's collecting data from the instrument they've found a way to cue the device to perform an action without the user needing to open an app. (black keys to bookmark) There is an app of course, but it connects directly to the device, with no annoying setup requirements. I've seen this same approach with several other devices - Xbloom coffee maker, Combustion thermometers, Week Aqua lights - it works really well. I'm understating it. It's astounding how pleasant it is to use devices like this.
As hardware continues to improve I expect this will be the default mode for pretty much every new technology appliance. Ambient operation, local data, local app, with cloud and accounts as _options_ to extend functionality if it's necessary.
This is so neat, thanks for sharing! I think the killer feature here is syncing with your phone/providing a good interface for searching. My Roland doesn't have auto recordings, but I record myself on it pretty frequently using the built-in functionality. My problem is that I have so many recordings now that moving them over to my laptop/phone and sifting through the garbage is extremely tedious.
From the FAQ it sounds like you have plans to extend the search feature set - excited to see what's in store! I feel like a similar parallel here might be the search functionality in Google Photos. If you can do for MIDI recordings what they did for photos that would be huge.
Amateur Jazz Pianist here. Purchased. I love pianoteq, but also tracking how much I'm practicing every week is great, and love the idea of the bookmarks.
I wrote a bash script [1] with similar functions for my piano, which produces ogg recording and raw midi recording, as well as splitting realtime rendered midi stream (via fluidsynth) into two audiostreams (I've mostly used it for Zoom calls with a piano teacher).
Wow, this is amazing! As a songwriter, the only feature that I’d really need to make it an insta-buy is always-recording audio as well (and then you bookmark it and it only keeps what you sang when you started playing, since obviously audio takes way more storage space). This is probably way out of scope right now, but just adding my feedback. The always-recording piano is brilliant, but I’m not sure how useful it’d be for me without the melodies I’m improvising to go along with it. Honestly, if I wasn’t broke, I’d invest in you building that feature (or entirely separate product?) because it’d be such a gamechanger (and then you could sell it to people who play any instrument).
What kind of quality do you need from the voice recordings? Adding an integrated microphone to that enclosure would be doable but will be limited by quality and where you can place it. I'm asking because I never have a microphone while playing, but I do sing so having a rough idea what I was singing would be nice but not vital.
Seems the ESP32 can to Speex encoding so I guess Chip could integrate this as well. If he manages to earn money on this.
You can monetize further: provide OEM boards and whitelabel apps/libraries for electric piano manufacturers so that you get a licensing thing and an ecosystem going. This door is not open for long, start cold-calling those companies now!
Looks really great! If I had a standalone keyboard I'd be considering this!
Your project got me thinking - here's one idea: Windows should get MIDI 2.0 support soon, incl. non-blocking MIDI reading if I understood correctly. That should make it possible to create a small background application that records all incoming MIDI from all (or chosen) connected MIDI devices. It would work very much like your recorder and could share the same mobile app?
This I would be interested in. Since it's a software only solution, it could be cheaper and lower entry barrier.
I'm not sure why you need midi 2.0 to achieve this? Recording midi on windows is totally trivial, and for this sort of application, only capturing the notes and relevant CCs (e.g. pedals) is all you need.
Yes, it's not really MIDI 2.0 related, but with the update they will also add multi-client support for MIDI 1.0, so I can record and still use the MIDI input as input to a virtual instrument.
Theyre specifically looking to record all incoming MIDI streams, which isn't possible without preventing other applications from using those MIDI devices. Only one program can listen/send to a midi device at a time
I purchased one! My use case is I have children just getting interested in piano and I think it will be awesome to have a log of their progress and playing over time to show them later.
My first thought was a tutor companion app so that a tutor [person] could monitor progress. Score their hours of practice, score their correct notes, correct timing, tone, speed etc.. Show the part of their practice pieces they make most errors on.
I suppose an AI could be trained to look for errors and act as a tutor, eg recommending practice pieces to counter bad habits or to encourage more competent play - like, your relative values are poor, practice this 'metronome' piece; or, you only play bar chords with your left hand, try this simple piece to focus on left-hand arpeggios.
This is amazing. It's amusing that you can be Roland or Yamaha and have decades of experience building digital pianos and all sorts of other digital music devices, yet even your high-end digital pianos that sell for $3000+ do not have this feature.
Would have loved to not have lost so many improvisations, and consciously recording every time before you start playing is too much hassle.
I have a 'high end' digital piano from Yamaha, and how I think about it is very similar - the digital instrument is so 'feature poor' compared to what is possible!
With that said, I'm grateful for the mechanical stability of it, and also the reliability of the interface.. Should not be taken for granted. It has buttons, not a touchscreen for example (Kawai digitals were sadly out because of this.)
Yamaha digital pianos have on-board recorder which is quite convenient and separates left and right hands.
However, the internal capacity is fairly limited and there's some tolerable yet a delay after the recording to save the buffer on the flash.
I don't think there's a ready way to copy such internal recording from the piano onto some external medium.
This device may be a nice 'upgrade' for such digital pianos. It'd be nice if the recordings can also be played for a selected hand, so that it could be used in learning.
I'm currently also doing a hardware project (also esp32-s3 based) after a stint in big tech, albeit in a completely different field.
For me, I'm finding that firmware development and app development is an absolute pain compared to mechanical design and PCB design. What was your experience like?
Without a doubt. I highly recommend Flutter though, it was a relative breeze!
For the firmware, I would have gone with Linux + a higher level language, probably Golang, because my iteration loop was almost 2 minutes w/ C & esp32. Way too long.
But at least it made the product very power efficient, and a bit cheaper.
also coming from linux + some bare-metal experience, esp-idf was absolutely horrifying.
fortunately it turned out that the s3 hardware (i've got a cardputer) isn't that crazy, and it is relatively easy, if almost completely untrodden, to program much of it from scratch (which is the familiar way to do embedded for me).
my iteration loop is ~30ms for a small program (i download directly to ram so as not to degrade flash).
people are now actively working on reverse engineering wifi, which so far has been the major reason to use espressif's software stack (their driver runs under freertos).
For the firmware, I would have stuck with Micropython if it weren't for its lack of dual core support (required in my case for simultaneous BT + fast sampling ADC).
About efficiency - ESP chugs gobs of power, comparatively (something like 50mA average, and that's with all the relevant power saving features enabled). By comparison, an nRF or an STM32W0 is single digit mA.
Love the product, love that you don’t need a cloud account. Congrats! I’m in the same situation where I seem to improvise better when I haven’t hit the record button.
Great job on this! With the ability for it to send midi over Bluetooth, it sounds like it could also act as a de facto replacement for my Yamaha Bluetooth USB adapter UD-BT01 (which has always been a bit fiddly), would that be a fair assumption?
Awesome, I just ordered one! Absolutely love the "always on" feature - kind of reminds me of the "Midi Dump Last X Minutes" you can use with many DAWs. Thanks!
What a wonderfully cohesive product that feels like it’s been begging to exist for decades. So cool to see this level of execution. Any suggestions for someone looking to make the jump from software to hardware?
I am currently building something ESP32-S3 based, after diving into Arduino as a pandemic project. I would pay $99 for an ebook (short, rough) about the entire process - design decisions, hardware prototypes, manufacturing, mobile app, etc. I would pay $199 if it included ability to send you questions with an eventual answer (no real-time response needed). Should mention I'm 99% a software person. Might be a market for this ... Just food for thought.
Love the idea. Raspberry Pis (even with swap disabled, logs in memory, etc.) really disabused me of the notion of remotely reliable microSD cards on a longish term. I would suggest considering eMMC.
> phones are a great option for occasional recording, but they don't record automatically, they don't show the notes you played, they don't track your playing history. inspired by the convenience of a phone, jamcorder is a natural step up.
There are MIDI interfaces for phones. It is astonishing that there isn't an app which it's the checkboxes.
This is great! Thanks for creating this - and as someone asked already, I would love to hear more about the process how you did this. Especially I am interested how you came up with injection molded enclosure (I thought it just costs some enormous amount to make the mold).
I was about to order but frankly it does not feel good to pay about the same amount of shipping (to Finland) as the product costs.
If you can do anything about it, I would be happy to order. 90EUR for shipping is just too much, 20-30'ish would be reasonable.
Check out Asendia. They are doing consolidated shipments which might work out nicely with this.
Works basically like this:
You collect couple of shipments to one bigger package.
Then you send that bigger package to Asendia logistics centre and they will send those individual packages to users.
I'd likely buy if I could ship to UK at a decent price! Please make this happen! Is there some way I can sign up to some future communication about this?
Not as bad as 90EUR, but shipping for me comes out as 71 NZD for me, which also seems a bit steep — would purchase almost instantly for a similarly priced shipping to that mentioned here, ~35 to 45 NZD?. Just noticed the default UPS Worldwide Saver® is quoted as 6-7 business days which (for me all the way down here) seems pretty speedy — I'd be totally happy with something cheaper but more paced like a snail!
It's amazing how hitting the record button makes you self aware. Even if it's just yourself in the room.
It can also be a mood killer when rehearsing with a band. Everyone is messing about and having fun, then suddenly REC ON! and everyone is almost dead serious in performance mode.
If this launched 5 years ago, I would have immediately purchased it! At the time I was using pencil & paper, Sibelius, or my phone to record improvisations and all of them were very poor solutions.
I've reached the point where I use a DAW (Reaper) + MIDI keyboard + sound libraries. Conveniently, Reaper can record & display everything as editable MIDI output. Pencil + staff paper can be great if you're slowly exploring something though.
This is awesome! One question, though: why not just work with straight audio and convert to midi with some basic* signal processing assuming a use case of instruments/sounds that aren't crazy synthesized. You could then also use this app for certain acoustic instruments as well and make this run all on mobile, potentially with an audio jack dongle/splitter if connecting to a digital instrument. Could also focus the audio processing on specific acoustic instruments out the gate and role out algorithms one at a time for each kind of instrument (e.g. piano vs saxophone)
> though: why not just work with straight audio and convert to midi with some basic* signal processing assuming a use case of instruments/sounds that aren't crazy synthesized.
Why would you start with that? Particularly since some of the best "digital pianos" are actually just MIDI keyboard controllers (weighted and all that) and don't actually generate any audio themselves?
As someone with a professional background in consumer hardware (and music production) my first thought is always "how can I solve this problem without hardware" first. Don't get me wrong--hardware is fun--but manufacturing and selling consumer hardware is brutal from a business perspective (of course fun in its own masochistic way). With hardware, you're dealing with annoying cashflow issues unless you can easily build to order (money tied up in inventory or even the supplies), supply chain management, inventory management overhead, logistics and fulfillment overhead, additional regulatory overhead.
My understanding from OP's product is that there is an inherent assumption that this is targeted towards people dabbling on standalone digital pianos that also typically have built-in synthesis/sound modules.
Anyway, I'm just giving my reasons for why wouldn't you start with a software approach to answer your question. Putting all that aside, OP's solution is really cool and, independent of the decision to go custom hardware, it seems like a great product solution for the problem it's solving.
To not start with MIDI means starting with audio. That means a microphone, preamp, digital audio interface ... all of which are significantly more expensive than the essentially zero-cost DIN or USB MIDI port that this device requires.
This device is a hardware solution. Hardware solutions are very popular in the music technology space, because nobody really wants to be screwing around with their laptop or whatever. A software solution that did the same means a computer that is running with a mic, preamp and audio interface at all the same times as the person produces sound.
The product in TFA has no dependence on the presence (or absence) of synthesis in the keyboard unit.
I'm suggesting that the user's phone that they already need (since the hardware pairs with the app in the article) is the hardware in this context. A phone already has a microphone and--if the environment is too noisy- many also support direct audio input through the phone's charge/data port with a simple off-the-shelf converter. Most smartphones can perform real-time audio progressing nowadays.
The trade-off is that midi is going to be lower error than an audio processing algorithm and much easier to implement out-the-gate, while it's certainly a step-up in the software engineering needed to develop and implement an audio processing algorithm to implement pitch/key detection as well as capture relative timing and velocity.
Correct the product in the article has no dependence on synthesis, but in pretty much all contexts in which someone will be using the device, there will be a corresponding audio signal present. Most people don't tap out music with a midi controller just in silence/without some sort of synthesis in conjunction.
Also to clarify, yes: I understand most digital pianos have midi baked in. It's an easy signal to get out. I think this is a great product concept and there's nothing wrong with it--people will buy it! Just spitballing if hardware and all the baggage a hardware business model comes with is necessary.
This just wouldn't work as a phone app. The main selling point of this is always on recording. When the keyboard is on, it records, nothing for you to do.
Only way this works in software is if you dedicate a device to always record and audio processing doesn't provide isolation. Ambient sounds may get picked up or loud noises could drown out your playing.
This item works, because it is hardware. I also wonder how this would work connected to something like the sonuus G2M V3. Would be nice if the audio processing was solved by someone else.
I still use a Yamaha Clavinova that I got in the early 90's...it has a floppy disc drive that still works for recording midi and is extremely convenient. Why such a feature with auto record didn't become a standard feature when SD cards came out is unfortunate.
Love the idea. As the device is recording all MIDI events, you can use it for any device with MIDI output (synth, drums, etc.). Are you envisioning expanding the app usage for other instruments than piano? (Would be great for drumming for instance)
+1 for this, I may end up buying this anyway but aside from the piano I have a Arturia Keystep Pro hooked up to a few synths that can sequence 4 channels, would be amazing to have this work with that setup and record all the midi channels and be able to select the channels in the app.
That's a feature I thought about adding! It could be as simple as tapping 'use midi out' + 'play' on the phone. But would it need 'looper' support to be useful? Curious your use cases.
I would strongly recommend adding the ability to play back through a MIDI device. For me, I can get quite attached to the particular piano sounds on different devices I have, and being able to hear my playing exactly as I heard it when I played it is great. This is also, in some cases, the only way to make use of a keyboard/piano's built-in speakers.
This would also possibly widen the market for the product a little. Playing non-piano instrument sounds, even very technical synthy stuff, on a keyboard should “just work” assuming that all outgoing MIDI data like Program Changes and Control Changes are recorded and replayed. The further you get away from pure piano, the less likely the result will sound right on anything other than the original keyboard/piano.
Incidentally I'm curious if SysEx are recorded. Sometimes those are also used. For example some of the effect controls on certain Yamaha products use SysEx for historical reasons.
If you do add playback over MIDI you may want to familiarise yourself with some of the “reset” mechanisms to avoid hanging notes and other such problems, and make the app send them at appropriate moments. All Notes Off, All Sounds Off and Reset All Controllers are particularly useful “special” Control Changes.
Just wanted to add, I read your product page — love the idea and simplicity of the overall design! This idea was the first thing I thought of wanting as a feature. Essentially I play a midi controller as a keyboard connected to a Zynthian running Pianoteq, and it'd be really nice to be able to hear that same sound back on a whim.
To be clear it isn't in any way a deal breaker as the Zynthian's CPU isn't quite good enough to run Pianoteq in the best quality anyway, and moving some MIDI off to a DAW later to create a higher quality version seems like a great option — just it'd be really cool to be able to play things back the same way I played them initially :)
Actually just asking for a friend. I think it does not have to loop or anything.
Just needs to let me "listen" through the midi out instead of the phone.
Caveat if course is, that I'd need to remember the synthesizer/settings/presets to have it sound the same again, but maybe half the fun to guess a bit and get happy accidents.
I used to play the piano. The intro video is fantastic, answered all the questions that I had _while_ watching the video, and somehow managed to sell it to the software engineer me, too. Thanks for showing us something amazing!
Instabuy from me. I play piano every day (not great at it) and find that the consistent playing gets me better over time. This will be a great way to track that, and maybe bookmark/share those rare sessions when it actually turns out good.
Looks awesome, does it also record the pedal? Also I am worried that if it just records in MIDI, the output sound may be different from what I was hearing when playing unless I somehow have exactly the same sound banks as my digital piano.
The MIDI keyboards I know will always output the pedal as part of the overall MIDI out. The pedal is connected to the keyboard via jack cables - or do you have a pedal that has its own MIDI output?
I believe you're correct. The 1/4" pedal goes into the device, but anything emitted from there on should be an aggregation of any MIDI going out (provided they're on the same channel).
There should be a way to run the MIDI back through your digital piano (or keyboard) to record the audio That said, software pianos are relatively easy to find with impressive sound banks.
This does look awesome. All my musical gear runs through a perpetually-on Ableton setup, so I'm not really the target for this. But I did order one as a gift for my band's singer as he never really figured out how to use a DAW and he does all his songwriting on a digital piano and uses a tascam audio recorder when he wants to save an idea. I think he'll really enjoy this.
This is pretty awesome! Nicely done! Some questions:
- Can the out be used as a thru MIDI port? So I could just jam the device into my workflow without having to acquire a new MIDI splitter?
- What of MIDI it captures, exactly? I understand note on/off, velocity and such is obviously captured, but what about CC values and everything else? Would be wonderful to be able to hook this up to synths that have more controls, and be able to capture those values too
Did you write your own support for USB-MIDI or are you using some ready-made library?
I remember once hacking on my own project with USB-MIDI support and it was a hassle. I was constantly missing MIDI messages. Had to build a "panic button" in to force "stop all notes".
Although my project was using SAMD21 (Cortex M0+), so possibly it wasn't fast enough for the use case.
There's something so whimsical about this kind of tech. The cost to store the midi is effectively zero, it would be so interesting to see it installed within a keyboard - a journal chronicling the player's progress over decades, and everywhere the keyboard has been. I could see something like this being included in the next OP-1
At least for me - friction. That said, this is definitely targeting more of the DP market (with built in sounds), since if you're using a midi controller like an NI Komplete keyboard you already have to connect DAW/VSTs.
Having to lug my laptop out, boot'up Bitwig, connect it to the piano, press record completely takes me out of the flow.
With this, I can flip on my Kawai and start banging away, then pull out the impromptu motifs for later refinement.
Ordered, and I'm grateful you showed HN. One of my teenage sons is a brilliant improviser, and I'm thrilled at the idea of having a plug-and-play way to capture what he plays.
Hi Chip, do you have a blog post or can detail the process you went through to produce the device? How did you design the internal components? How did you get it produced?
I have a very minor feature request: MIDI loops! Add a button on top of the device: Press once to record, press again to stop. Quantize on/off available via the app.
Very cool. I recently switched from an acoustic piano to a digital one. Would love to read something about your process getting to market and building the hardware.
I am not a musician so I have no use for this but this is a delightful idea that arguably belongs as a standard feature of every MIDI device being made now.
Indeed! I just came back to post the exact same adjective after purchasing one as a Christmas gift for my teenage son. I think he'll love it, and I'm excited to get him such a cool present that he doesn't even know exists! (Though as spiffy as this is, there's a good chance that's not true by Christmas.)
Congrats on the launch and bravo on such a well-polished everything - product, UI, website, etc. Very impressive.
I had a Fantom X6 and one feature I loved about it is that it was always recording in the same manner as this, you could just push the "Skip Back Sampling" button and then save or do whatever with it. It was also good because the audio samples were synced to the midi clock. A cool feature.
Yes, you have to push the button to access it after, but it's always recording up to the length of it's buffer (which depends on how much RAM you have installed). From my very hazy, vague memory it was ~5 minutes.
All-around great product - the best part for me is how there's an offline option, no mandatory app or subscription BS. There's too much of the latter nowadays.
I am as far away from the target market as possible, but just want to say a lot of things about this device, from the use cases, the design and the UI, feel so delightful. Good luck with the launch.
"Do i need to use the app? - No. You can also access your recordings directly from the SD card and by using wifi setup you can do things like set the date & time without the app.", from the FAQ
Yes there are some limited times you might want to turn off recording. I've given this some thought. For now, unplugging is the simple answer.
Actual sounds are coming via Pianoteq though.
Sample: https://www.youtube.com/watch?v=PSHis77Y1f8
I use OBS to combine all the various elements.
I ordered one to use secretly. My dad’s 90 and was a working musician since he was 15 or so. He’s always wanted to make recordings of his playing but gets distracted by the technology, even something relatively plug and play like Garage Band. I’m looking forward to plugging this in and just capturing his playing without distraction.
Should be pretty straightforward! I'd want a decent UI for it!
Could there be a Create Bookmark Now button in the app as a fallback?
And if you choose, you can make it slightly harder to invoke by requiring an additional 'function' note to be held down.
I haven't felt need for a bookmark button in the app yet, but it did cross my mind.
This might be the first Show HN that I insta-purchased after reading your landing page. The mobile interface looks extraordinarily well thought-out.
This is an absolutely phenomenal use of the espressif esp32s3 in a hardware that I will likely use daily for the rest of my life.
Kudos!
You should have seen the original UI concepts I made, they were terrible!
**
My current setup in comparison :
-Had to grab the USB-B cable from the printer
-Laptop on battery balancing on the corner of the digital piano
-Launching a custom script piano.sh (I remember that the initial install process on ubuntu was painful) (The script (3 lines) : qsynth& aconnect 24 129 rosegarden& )
-Then hit record on rosegarden
-It works I can get the music out of my head
-Then I fight with the UI to select the time position to be able to playback
-Then I select another track to record another layer on top while the playback is running, I misconfigured the recording and it recorded over my previous track.
-Then I have lost my music.
-Then I test what happens when I change the instruments on the piano : rosegarden doesn't record the instrument change and the soundfounts are different than what my piano have so it doesn't sound exactly the same
-Then I close rosegarden, remember why I hate it, plug a jack in the line-in start audacity, hit record, play music to calm me down.
-Save wav file in a lossless compression format. Store it in a file somewhere where it most likely won't ever be played again, telling myself in a few years I'll just run a script that will transcribe it and organise it directly.
**
My setup is awful from a cold start, but gets better with practice when I remember how to use the UI. The ability to playback an existing track while I record a new track is what I used it for. And the metronome. The main problem with midi is that the sound fonts are not exactly the same.
I probably won't buy it though, I don't want yet another app. I am not sure if it will work in 10 years. The sound is not exactly what I played. It something that should have been an option in the piano and not a $99 option.
> I am not sure if it will work in 10 years.
This was a big concern of mine too. That's why as much of Jamcorder as possible is open standards.
There is also a local web interface, http://jamcorder.local, that is fully extensible by the user. And the BLE and Wifi APIs are not locked down. You can implement your own app.
The biggest limiting factor IMO, are Wifi and BLE still going to be in common use in 30 years? That's hard to predict. But the SD card will still be there. Its a tough problem.
I think that's about as good as you can do, short of putting a screen on it.
----
There's even a feature identical UART interface for the device. Surely that will still work in 30 years.
It'd be so easy to do a version of the "infamous Dropbox comment" on this ("you can already build such a system yourself quite trivially by getting a MIDI cable, an audio interface, and a raspberry PI...") but of course what you have is exactly a sort of Dropbox Of MIDI here where it Just Works™ and backs up all your music automatically with no hassle.
This is a really cool example of ambient technology. Typically when people talk about ambient technology they're talking about something like an e-ink display that is pushing information to you, but in a way that doesn't require interactivity and isn't screaming for your attention. This is a little different in that it's always _receiving_ information from you without any need for interaction or maintenance, except on your terms.
The interaction model is pretty clever too. Since it's collecting data from the instrument they've found a way to cue the device to perform an action without the user needing to open an app. (black keys to bookmark) There is an app of course, but it connects directly to the device, with no annoying setup requirements. I've seen this same approach with several other devices - Xbloom coffee maker, Combustion thermometers, Week Aqua lights - it works really well. I'm understating it. It's astounding how pleasant it is to use devices like this.
As hardware continues to improve I expect this will be the default mode for pretty much every new technology appliance. Ambient operation, local data, local app, with cloud and accounts as _options_ to extend functionality if it's necessary.
From the FAQ it sounds like you have plans to extend the search feature set - excited to see what's in store! I feel like a similar parallel here might be the search functionality in Google Photos. If you can do for MIDI recordings what they did for photos that would be huge.
[1] https://github.com/seletskiy/dotfiles/blob/master/bin/piano
Seems the ESP32 can to Speex encoding so I guess Chip could integrate this as well. If he manages to earn money on this.
I'll add you when I create one.
Appreciate the interest :)
Btw, for anyone that purchases, there is also a (very new) subreddit!
https://www.reddit.com/r/jamcorder/
Your project got me thinking - here's one idea: Windows should get MIDI 2.0 support soon, incl. non-blocking MIDI reading if I understood correctly. That should make it possible to create a small background application that records all incoming MIDI from all (or chosen) connected MIDI devices. It would work very much like your recorder and could share the same mobile app?
This I would be interested in. Since it's a software only solution, it could be cheaper and lower entry barrier.
See announcement here: https://devblogs.microsoft.com/windows-music-dev/windows-mid...
I suppose an AI could be trained to look for errors and act as a tutor, eg recommending practice pieces to counter bad habits or to encourage more competent play - like, your relative values are poor, practice this 'metronome' piece; or, you only play bar chords with your left hand, try this simple piece to focus on left-hand arpeggios.
Would have loved to not have lost so many improvisations, and consciously recording every time before you start playing is too much hassle.
With that said, I'm grateful for the mechanical stability of it, and also the reliability of the interface.. Should not be taken for granted. It has buttons, not a touchscreen for example (Kawai digitals were sadly out because of this.)
However, the internal capacity is fairly limited and there's some tolerable yet a delay after the recording to save the buffer on the flash.
I don't think there's a ready way to copy such internal recording from the piano onto some external medium.
This device may be a nice 'upgrade' for such digital pianos. It'd be nice if the recordings can also be played for a selected hand, so that it could be used in learning.
For me, I'm finding that firmware development and app development is an absolute pain compared to mechanical design and PCB design. What was your experience like?
For the firmware, I would have gone with Linux + a higher level language, probably Golang, because my iteration loop was almost 2 minutes w/ C & esp32. Way too long.
But at least it made the product very power efficient, and a bit cheaper.
Btw, you should follow the hot reload thread on Github for ESP-IDF: https://github.com/espressif/esp-idf/issues/12642
fortunately it turned out that the s3 hardware (i've got a cardputer) isn't that crazy, and it is relatively easy, if almost completely untrodden, to program much of it from scratch (which is the familiar way to do embedded for me).
my iteration loop is ~30ms for a small program (i download directly to ram so as not to degrade flash).
people are now actively working on reverse engineering wifi, which so far has been the major reason to use espressif's software stack (their driver runs under freertos).
About efficiency - ESP chugs gobs of power, comparatively (something like 50mA average, and that's with all the relevant power saving features enabled). By comparison, an nRF or an STM32W0 is single digit mA.
I'll buy it just because of this.
There are MIDI interfaces for phones. It is astonishing that there isn't an app which it's the checkboxes.
I was about to order but frankly it does not feel good to pay about the same amount of shipping (to Finland) as the product costs.
If you can do anything about it, I would be happy to order. 90EUR for shipping is just too much, 20-30'ish would be reasonable.
Yes I have a lot to share about the production process. Blog posts are planned! It was quite fun!
Works basically like this: You collect couple of shipments to one bigger package. Then you send that bigger package to Asendia logistics centre and they will send those individual packages to users.
So relatable.
Congrats, looks like a great product. I just ordered one for my piano-player buddy for Christmas.
It can also be a mood killer when rehearsing with a band. Everyone is messing about and having fun, then suddenly REC ON! and everyone is almost dead serious in performance mode.
If this launched 5 years ago, I would have immediately purchased it! At the time I was using pencil & paper, Sibelius, or my phone to record improvisations and all of them were very poor solutions.
I've reached the point where I use a DAW (Reaper) + MIDI keyboard + sound libraries. Conveniently, Reaper can record & display everything as editable MIDI output. Pencil + staff paper can be great if you're slowly exploring something though.
For the room
The worlds smallest midi synthesizer https://www.youtube.com/watch?v=KTvVS8guBsY
https://hpi.zentral.zone/flash
Why would you start with that? Particularly since some of the best "digital pianos" are actually just MIDI keyboard controllers (weighted and all that) and don't actually generate any audio themselves?
My understanding from OP's product is that there is an inherent assumption that this is targeted towards people dabbling on standalone digital pianos that also typically have built-in synthesis/sound modules.
Anyway, I'm just giving my reasons for why wouldn't you start with a software approach to answer your question. Putting all that aside, OP's solution is really cool and, independent of the decision to go custom hardware, it seems like a great product solution for the problem it's solving.
This device is a hardware solution. Hardware solutions are very popular in the music technology space, because nobody really wants to be screwing around with their laptop or whatever. A software solution that did the same means a computer that is running with a mic, preamp and audio interface at all the same times as the person produces sound.
The product in TFA has no dependence on the presence (or absence) of synthesis in the keyboard unit.
I'm suggesting that the user's phone that they already need (since the hardware pairs with the app in the article) is the hardware in this context. A phone already has a microphone and--if the environment is too noisy- many also support direct audio input through the phone's charge/data port with a simple off-the-shelf converter. Most smartphones can perform real-time audio progressing nowadays.
The trade-off is that midi is going to be lower error than an audio processing algorithm and much easier to implement out-the-gate, while it's certainly a step-up in the software engineering needed to develop and implement an audio processing algorithm to implement pitch/key detection as well as capture relative timing and velocity.
Correct the product in the article has no dependence on synthesis, but in pretty much all contexts in which someone will be using the device, there will be a corresponding audio signal present. Most people don't tap out music with a midi controller just in silence/without some sort of synthesis in conjunction.
Only way this works in software is if you dedicate a device to always record and audio processing doesn't provide isolation. Ambient sounds may get picked up or loud noises could drown out your playing.
This item works, because it is hardware. I also wonder how this would work connected to something like the sonuus G2M V3. Would be nice if the audio processing was solved by someone else.
This would also possibly widen the market for the product a little. Playing non-piano instrument sounds, even very technical synthy stuff, on a keyboard should “just work” assuming that all outgoing MIDI data like Program Changes and Control Changes are recorded and replayed. The further you get away from pure piano, the less likely the result will sound right on anything other than the original keyboard/piano.
Incidentally I'm curious if SysEx are recorded. Sometimes those are also used. For example some of the effect controls on certain Yamaha products use SysEx for historical reasons.
If you do add playback over MIDI you may want to familiarise yourself with some of the “reset” mechanisms to avoid hanging notes and other such problems, and make the app send them at appropriate moments. All Notes Off, All Sounds Off and Reset All Controllers are particularly useful “special” Control Changes.
thanks for your thoughts, I'll add midi out playback :)
and yes, head deep in midi knowledge by now but that is a good point about hanging notes.
To be clear it isn't in any way a deal breaker as the Zynthian's CPU isn't quite good enough to run Pianoteq in the best quality anyway, and moving some MIDI off to a DAW later to create a higher quality version seems like a great option — just it'd be really cool to be able to play things back the same way I played them initially :)
Just needs to let me "listen" through the midi out instead of the phone.
Caveat if course is, that I'd need to remember the synthesizer/settings/presets to have it sound the same again, but maybe half the fun to guess a bit and get happy accidents.
So if the synth supports MIDI CC, the settings would be 'replayed' :)
Is that to be inline with other IoT devices UX, or is there a technical reason like esp devices cannot be in AP mode unless initialized at boot?
In theory it should be fixable now.
Since most sustain pedals come through as a CC message I think it's supported.
- Can the out be used as a thru MIDI port? So I could just jam the device into my workflow without having to acquire a new MIDI splitter?
- What of MIDI it captures, exactly? I understand note on/off, velocity and such is obviously captured, but what about CC values and everything else? Would be wonderful to be able to hook this up to synths that have more controls, and be able to capture those values too
All midi messages are captured, on all 16 channels.
btw check out http://reddit.com/r/jamcorder for ongoing discussions.
Most interesting!
Did you write your own support for USB-MIDI or are you using some ready-made library?
I remember once hacking on my own project with USB-MIDI support and it was a hassle. I was constantly missing MIDI messages. Had to build a "panic button" in to force "stop all notes".
Although my project was using SAMD21 (Cortex M0+), so possibly it wasn't fast enough for the use case.
modern ESP-IDF: https://github.com/espressif/esp-idf/pull/12566
versus 3 years ago: https://github.com/chipweinberger/xesp-usbh
There's something so whimsical about this kind of tech. The cost to store the midi is effectively zero, it would be so interesting to see it installed within a keyboard - a journal chronicling the player's progress over decades, and everywhere the keyboard has been. I could see something like this being included in the next OP-1
Having to lug my laptop out, boot'up Bitwig, connect it to the piano, press record completely takes me out of the flow.
With this, I can flip on my Kawai and start banging away, then pull out the impromptu motifs for later refinement.
I assume I can just drop this guy in between my midi controller and my primary interface since it has MIDI I/O, yeah? Does it pass through?
But definitely not humanly noticeable.
Indexing. The problem you face after recording too much.
If that's not enough, stay tuned! more tools like grouping by chords, songs, melodies, faster navigation, & search, are planned!
Probably needs to have a song database, so it can index by song name.
[1]: https://loogguitars.com/collections/loog-piano
I have a very minor feature request: MIDI loops! Add a button on top of the device: Press once to record, press again to stop. Quantize on/off available via the app.
Awesome stuff!!!
are you using flutter and blue plus ??? because I see your post and someone on reddit asking about bluetooth then I get rerouted on same web page lol
its crazy coincidence because I open both page at the same time
Would love to see anything of the visualizer. Understand it’s beta, but curious what direction it would go
Looks like it does.
Is it recording 25,000 hours of actual audio which it analyzes, or is it recording midi data?
And I love the name.
Congrats on the launch and bravo on such a well-polished everything - product, UI, website, etc. Very impressive.
just remeasured it to double check.
I had a Fantom X6 and one feature I loved about it is that it was always recording in the same manner as this, you could just push the "Skip Back Sampling" button and then save or do whatever with it. It was also good because the audio samples were synced to the midi clock. A cool feature.
Up until now I've only known of some DAWs & custom raspberry-pi like solutions.
Though the Fantom X6, you have to push 'record' afterwards. Still cool, though!
Thanks for sharing!
If you want historical, it's also exposed over the web interface and wifi API.
just curious. Thanks.
i.e. http://jamcorder.local
It also has an open API, that's pretty simple. So you could integrate it into other apps or open source software.
Also: Less space than a Nomad.