Considering ISCOLCPUS reserved core implementation... Any drawbacks? #826
Replies: 8 comments 21 replies
-
|
Hi @josephpvincent. Welcome to MiniDexed. Do you mean ISCOLCPUS as in "port MiniDexed to Linux"? |
Beta Was this translation helpful? Give feedback.
-
|
Sort of unrelated, but I've also been looking at "beefing" up the YM2162 HDL implementation used in Mister FPGAs Genesis clone to effectively work as a DX7: Also been looking at playing with this one since the operator is laid out more "primitive-like" (and it's in VHDL which I'm more fond of): Would be a very good starting point for a minidexed-like implementation on an FPGA. I've had that YM2612 ported to Xilinx for a bit now. Resources on a small fpga suggested that we could run nearly 100 of them in parallel and I don't think it'd be more consumptive to do it with a DX7. Two ARM cores could run minimal Linux and manage MIDI, displays, networking, and any other controls. Not sure which way I'd like to go first, but I think a minimal Minidexed OS would be the quicker of the two to get to some great results and provide something beneficial to the community. Any thoughts on this? It would be a pretty different implementation on an FPGA, but I'd be happy to contribute for a truly hardware-based MiniDexed if it'd be of interest to the community. Is there anything in particular that limits two DX7's per core on RPi Zero2, RPi 4, or RPi 5? Or is there still some overhead left that could be squeezed? |
Beta Was this translation helpful? Give feedback.
-
|
It is 2 TGs on one core, then 3 TGs on each of two more for a pi 2, 3, Zero 2. The remaining core handles all IO. By the way, there is an implementation of Dexed on Zynthian that might be worth a look... Kevin |
Beta Was this translation helpful? Give feedback.
-
I've toyed with the idea of using buildroot, but haven't seriously given it a go yet. One benefit of an embedded Linux variant might be that it could allow other SBCs to be used. I've wanted to try an OrangePi or two for example, but it is pretty way down the todo list for me at the moment :) I'm more likely to get a new picoDexed going on an RP2350 first... I think a new port would have to be a new xxxDexed completely though. I don't see it being worth the effort of maintaining a fork of MiniDexed. If we were to start supporting alternative bases other than circle, then we'd either have to reimplement circle on another platform or introduce some kind of basic HAL which starts to negate the benefits of going bare metal in the first place :) Synth_Dexed is already used in MiniDexed (Rpi) and MicroDexed (Teensy) and my own experimental picoDexed (RP2040) so that seems the natural "porting" point to me. I know that last time it was raised Rene had no particular interest in porting circle to another SBC and I can say I blame them - it is too much of a dynamic, and often unsupported, path in my view. With the performance of a Pi 3A+ being pretty good for 8 TGs and further expansion to 16 with a 4 or 5, I'm not sure what the benefits of an FPGA would be (as cool as it would be to see, of course)? One interesting avenue I could see being really interesting might be to develop an effects processing chain outside of MiniDexed that could hang off the 4 I2S channels of a Pi 5 and give hardware accelerated DSP effects with clever mixing for all 8 TGs individually... Imagine an 8-channel, hardware DSP effects "HAT" - that would be pretty cool imo :) But again, with the headroom of a Pi 4 and 5, its hard to argue that this shouldn't all be done in software (see #795 for example). Kevin |
Beta Was this translation helpful? Give feedback.
-
|
MiniDexed doesn't even use Circle's fast=true mode yet, so the cores run at 600-700 MHz. There's still some room for improvement there. |
Beta Was this translation helpful? Give feedback.
-
I suppose in my opinion, another benefit would be that it's be a baseline framework for much more capable FM synthesis, but I agree that it may ultimately be overkill.
Ah, now I do believe that extreme low latency digital effects processing could be interesting on an FPGA implementation. I've been toying with the idea of making a hat with one of these exactly for that purpose - Not only could this act as a good audio DSP hat, but with the large amount of IO, I could also stick an HDMI RX on here and convert it to MIPI DPHY to give an HDMI input to the RPi to allow for real time video processing. Different application though. With some clever CDC and a high speed DSP clock I could see a lot of ability for accelerated DSP like you mentioned. |
Beta Was this translation helpful? Give feedback.
-
Circle does have certain graphical display support either via SPI displays or HDMI, but we've not gone down that route with MiniDexed. But Rene's minisynth uses HDMI with optional touch screen support: https://github.com/rsta2/minisynth Kevin |
Beta Was this translation helpful? Give feedback.
-
Noted, I just think that for good display support using nice GUI frameworks like Qt, it's much less effort to have a single core run it using traditional Linux and then port what's been done here to reserved cores for the speed aspect, but that's just me. :) |
Beta Was this translation helpful? Give feedback.
-
|
My 2 cents:
|
Beta Was this translation helpful? Give feedback.
-
|
Dexed using Zynthian has typical latency issues common with scheduled, but notably they do have a reserved core feature which does improve that. So I'd suggest researching that sort of implementation if you want to understand the usefulness more. It's still more limited than minidexed, though I'm not 100% certain why |
Beta Was this translation helpful? Give feedback.
-
Well, curiously, in the last half of a week that I've been discussing this back and forth with everyone on here, I've mocked together an FPGA passthrough hat. I've been able to design an I2C register interface in the FPGA which allows for resetting and control on certain parameters for now, The signal flow in the FPGA is: I've also added a SI5351 clock board which should allow for BCK, LRCK, and (optional) MCK generation. The total of all of this is less than $100, so I don't see how this would be a cost concern for most fiscally considering that one would gain an insane amount of headroom dsp wise. There's an infinite amount of processing that you cannot do with software at less than 2 sample latency. The audio coming out of my frankenstein'd proto is extremely clean considering how grotesque the signal integrity is. If you want some samples of audio I'm happy to provide. I can provide them I think "experimenting with FPGAs" sounds a little downplayish, but that's just me. There is no playing around when it comes to hardware. It either works or it doesn't. This FPGA module can be had for less than $25 so the "costliness" mentioned seems a little over exaggerated. In a response to the note of complexity, if I created a hat for this that could sell with all of the above for less than say, $70, with preloaded FPGA design and available image/driver, would that not ease the complexity burden? In less than a week I've brought up a proto type design in software and ASIC-level hardware that's functional and enables I2C control so I don't see what the fuss is about, granted I do FPGA design, custom linux design, baremetal driver design, and DSP systems for a decent living.
Noted.
I mean, I somewhat agree, but when considering a real time audio system and trying to maximize the ability of everything involved, I think choosing what the scheduler can and can't assign is a valuable tool, so I also somewhat disagree with that sentiment. For example, I think that enabling display, gui, and networking support on a kernel managed core or two is amazing, but controlling things like I2S control and signal processing on a reserved core to vantage determinism would be wise from both a DSP and hardware approach, There does exist a "best of both worlds" scenario that trumps whatever a fully linux scheduled approach would give. Porting what circle has is not going to be that crazy considering what something like Yocto would enable as far as custom distribution creation. Just seems like cherry picking some kernel level memory access to streamline everything. I design RF DMA pipelines at the ASIC and driver level so audio is somewhat trivial in comparison. I'm not going to starve a DDR bus with any of this, for instance. I understand the reluctance from a lack of experience perspective, but removing needs for work-arounds and pioneering in a clean way should be a developer's ambition too. Couldn't there be merit in dev'ing a system that's quite a bit more futureproof? |
Beta Was this translation helpful? Give feedback.
-
|
Looks like you have a lot of experience with FPGAs. Impressive! How do you think a DX7-like synthesizer would benefit from using one? |
Beta Was this translation helpful? Give feedback.
-
In the realm of DX7 being implemented on an FPGA, not sure that in and of itself it'd be extremely beneficial to a DX7 emulation due to it being an old hardware that used quite a lot of "hacks" to get it functional with the least amount. Also, because as has been stated, there are 8 now running on a baremetal Pi implementation, so that seems pretty excellent on its own. If a DX7 were implemented on an FPGA, I probably wouldn't suggest trying to make it a hardware exact emulation and focusing on making a baseline FM synthesizer which is arranged similarly to a DX7 while removing the aforementioned "hacks" that were necessary to the hardware of the time since there isn't the same sort of limitation there. That'd then act as a good basis for a more monstrous FM synth (think 128 voices per synthesizer per each of 8 DX-7s at 192kSample, maybe things like cordic implementation instead of the sine-log rom, etc). Would it be a picture perfect DX7? No, but it'd be an upgraded form that should be able to readily take DX7 presets. Would also be a great foundation for different forms of synthesizers. I think an FPGA could do an amazing (better than a CPU) job at emulating an Analog synthesizer, as well. If I already had a good FM synthesizer implementation in an FPGA, then I'd be looking at doing a MOOG or a Juno clone with one, because truly good emulations of analog components in real time is typically pretty DSP heavy. More useful and quick than this implementation, I'd think that the hat I've proto'd could find some use in post-processing. There could pretty easily be made an implementation that vantages I2S time division multiplexing to sample the 8 DX7s here and give them each there own post effects signal path which ends in a mixer at the output. On each path, with just the FPGA, we could do chorus, multi stage filtering, tremolo, bit-crushing, overdrive, flanger, additional modulators, amongst other less memory intense effects. Could easily also do a Vocoder effect too. I want to say that each voice could ultimately be sent separately too, but the datarate for that is too high for I2S alone, even with multichannel I2S devices from a PI 5 ( If minimum 48kSample for HiFi and we have 8 channel streams at 192k, then even with some TDM, that'd only allow for 4 voices per channel so 32 voices, but I haven't looked at the Pi 5 yet to verify any of this). It might then need to take a fancy implementation of some SPI streams in parallel to make that functional as well as some packet structuring to help with aligning samples properly. With the addition of an SRAM module for extra filtering, like this we could do various delays, convolution reverbs, looping, IR modeling, and more memory intensive effects pretty easily as well. The most obvious benefit here is going to virtually no latency while giving the RPi more ability to do general purpose processing that it really excels at. Also should note that while there would be a limit for how many effects could be chained, I'd think that per chain the options for arrangement would be far greater than a Pi could handle. Especially considering parallelizing/pipelining DSP operations such that there can be 10 or more DSP operations in a single 200-400MHz cycle. In addition, the FPGA could more easily allow for expanding interface support to the Raspberry Pi without impacting the performance of the Pi's own DSP work. If I can bump that DSP clock to 400MHz reliably, which that FPGA's internal PLL allows - Num of Single Operations on stereo channel between samples- Now say there are 8 FIR filters in the signal path with 100 taps - Now say a reverb and delay both take 200 cycles each as a conservative number - Now say each remaining effect takes something in the order of 20 cycles to do correctly as a conservative number - 2966/20 = 149 remaining effect slots 8 Fir Filters + 1 Reverb + 1 Delay + 149 remaining effect slots = 159 effect slots in a chain To be conservative again, I'll take a fourth so ~ 40 effect slots on a single channel. This is all imagining a single channel with a single multiplier. That FPGA I'm using has 28, so this should theoretically be able to scale to 28 stereo channels with 40 slots per stereo stream (40 for right and 40 for left - 56 mono channels). Will likely be even more than that since I'm playing it conservative, but also memory resources will be at play. If both the DX7-like synthesizer and the effects were stuck together, I'd conservatively assume you could do 27 stereo channels at the same 40 slots per stereo stream. Maybe a few less channels if there's a cordic implemented. For wavetable type synthesis I could also see a benefit of being able to grab incoming audio on an input ADC for different tables. Might also be advantageous to be able to do an effects chain there for preprocessing in that case or in the case of creating a general multi effects I/O. Add a Multiplexer to a faster ADC and you could also easily do multiple CV channels in the case of a eurorack style unit. Nothing unique to an FPGA there, though, most microcontrollers would be fine with that on their own. |
Beta Was this translation helpful? Give feedback.
-
|
I'll also add that there is a DX7 on an FPGA now with 64 voices - Sounds good from the demos I've seen, but I'm positive that it's not even close to fully optimized. It's entirely closed source so I don't have a means to gauge what he's done here exactly, but in porting the YM2612 chip before, there was so much headroom that there's no way he's filled up that Artix A35T. Going way back to my original link to the Zynq board that has a build in processor and some DDR, if minidexed was ported to that $45 board, I could easily see much better acceleration ability due to the DDR being exposed to the FPGA through DMA ports. I've been able to get 400MBytes per second through a processor to FPGA DMA on a similar board. I could do more than that, but I didn't want to start starving the processor of the DDR bus and causing unexplained behavior. That board has an equivalent FPGA on it to the XFM2 I linked has, as well. A final note - I've been able to program the FPGA on that hat I made from the Raspberry Pi via Linux using OpenFPGALoader and the IO pins acting as JTAG. That means there could be multiple designs for an FPGA hat like this each offering different features, like patches. |
Beta Was this translation helpful? Give feedback.
-
|
Adding that I just got a Yocto Minimal Distribution made for the Pi Zero 2W which boots relatively quickly and I'm working out an I2S DMA driver recipe that bypasses ALSA and PulseAudio. My plan is to signal to userspace when the DMA buffer is getting close to empty to let a userspace app fill more samples into the kernel space buffer. This has worked best for me in DMA drivers before so I'll let everyone know when I get around to making more progress. |
Beta Was this translation helpful? Give feedback.
-
|
Here's an early look at progress on a FPGA hat board with audio in/out via a nice codec that I've worked with before. Definitely going to be making a couple of changes with cap sizes when I pick parts but it's looking nice thus far. It's tight, so I don't have room for an encoder/controls without potentially compromising the analog side, but MIDI is an option. |
Beta Was this translation helpful? Give feedback.
-
|
Hello! I'd consider adding full MIDI IN-OUT-THRU, perhaps on 3.5mm jacks. It seems useful considering that the MIDI Thru functionality of MiniDexed allows to use it also as a USB-MIDI host or simple router between traditional and USB MIDI. |
Beta Was this translation helpful? Give feedback.
-
Thanks for the input. I'll check and see that I can do that with the extra space from replacing the MIDI jack. I also want to add an SRAM at a further revision on the bottom side of the board. Won't make it in here, but it'd be excellent for the extra RAM needed for fancier DSP effects. If I could add two, even better. It's an interesting board design, because even without the RPi, this could be completely functional from the FPGA alone. Cost is going to get trickier the more I add, though. I'm targeting to be able to sell something like this for less than $100, but it's going to be hard to justify if pcb + assembly ends up being more than something like $60. I can procure that FPGA SoM for around $15 and the rest is to be determined. Will continue updating. |
Beta Was this translation helpful? Give feedback.
-
|
It'd be a heck of a development kit though. This would apply to so much more than just minidexed in the long run the way it's set up. I can't think of such a powerful kit with as many options going for less than $300, so. It's like all of the higher end HiFi Berry feature set but with extreme co-DSP processing and MIDI capability. My thoughts are that it should not be too hard to build a mainline driver for something like this, but additionally offering a custom distribution catering to DSP performance would be quite an extra bonus. |
Beta Was this translation helpful? Give feedback.
-
|
And here's with MIDI IN and a THRU/OUT through 3.5mm jacks. Also considering now moving the FPGA to the right just enough to allow for a right angle 40 pin breakout so instead of hard placing a SRAM, one could use the Sipeed SRAM PMOD breakout. If it ends up being not possible, then I have plenty of unused space on the bottom of the board I could leverage and just place a single SRAM chip down there. The 20k primer board has built in DDR, so that's also curious for another board later down the line. Similar in cost to the 25k I'm using, but it's quite a bit larger. Definitely wouldn't fit on a hat board since it's a sodimm, so I'd likely be looking at a compute module board if that were the case. Giant 40 pin connectable boards tend to be quite awkward as far as the Pi mounting is concerned. The exception would be a Pi Zero, which would allow for mounting it like a hat to the board by soldering the header socket to the bottom of the Zero. In such a case I may at the very least add compatible mounting holes for Pi 4/Pi 5 so they could be attached via a ribbon cable. A bit to consider. |
Beta Was this translation helpful? Give feedback.
-
|
I really don't know much about the topic at all, but from time to time I hear about "MiSTer FPGA". Would that be suitable hardware? |
Beta Was this translation helpful? Give feedback.
-
|
Yeah, so Mister is a platform that means to emulate old gaming platforms on an fpga. I have worked on some of that project on a minor scale. The issue with it is that it’s focused on an Altera platform that keeps it stationary due to the platform involved. Mister fpga itself is not an FPGA. It’s just an ecosystem. The cheapest build for a Mister FPGA platform is near $500c , so to answer your question, no I don’t find that a relevant ecosystem at all. It’s both expensive and not geared towards what would be efficient, so I think even with some basic research into the matter, you’d see quickly that it’s not a great option for state/hardware for what I’m describing |
Beta Was this translation helpful? Give feedback.
-
|
Furthermore, mister fpga compatible boards have jumped heavily in price, so it’s a real prime time to develop a cheaper and different system |
Beta Was this translation helpful? Give feedback.
-
|
Heya! Been a while since I last posted but I just wanted to add that I did do a spin of that board with my company and are now finishing up on a production revision that will be for sale. Have got Minidexed working with it and additionally have been able to record audio input with the codec driver I've written for circle. Only have a master EQ implemented in the FPGA as parallel cascaded biquad sections for each of the 10 bands, so not really utilizing the FPGA a lot yet, but it is a great first step with this all. I could imagine some really interesting use cases for this board both using an OS and in bare metal with circle. I've decided to pass through the IO in the final revision such that compatible hats can be stacked for displays and controls. Currently working on an add on hat that enables 4 channel Gate/CV in and 4 channel Gate/CV out, so this could be a good platform for folks wanting to convert CV -> Midi or the inverse. An added bonus is that the codec includes Programmable Gain Amplifiers on the audio inputs and outputs, so a user can adjust levels for input audio as well as outgoing audio. Would be a great platform to implement a Synthesizer, a effects pedal, a USB audio interface, an audio profiler, and likely more. We're looking to sell this retail for ~$170, so it's not going to be super cheap unfortunately, but when you consider that the evaluation board for that codec is $180 and a less capable hifiberry hat is >$100, we think it's a fair price considering the capabilities of the hardware as well as what it costs for us to manufacture. It isn't easy to make this hardware, write the OS/baremetal drivers to support them, and provide an examples/an ecosystem for the FPGA side of the operation.
If anyone's curious, I have a repository set up here which I'll be adding resources and source to as my team and I prep this for release. I'll likely post a demo video up with this running Minidexed as well as Zynthian when the new boards come in and I've verified full functionality. Hope you're all doing well! |
Beta Was this translation helpful? Give feedback.
-
|
Wow that is impressive @josephpvincent. Hats off! 👍 |
Beta Was this translation helpful? Give feedback.
Uh oh!
There was an error while loading. Please reload this page.
Uh oh!
There was an error while loading. Please reload this page.
-
Hi, I'm an FPGA/firmware developer who runs a small firm that focuses on DSP applications. Long term musician and have really come to love the minidexed project. I'm actually considering a port for an FPGA so parts of the processing can be handled in live hardware and effect support consideration. Xilinx FPGAs with built in ARMs have been getting much cheaper over the last few years, but I digress.
I'm considering doing some porting to get this functional in tandem with a Linux OS using iscolcpus to isolate CPU cores. If we isolated 3 cores from a Pi Zero2, for example, that'd allow for 6 DX7 emulations on the remaining 3 cores. Still a very nice amount. I know that there are some small issues with interrupts causing small delays in this arrangement, but I believe we would still see a similar real time benefit that we see here using circle. Trust that I understand that it'd be quite a porting task, but this would then allow for beautiful, up to 4k full color displays with nice development tools over either HDMI or the native display port. IPC could readily be handled using UNIX sockets.
Has anyone here considered this before and ran into any issues? Would love to hear the community's opinion on the matter and if something like this would be feasible/interesting.
Beta Was this translation helpful? Give feedback.
All reactions