Dreams Audio Tech

Dreams Audio Engine

Dreams contains some powerful tools to help you understand the audio performance of your creations. Most creators won't need to bother with them, but for those who want to get extra power out of the engine, they might come in useful! You can use the tools to figure out why some of your sounds might be cutting out, getting distorted or playing with a delay.

First off, let's take a dive into the audio engine technology in Dreams. Sound in Dreams usually comes from sound gadgets in your scene. When a sound gadget is triggered by logic, it will create an instance of itself. It's possible for a gadget to create more than one instance at the same time if, for example, it's triggered again before its previous instance finished playing.

Each sound instance can then create a number of voices. A sound effect will generally trigger one instance, with one voice - the voice is what is actually playing the audio sample data. An instrument clip will create one sound instance, which will create multiple voices, one for each note in the clip. Here’s an example of what that flow looks like and what can be included in these units.

Going even further, Dreams uses a granular engine, and each voice actually plays the audio in individual lumps called grains. These are a bit like the flecks used in Dreams’ graphics!

The Audio Analyser

This tool lets you see how much of these resources your creation is using at any point. You’ll find a helpful video about the audio analyser right here!

Run analyse audio and play your creation, and the tool will gather information over time, showing you the numbers involved as you play. Exit the tool and you’ll get the analysis overview (pictured below). This is a report representing the data from your playthrough on a graph, on which you can see which parts of your creation are expensive. You can scrub through the graph to any point in time, and the image to the right of the graph shows you what was happening at that time.

The most important statistic here is audio cost. This is the combined cost of voices, instances, grains and all the effects such as reverbs, delays, compressors, and so on. The closer this gets to maximum, the more the engine will start cutting out voices to avoid hitting 100% - you'll usually get a warning if this happens. If the cost reached 100%, Dreams wouldn't be able to output audio properly, so it’ll do everything it can to avoid that.

Instances and voices have limits, and if you hit these, new sounds may fail to play, or ones which are already playing may be cut out. The limit for instances is 128. For voices it's a bit more complicated, but we've made the graph cap out at 128 as well. You could, in theory, have more voices running, but usually they will have a big impact on performance above that number, depending on which system you’re running Dreams on.

The more voices and instances that are running, the more the audio cost will increase. By looking at the graph you should be able to figure out where you need to optimise.

Note that not all voices are created equal. The granular engine means that some voices may be playing with more overlapping grains than others, and therefore cost a lot more. The grain cost stat is an average of this cost from all the playing sounds in the scene. Most voices have an average grain cost of 1x, but it can go up a lot if you play with the granular synthesis and granular randomisation tweaks on the sound gadgets. The diagrams below illustrate examples of expensive and cheap grain overlap settings.

You can set the grain overlap on the granular synthesis page in a sound’s tweak menu. This tweak also appears on the granular randomisation page, where a number of other sliders can also affect grain cost.

Voices can also become more expensive if you’ve activated sound features like the oscillator, distortion, dynamics compressor, and so on. The cheapest voices are usually simple sound effects, playing with no exciting tweaks employed.

So how can you use this information to optimise your creation? The most common thing you'll want to do is lower the polyphonic limit on your sounds. This is a limit on the number of voices that can be created by a sound. If done carefully you might not even notice a difference. Here’s a couple of examples where polyphony reduction might be useful:

• A music clip that plays many very long notes over each other, such as a piano or a synth with a long release envelope.
• An explosion or weapon sound that plays over itself by being triggered in very quick succession.

Polyphonic limit normally defaults to 20, which means that most sounds can play 20 instances of themselves unnecessarily, for example if triggered repeatedly very rapidly.

Another option is to look at sounds that have large grain overlaps and reduce them, although bear in mind this will usually impact the cool effect those sounds were using granular options to make.

You might also want to head to the sound fx page to turn some or all of these off, which should also help make most of those sounds cheaper.

And of course, ultimately, if nothing else helps, you might want to just look around your scenes for sounds that could be deleted altogether.

Audio Thermometer and Sound Readiness

Let's talk about the Audio Thermometer! But first, let’s look at the engine again.

Dreams audio comes in samples, which is cut into slices. Samples are what your microphone recordings create, and slices are what you see in a sound's slice mapper.

There is a limit on the total number of unique samples (referred to as Unique Sounds in the thermometer) and we also have a limit on the amount of memory these samples use.

For example, you could run out of samples by having a large number of very tiny recordings, or you could run out of memory by having only a few very large, long recordings. The other limit is on the amount of custom imported or recorded sounds. There is a very useful thing to understand here that underpins and motivates the entire Dreams audio system:

All MM audio content, like instruments or SFX from the collections, has 0 download cost.

Audio data is usually very expensive in storage and download times, so the inclusion of a standard sample set in Dreams is part of what makes coMmunity creations loading extremely fast. Whenever you use audio content uploaded by yourself or others, it’ll impact the loading time of creations, for example making level transitions longer and not as seamless. This is much worse if you have imported long audio samples, like entire songs.

For this reason, it’s always much better use music created in Dreams using the Mm Instruments or derivatives of them – in many cases the Mm sample library is more than enough to create a large variety of sounds using the built-in effects. At the very least, if you’d like to keep your creations loading very quickly, you should make sure your audio imports are as trimmed down and reused as much as possible.

It’s important to understand that trimming your samples in a sound's slice mapper won't reduce any of these costs. Samples themselves can't be changed after recording, so you should always aim to import them as close as possible to how you want to use them. If you're importing a file into Dreams which will be sliced into a sound, make sure you've already edited it to remove as much silence or un-needed audio as possible. If you hit any of the limits you won't be able to publish or save your creation.

Here’s an example of two sounds and their costs. The one on the left uses two slices from two different samples. The one the right uses two slices from just one sample. The one on the right is cheapest, but it could’ve been made cheaper by trimming out the unused bits before import. The one on the left is pretty wasteful! At Mm we prefer to create SFX and instruments with slices cut out of one sample, to minimise the overall cost.

There is a very important limit which has always existed but wasn’t always exposed or explained. It's called sound readiness.

For a sound to play when triggered, its audio data must be "loaded". If audio data isn't ready, there will be a delay when the sound is triggered, which is very bad in cases such as music or voice recordings which are meant to be in time with animations.

But as much as we love audio, if we loaded every single slice in every single sound in advance, we wouldn't have any memory for all the cool other parts of the game, so the engine is constantly compromising. We call this system pinning - each sound tries to ensure that at least one of its slices is ready to play at any one time. This helps, but if you have too many sounds in your creation all trying to load different bits of audio, you’ll run out of memory too, which means the problems with delayed starts will crop up again occasionally.

Previously in Dreams this would happen silently, but now you'll get a warning and can also use a special tool to figure out which sounds are the culprits.

For your creations, especially ones that are meant for others to stamp in their levels, you should always try to minimise this thermometer cost - no one wants to stamp your element and find out it makes their voice lines play late! If that happens though, they can also optimise by cutting other things. Let's take a look at how.

First of all, in Sound Mode there is a thermometer bar that will appear when the sound readiness cost goes up. We added the show/hide menu to Sound Mode so you can make the thermometer always visible. Normally though, you’ll only ever have to worry about it if you’ve hit the sound readiness limit. If you click that bar, you’ll end up in the sound readiness thermometer, where you can select more details to look at a breakdown of the 20 most expensive sounds in the scene. Remember – it’s only in terms of sound readiness cost!

Also included is a breakdown per sound channel. You can select any of the entries in the list, and you’ll be taken to the location of that sound in your scene (you might need to dig it out of a microchip or timeline). You’ll also get a blue flashing indicator showing where the sound is for a while.

As you may notice, some sounds are duplicated in the list. If you want to free up that sound readiness budget, you’ll have to deal with the duplicates, since all of them are using up memory.

What you might do next is either delete the most expensive sounds or trim them down in the slice mapper. Trimming doesn’t help audio cost, but it does help sound readiness cost, as you only pay the cost of one slice rather than an entire sample. Here’s an example of two sounds using different parts of one longer sample.

The cost comes from the part of that sample that both slices use – it won’t count the shared bits twice!

Very commonly, sound readiness issues will be caused by unnecessarily long sounds. Here’s an example. There’s an ambient background sound here – Forest Day – which is using 5.24% of the budget, almost as much as the entire song that’s also in this scene! In this case we’re far under budget, but if we needed to, we could probably trim that Forest Day sound down in its slice mapper.

So now it’s time for another tip: imported audio like long pieces of pre-recorded music will very quickly and inefficiently use up the sound readiness budget, and it’s part of why using music made inside Dreams is always much more effective for keeping the audio running smoothly.

That just about covers it. We hope you find these features useful. Bear in mind there are a lot of edge cases, caveats and tricky bits to look out for though, such as…

• Each sound only readies one of its slices (But! See below!), so when there’s more than one you might need to do a bit of exploring to find out which one it is.
• Except for instruments with row mapping turned on! Those will ready one slice for each row in their piano roll that is occupied by notes. That means they can get pretty expensive!
• Sounds that use almost the same parts of a sample but not quite will register their cost as being whatever bit of the sample they use, even if it’s shared. This is so that you can see how much they need to be readied. However, as mentioned above, shared bits of audio don’t double up in the total cost. Basically, sounds in the list become separate entries when they’re trimmed differently, even if they use almost identical audio data.
• There’s no current way to choose which slice should be readied, but who knows what the future might bring!