Monthly Archives: September 2019

Oscillators: Introduction

Oscillators


The most basic sound-generating process in digital sound is the oscillator. An oscillator creates a periodic waveform of some shape (sine, square, saw, triangle, etc) and repeats it in time. The number of times it repeats per second is its fundamental frequency. This is measured in units of Hertz (Hz) or cycles per second. The reciprocal of frequency is the period, or the number of seconds per cycle.

As a concept


Unit Circle

It is helpful to visualize a sine oscillator as a wheel and spoke.

Unfasor

The angle of the “spoke” is the phase at a given moment in time while the frequency are the number of rotations per second. The length (magnitude) of the spoke is the amplitude at any given point.

Phase

The phase of a waveform, often denoted with the Greek letter phi (Φ), is a means to define a specifc point on a waveform. Phase change is measured on a unit circle, that is, a circle with a radius of 1.0. As the circumference of the circle is 2πr, one rotation has 2π of phase. The unit used to measure phase is the radian.

Sampled Signals

When programming digital audio, we will be working primarily with sampled waveforms. A waveform is generated by calculating samples at a certain (and usually a fixed) rate. This is the sample rate (seconds per sample).

To convert time to a number of samples, simply multiply the time in seconds by the sample rate. For instance: an oscillator at 200 Hz has a period of 1/200 (or .005) seconds. To find the number of samples for this period with a sample rate of 44100; multiply 44100 * .005. The answer is 220.5 samples.

Adding the USB OTG MIDI Driver – Polyphonic framework

The micro USB port on the STMF4Discovery board is a OTG (“on the go”) USB port. This means it can function as both a slave or master USB port. To use it as a MIDI host (so we can use MIDI keyboard and fader controllers) we need a OTG adapter. Here is one from Amazon. These are typically used to attach flash drives to Android phones. The STM32Cube has an example of USB Host and Device code in the Middleware folder. However, instead of using that to develop a class-compliant MIDI host, I have simply adapted Xavier Halgand‘s USB MIDI host code to work with the current version of the STM HAL libraries. Xavier is the author of the wonderful Dekrispator synthesizer for the STM32F4Discovery board.

My MIDI framework with a basic (terrible sounding) polyphonic synthesizer can be downloaded here: f4disco-midi. You should be able to to unzip, place it in your workspace folder, and import to STM32CubeIDE.

You will see a number of things added to this project, including a struct and enum to support each synthesizer voice. Each voice represents an independent sound generator, and these can become fairly complex. Our voice will be a simple, unaliased, sawtooth oscillator with a fixed ASR gain envelope. It requires the following for each voice:

typedef struct synthVoice
{
    int active;
    int note;
    int velocity;
    float volume;
    float frequency;
    float phase;
} synthVoice;

typedef enum
{
    INACTIVE = 0,
    ATTACK,
    DECAY,
    SUSTAIN,
    RELEASE
} voiceState;

The active variable will indicate the various states that each voice can be in, these states are in the following enum. The remaining members of the synthVoice struct are synthesis parameters. Later you will see 16 synthVoices being allocated

synthVoice voice[16];

In the function main(), the USB MIDI driver is initialized, and then two functions are called in the main while loop. The function MIDIApplication() parses any MIDI received through USB, and the function USBH_Process(&hUSBHost) maintains the USB connection – it allows plugging and unplugging of USB devices.

/*## Init Host Library ################################################*/
 USBH_Init(&hUSBHost, USBH_UserProcess_callback, 0);
/*## Add Supported Class ##############################################*/
 USBH_RegisterClass(&hUSBHost, USBH_MIDI_CLASS);
/*## Start Host Process ###############################################*/
 USBH_Start(&hUSBHost);

// Initialize and start audio codec
BSP_AUDIO_OUT_Init(OUTPUT_DEVICE_HEADPHONE, 90, 48000);
BSP_AUDIO_OUT_Play((uint16_t *)codecBuffer, 128);
 /* USER CODE END 2 */
/* Infinite loop */
 /* USER CODE BEGIN WHILE */
 while (1)
 {
     HAL_Delay(1);
     /* USER CODE END WHILE */
     // MX_USB_HOST_Process();
    /* USER CODE BEGIN 3 */
    MIDI_Application();
    /* USBH_Background Process */
    USBH_Process(&hUSBHost);
 }

MIDI_Application() calls a function I wrote called ProcessMIDI(). This processes one MIDI message, identifying the byte code for the type of MIDI message, and copys the data from the message. I have implemented the parsing for “note on”, “note off” and “controller change”. Framework for other MIDI messages (status type), is included. To parse all of the types of MIDI message, you will need to refer to the MIDI Specification (easily found online).

void ProcessMIDI(midi_package_t pack)
{
	int i;
	uint8_t status;

	// Status messages that start with F, for all MIDI channels
	// None of these are implemented - though we will flash an LED
	// for the MIDI clock
	status = pack.evnt0 & 0xF0;
	if(status == 0xF0)
	{
		switch(pack.evnt0)
		{
		case 0xF0:	// Start of System Exclusive
		case 0xF1:	// MIDI Time Code Quarter Fram
		case 0xF2:	// Song Position Pointer
		case 0xF3:	// Song Select
		case 0xF4:	// Undefined
		case 0xF5:	// Undefined
		case 0xF6:	// Tune Request
		case 0xF7:	// End of System Exclusive
			status = runningStatus = 0x00;
			break;
		case 0xF8:	// Timing Clock (24 times a quarter note)
			HAL_GPIO_TogglePin(LD5_GPIO_Port, LD5_Pin); // RED LED
			break;
		case 0xF9:	// Undefined
		case 0xFA:	// Start Sequence
		case 0xFB:	// Continue Sequence
		case 0xFC:	// Pause Sequence
		case 0xFD:	// Undefined
		case 0xFE:	// Active Sensing
		case 0xFF:	// Reset all synthesizers to power up
			break;

		}
	}

// MIDI running status (same status as last message) doesn't seem to work over this USB driver
// code commented out.

//	else if((pack.evnt0 & 0x80) == 0x00)
//		status = runningStatus;
	else
		runningStatus = status = pack.evnt0 & 0xF0;



	switch(status)
	{
	case 0x80:	// Note Off
		// turn off all voices that match the note off note
		for(i = 0; i<16; i++)
		{
			if(voice[i].note == pack.evnt1)
			{
				voice[i].active = RELEASE;
				keyboard[voice[i].note] = -1;
			}
		}
		break;
	case 0x90:	// Note On
		if(pack.evnt2 == 0) // velocity 0 means note off
			// turn off all voices that match the note off note
			for(i = 0; i<16; i++)
			{
				if((voice[i].note == pack.evnt1) && (voice[i].active != INACTIVE))
				{
					voice[i].active = RELEASE;
					keyboard[voice[i].note] = -1;
				}
			}
		else
		{
			// if this key is already on, end the associated note and turn it off
			if(keyboard[pack.evnt1] != -1)
			{
				voice[keyboard[pack.evnt1]].active = RELEASE;
				keyboard[pack.evnt1] = -1;
			}
			// find an inactive voice and assign this key to it
			for(i = 0; i<16; i++)
			{
				if(voice[i].active == INACTIVE)
				{
					voice[i].active = ATTACK;
					voice[i].note = pack.evnt1;
					voice[i].velocity = pack.evnt2;
					keyboard[pack.evnt1] = i;
					break;
				}
			}
		}
		break;
	case 0xA0:	// Polyphonic Pressure
		break;
	case 0xB0:	// Control Change
		switch(pack.evnt1) // CC number
		{
		case 3 	:
			break ;	// tempo
		case 7 :
			break;	// master volume
		}
		break;
	case 0xC0:	// Program Change
		break;
	case 0xD0:	// After Touch
		break;
	case 0xE0:	// Pitch Bend
		pitchbend = pack.evnt2 * 128 + pack.evnt1;
		break;
	}
}

My implementation of ProcessMIDI() doesn’t do voice allocation (though it does look for inactive voices before assigning a new one), and isn’t doing anything with the data from CC (controller change) messages. There is lots of room for improvement, this is just intended as a starting point.

The audioBlock() audio callback function has been modified to generate up to 16 voices, each synthesizing a harsh sawtooth wave. The data set in ProcessMIDI() is being used to set the frequency of each voice, and to manage the ASR envelope in each voice. Note that when a note off is received by ProcessMIDI, the voice is not made inactive. Instead, the voice is put in the RELEASE part of the envelope, and is only made INACTIVE when the envelope reaches zero.

void audioBlock(float *input, float *output, int32_t samples)
{
    int i, v;
    float increment1, increment2;
    for(i = 0; i < samples; i += 2)
        output[i<<1] = output[(i<<1) + 1] = 0.0f;
    for(v = 0; v < 16; v++)
    {
        if(voice[v].active != INACTIVE)
         {
             voice[v].frequency = mtoinc[voice[v].note];
           for(i = 0; i < samples; i += 2)
            {
                voice[v].phase += voice[v].frequency;
              if(voice[v].phase > 1.0f)
                    voice[v].phase = voice[v].phase - 1.0f;
                output[i<<1] += ((voice[v].phase * 2.0f) - 1.0f) * voice[v].volume * 0.125f;
                output[(i<<1) + 1] = output[i<<1];
              if(voice[v].active == ATTACK)
                {
                    voice[v].volume += 0.02f;
                  if(voice[v].volume >= 1.0f)
                        voice[v].active = SUSTAIN;
                }
              else if(voice[v].active == SUSTAIN)
                    voice[v].volume = 1.0f;
              else if(voice[v].active == RELEASE)
                {
                    voice[v].volume -= 0.0002f;
                  if(voice[v].volume <= 0.0f)
                        voice[v].active = INACTIVE;
                }
            }
        }
    }
}

This code should be useful as a starting point for synthesizer creation. Those creating drum machines or effects processors will want to handle MIDI messages differently.

Setting up the STMF4 to connect to the CODEC and make sound

For this next change we will need to add some source code which has been developed to support the circuit board and the specific devices on the board. First, in the IDE, you will need to create a folder in the “Drivers” folder. Name this new folder “BSP”. Now we will copy some code into “BSP.”  When you started the IDE it should have downloaded the board and chip support code. Look for the folder STM32Cube and under that Repository:STM32Cube_FW_F4_V1.24.1. Under that you will find Drivers:BSP. This stands for board specific package. Find the folder for STM32F4-Discovery and make a copy of it in your project by dropping the folder on your “BSP” IDE folder – select “Copy…”

You now need to do the same for BSP:Components. Drag and drop “Components” on “BSP”. Finally, there is a folder with code that supports the on-board microphone. It is Middleware:ST:STM32Audio. Drag and drop “STM32Audio” on the IDE folder “Middleware:ST”. When done, your project will look like this

Screen Shot 2019-09-19 at 3.22.58 PM

Once this is done, we should be able to Build the project and have no errors.

Making sound

To make sound we need to call functions to initialize the CODEC, start the CODEC and we need to periodically call functions to compute samples for the CODEC. The BSP functions in stm32f4_discovery_audio.c make the first two steps fairly simple. To initialize the CODEC call the functions:

BSP_AUDIO_OUT_Init(OUTPUT_DEVICE_HEADPHONE, 90, 48000);
BSP_AUDIO_OUT_Play((uint16_t *)codecBuffer, 128);

after  “/* USER CODE BEGIN 2 */.” We also need to a include statements on the top of main.c so that main.c can see the declarations of the functions and macros we want to use. Add:

#include "../Drivers/BSP/STM32F4-Discovery/stm32f4_discovery_audio.h"

after “/* USER CODE BEGIN Includes */.” Finally, we need to define the array “codecBuffer”. We can declare it as a global (as well as globals for the I2S serial device) after “/* USER CODE BEGIN PV */”

int16_t codecBuffer[64];    // 32 samples X 2 channels
extern I2S_HandleTypeDef       hAudioOutI2s;
extern I2S_HandleTypeDef       hAudioInI2s;

Finally, we need to create five callback functions which will be called whenever the CODEC needs more data. The first two handle interrupt signals from the I2S serial device on the STM32F4. These call the DMA processor whenever the I2S device has received or tranmitted data from/to the CODEC. The DMA processor then calls one of the next two functions when it has transmitted a certain amount of data. This is done every half block, so that we have time to fill one half block while the other is playing. At the moment we aren’t doing any synthesis, so we won’t hear anything coming from the CODEC headphone port

void I2S3_IRQHandler(void)
{
     HAL_DMA_IRQHandler(hAudioOutI2s.hdmatx);
}
void I2S2_IRQHandler(void)
{
    HAL_DMA_IRQHandler(hAudioInI2s.hdmarx);
}

void BSP_AUDIO_OUT_HalfTransfer_CallBack(void)
{
}

void BSP_AUDIO_OUT_TransferComplete_CallBack(void)
{
    BSP_AUDIO_OUT_ChangeBuffer((uint16_t *)codecBuffer, 64);
}

void BSP_AUDIO_OUT_Error_CallBack(void)
{
    /* Stop the program with an infinite loop */
    while (1) {}
}

Now we can create a very simple sound synthesis function – two sawtooth generators. The synthesis function is called from the audio out callbacks. After the first half is complete, one should fill the first half with new samples. After the full buffer is complete, one should fill the second half with new samples. These samples are converted from float to integer by multiplying them by the maximum 16 bit value of 32767.0.

void BSP_AUDIO_OUT_HalfTransfer_CallBack(void)
{
    int i;
     audioBlock(inBuffer, outBuffer, 16);
    for(i = 0; i < 32; i+=2)
    {
        codecBuffer[i+0] = (int16_t)((outBuffer[i]) * 32767.0f);
        codecBuffer[i+1] = (int16_t)((outBuffer[i+1]) * 32767.0f);
    }
}

void BSP_AUDIO_OUT_TransferComplete_CallBack(void)
{
    int i;
    BSP_AUDIO_OUT_ChangeBuffer((uint16_t *)codecBuffer, 64);
    audioBlock(inBuffer, outBuffer, 16);
    for(i = 0; i < 32; i+=2)
    {
        codecBuffer[i+32] = (int16_t)((outBuffer[i]) * 32767.0f);
        codecBuffer[i+33] = (int16_t)((outBuffer[i+1]) * 32767.0f);
    }
}

A few global declarations need to be added at the top of the main.c for the buffers and synthesis parameters.

// Sound globals
void audioBlock(float *input, float *output, int32_t samples);
float saw1, saw2;
float inBuffer[32], outBuffer[32];

The audio synthesis function audioBlock will calculate the 2 sawtooth waves, and keep the result in saw1 and saw2. A sawtooth is simply an increasing signal which gets reset to a minimum sample value when it exceeds a maximum sample value. In digital audio, we typically use the minimum and maximum values of -1.0 and 1.0. The amount of increase each sample is dependent on the frequency. The sawtooth needs to rise a value of 2.0 (from -1.0 to 1.0) every period. For a given frequency f, our increase is (f * 2.0)/sampleRate.

void audioBlock(float *input, float *output, int32_t samples)
{
    int i;
    float increment1, increment2;
    // 0.0000208333f is 1.0/48000.0
    increment1 = 350.0f * 2.0f * 0.0000208333f;
    increment2 = 440.0f * 2.0f * 0.0000208333f;
    for(i = 0; i < samples; i++)
    {
        saw1 += increment1;
       if(saw1 > 1.0f)
            saw1 = saw1 - 2.0f;
        saw2 += increment2;
       if(saw2 > 1.0f)
            saw2 = saw2 - 2.0f;
        output[i<<1] = saw1;
        output[(i<<1) + 1] = saw2;
    }
}

This framework is a good starting point for digital synthesis.

In the next lesson, I will provide a sample project with a MIDI host driver so that a class-compliant MIDI keyboard can be connected to the microUSB port (a microUSB>USBA OTG adapter will be required).

STM32CubeIDE: setting up an IDE for STM32F embedded processors

STM now offers a customized version of Eclipse which is configured with the compiler, debugger and other tools (the toolchain) for STM programming as well as STLink, the software which enables a link to the STM32F hardware through USB. This IDE will work with any of the STM32F discovery boards and includes CubeMX, a simple code generator which builds the initial project.

The IDE can be downloaded from this page: https://www.st.com/en/development-tools/stm32cubeide.html . If the link changes, you should be able to find it by searching for STM32CubeIDE on the www.st.com website. You will need to create an account on www.st.com to download.

You will find installation instructions on the STM32CubeIDE under “Resources”.  Click on “Resources” then “User Manuals” and you will find “STM32CubeIDE installation guide” (PDF). Download and open. At this point it will be useful to download the “STM32CubeIDE quick start guide” as well. Follow the installation instructions: install STLink first, then the IDE (do not delete the installer after installing STLink). If on macOS you may need to bypass Gatekeeper by holding down the option key when opening the installer, and the IDE for the first time. If all goes well, you will be able to start STM32CubeIDE. Under macOS, if it complains that STM32CubIDE is corrupted when starting it, you may have to clear the file attributes of the app. This is done by entering

sudo xattr -rc /Applications/STM32CubeIDE.app

in the Terminal app, and entering your password when prompted.

Setting up a sample project

There are several videos on youtube showing the creation of a sample project. I found the one from STM (“How to use STM32CubeIDE”) to be most complete: https://www.youtube.com/watch?v=eumKLXNlM0U. Use this video as a guide to creating your sample project and learning the compiler and debugger. However, be aware that some of the videos on line are already out of date.

I will be describing a project for the STM32F4 Discovery Kit. Before starting, attach the discovery board to the computer with the miniUSB jack (not the microUSB jack). The miniUSB is used for application loading and debugging.

IMG_1883

When initially creating an STM32 project (“New:STM32 Project” in “File” menu), the IDE will download an update. If this gets stuck, you will need to force quit STM32CubeIDE, re-open, and set Preferences:STMCube:Firmware Updater to “Manual Update.” You may also need to disable automatic updates in “Install/Update.”

Screen Shot 2019-09-17 at 1.57.11 PM

You should now be in the Target Selection window. Click “Board Selector” and select “STM32F4Discovery” (about 3/4 down the list). Click “Next”, and enter a project name (“LED blink” would be good). Options should be “C”, “Executable” and “STM32Cube.” Click “Next.”

Now we will be in the Device Configuration Tool. In the STM32 each pin can be used for multiple functions (flashing LEDs, communicating to CODECs, PWM, USB, etc.). This tool allows us to specify what each pin is used for, and in most cases it will generate code to use the pin in the desired way.

Screen Shot 2019-09-17 at 2.42.23 PM

For our first example, we won’t be changing anything from the default. Under the “Project” menu select “Generate Code.”

To get started, we will insert code to blink the blue and red LEDs. The youtube video above shows how to do this at 1:30+. Under the left panel “Project Explorer” open your project and under that open the “Src” arrow. Double click on main.c to open this file in the editor.

Scroll down to the main(void) function. At the end of the function there is a while(1) loop. We will add some STM specific functions that come from the HAL (hardware abstraction layer) library. Before the line /* USER CODE END WHILE */ add:

HAL_Delay(200);
HAL_GPIO_TogglePin(LD5_GPIO_Port, LD5_Pin);
HAL_GPIO_TogglePin(LD6_GPIO_Port, LD6_Pin);

HAL_Delay waits a number of milliseconds before the next instruction. HAL_GPIO_TogglePin changes the output voltage of a pin from high to low or low to high. The port and pin numbers are found in main.h, which is found under the “Inc” arrow in the project. These pin and port numbers are specific the STM32F4Discovery board.

Now we can save, compile, download and run the program on our STM32F4Discovery board. Save main.c, and select “Debug” under the “Run” menu. If prompted, select “STM32 MCU” for the type of debugger session. The IDE will ask if you want to switch to debug view. Once in debug view, you will see it has stopped on the first line of the main() function. Click the green arrow (“Run”) and you should now see the red and blue LEDs flashing.