Richard Beattie

Projects · 2023 · robots · electronics

Feline Programmable Gate Array

An FPGA robot cat that finds and follows its owner's voice

In Fall 2023 I took 6.2050 (Digital Systems Laboratory), which covers digital logic, FPGAs, SystemVerilog, communication protocols and more. Weekly assignments included:

A large part of the class was a final project, which I did with Andi Qu. At the start, Joe Steinmeyer, the lecturer, showed some previous projects, all of which set the bar high:

We decided to focus on what FPGAs do better than regular computers: real-time processing. So we built a robot that recognizes its owner’s voice, localizes them in a room, and moves towards them. Effectively a robotic cat, or Feline Programmable Gate Array.

Here’s the final result. The demo video isn’t the best, but you can see (especially towards the end) the robot moving towards me.

Design goals

We split the project into sub-goals:

System overview diagram System overview

Below are my parts. For the full details, read our report or see the code.

Audio input

We used SPH0645LM4H-B I2S MEMS microphones on Adafruit breakout boards. Localization needed four of them on a breadboard: three arranged in a triangle around one central microphone. The idea is that the sound reaches each microphone with a different phase.

The microphones send audio over I2S. In standard I2S, data is written on the falling edge of the serial clock (SCK):

Standard I2S timing diagram. DATA changes on the falling edge of the clock Standard I2S: data changes on the falling edge of the clock

The SPH0645LM4H-B, however, writes data on the rising edge:

SPH0645LM4H-B timing diagram. DATA changes on the rising edge of the clock SPH0645LM4H-B: data changes on the rising edge of the clock

So I wrote a custom I2S controller, which generates the serial clock (SCK) and word select (WS) signals, and a receiver that reads the data from the microphones.

I spent about three weeks trying to get these microphones working. Even after hours with the TAs and Joe, I couldn’t figure out why the audio was so quiet. Then one day, looking at the data on an oscilloscope, I noticed the six most significant bits were always 1. Left-shifting the audio by six bits made it much louder and easier to work with.

Filtering and decimating

Human speech typically sits between 20 Hz and 3 kHz, but we sampled at 32 kHz, so I passed the audio through a low-pass finite impulse response (FIR) filter. I used the Xilinx FIR Compiler IP, which also supports decimation, to bring the sample rate down to 8 kHz.

Phase and amplitude

Both localization and voice biometrics needed the phase and amplitude of the signal. I used the Xilinx FFT LogiCORE™ IP with four channels (one per microphone) on 512 samples, which we found experimentally balanced resource usage and accuracy.

Sound localization

The phase difference between microphones tells you the direction of the sound source.

I took the 512 FFT coefficients and discarded the upper 256, since by the Nyquist–Shannon sampling theorem only the lower 256 are usable. Then I converted them from rectangular to polar form. For each frequency I:

  1. Calculated the phase difference between the central microphone and each of the others
  2. Scaled each peripheral microphone by the phase difference and the amplitude at the central microphone
  3. Summed the scaled locations
  4. Assigned the resulting direction vector to one of 16 bins

Structure of localization module Structure of the localization module

Once all 256 frequencies are processed, the bin with the greatest magnitude gives the direction of the sound source.

16 angle bins and their corresponding angles The 16 angle bins

Motor control

Servo motor control

To test localization, I generated PWM signals to drive a servo, mapping each bin to a different duty cycle so the servo pointed at that angle.

DC motor control

I first tried continuous servos to move the robot, but they made enough noise to interfere with localization. Instead we used two Adafruit DC Gearbox TT Motors. These have three inputs: IN1, IN2 and EN. IN1 and IN2 set the direction, and since the motors only ever turned clockwise, they were always low and high respectively. EN enables the motor, so I drove it with a PWM signal to vary the speed. I chose 20 kHz so it wouldn’t interfere with localization or voice biometrics.

← All projects