Two versions of the speech recognition and processing block have been created, the full version, which fits on a Flex 10K70 series FPGA, and a smaller version that fits on the Flex 10K20 series FPGA. The full version has been simulated without known error; however, due to time constraints, it has not been testing in hardware. Hardware testing of the full version with undoubtedly reveal some errors, since complete simulation of the system requires too much time. The smaller version has been tested on the FPGA.

To make the smaller version fit on the 10K20 FPGA, the energy analysis method was removed, leaving the zero-crossings analysis as the speech analysis method. Removing the energy analysis required that the zero level for the input speech data be set as a constant at 128, the midpoint between the input data range of 0 to 255. Relying only on the zero-crossings analysis means that other words and loud noises will be recognized as one of the four trained words, as the recognition algorithm simply chooses the trained word with the closest match to the incoming word.

Space limitations have affected our design in other areas. The recognition and training algorithms have been limited to a total of four words, which allows for an easy comparison algorithm. More words require much more complex algorithms, which would take up a lot more space. Other features, such as a digital filter, have also been eliminated due to space and time constraints.

Using just zero-crossings analysis for the recognition algorithm limits the recognition accuracy. Zero-crossings analysis provides a crude representation of speech and should not be expected to provide above 90% recognition accuracy. The accuracy of this method depends on how different the trained words are from each other. The greater the difference, the better the recognition accuracy. Thus, the recognition accuracy depends on the set of words selected for training.

The word boundary detector is able to distinguish the start and end of words from the ambient noise; however, some modifications on the original algorithm were needed. Initially, the word boundary detector would often indicate the end of a word during the middle of a multi-syllable word, and then trigger again at the start of the next syllable, which confused the training and recognition algorithms. To fix this problem, the algorithm was modified so that the end of a word is detected when two zero-crossing counts fail to exceed the threshold. This modification allows for a low zero-crossing count in the middle of the word without triggering the end of the word signal.

In the creation of the ADC interface, various obstacles were overcome. These ranged from deciphering poorly documented work from previous terms to determining the physical reasoning as to why certain approaches were not working to faulty test equipment. At first, the input ADC circuit was to be taken directly from a previous design for the same purpose completed by students in the previous term. Upon further investigation of this circuit, it was not clear as to why certain stages and components were used, nor even how they were connected. In light of this, it was decided that a re-design of the circuit would be beneficial from both the educational aspect and for reasons of comfort in the ability and stability of the circuit itself. In the re-design of the circuit, the first obstacle was to interpret the minimal detail provided on the specification sheet for the microphone element that was purchased. The only information given was a recommended voltage over the device and a small diagram on the connection. With no knowledge of the inner workings of the component, it seemed an impossible task to successfully connect the device. The signal emanating from the element itself appeared to have no measurable voltage and thus was assumed to require at least pre-amplification for the signal to be verified. This problem was brought to solution within the hour after the assumption that the component had variable impedance. Later on this was confirmed by the professor, but with one small twist - that is the device, originally assumed to be of variable resistance, turned out to have a variable capacitance. The next stumbling block faced was that our pre-amp appeared to be clipping the signal at one-tenth of its intended value. This was quickly revealed to be due to a combination of faulty equipment (an oscilloscope probe was set to 1x internally although the switch upon it indicated a setting of 10x) and a resistance being out by a magnitude. These two points were resolved and the design continued. Borrowing from the schematic of our colleagues in the previous term, it was assumed that the ADC0809 chip requires a stable signal held for proper conversion to take place. After much time and energy devoted to study of the specification sheet and operational testing, the sample and hold chip would not yield consistent results. In fact, the amplified signal actually appeared to be affected merely by connection to the powered chip. The entire stage was opted out of the new design under the new assumption that the ADC0809 chip would be self-sufficient in its conversion of signals. Testing later confirmed this assumption. The actual test performed to conclude this involved feeding the ADC0809 chip a steady clock pulse and connection of its input to an oscillating source. The start of conversion signal was triggered and the output was observed upon the oscilloscope. The frequency of the signal was varied and the output was observed. The results of this allowed for confirmation of the above-made assumption. Once the circuit was operational on both ends, and before connection, the signal’s amplitude was capable of reaching voltages that were twice the allowable limits to input to the ADC integrated circuit and therefore required to be limitations. The actual manner of implementing this was tried three separate ways. The first attempt involved running two diodes in series from ground to power and running the signal through the node between them. Theoretically, this would froward bias one of the diodes at the -0.7volt and +5.7volt marks. In practice, the voltage of the signal at that node ranged from –0.7volts to approximately 7.0 volts. This was deemed unacceptable and the idea was disregarded from further consideration. The next attempt involved utilizing the operational amplifier’s property to not being able to surpass its supply voltage. In light of this, a buffer was connected with the supplies connected to ground and +5volts. For reasons not fully investigated, the circuit seemed to oscillate up to nearly +8volts at the application of a loud whistle to the circuit. The investigation ceased upon the implementation of our third and final attempt to limit the signal’s swing with a 5.1volt Zener diode. This final implementation allowed for a simple method of limitation of the signal near the –0.7volt and 5.2volt levels. In the end, the design of the input ADC circuit proved to work fully.

The on-chip support for the ADC converter is done with the help of our aforementioned colleagues. The actual interfacing code was modified slightly to tailor it to our specific needs. The details of our modifications can be seen in the code and involve adding an extra process to their input_reader.vhd to ensure synchronicity of the input data.

Test cases that were used to confirm proper operation

Testing for the control path was done extensively in simulation and confirmed with the results of real-time operation. Alone, a battery of simulations was performed in order to examine the behavior of the entity under various situations. In particular, simulations were done to test the training and initialization in every possible configuration. This was to ensure that nothing the user could do would cause the device to operate incorrectly. In the proceedings of these simulations, some of the problematic issues that arose did not pertain to the actual digital operation, but to that of the overall device. The sequence of training was very highly dependent upon the successful operation of the device. If, for instance the voice training was done without the knowledge of the ambient noise, the information in the voice training could potentially be rendered useless. Another problem, this one more directly related to the digital implementation, was the issue of when the surf function was sensitive to the command to cease and exit into normal active operation. It was found that if the command to cease was issued at a particular time, while the surf function was counting its delay, the request would be ignored. This proved to be potentially fatal to the reliability of the surf function and was uncovered only after simulations were done to specifically target that aspect of the design. In real-time testing, the control path was tested used dummy signals and longer time durations to first confirm the accuracy of the simulations before the large-scale integration of the various parts. Although it proved rather tricky to manipulate the various signals manually and with the correct sequence, the majority, if not all of the functionality was confirmed to the designer’s satisfaction.

The first attempt at implementing the IR module had the following results:

• Each individual component (demodulator, transmitter, code-selector, control-path and memory interface) simulated with the Max+plus2 waveform editor with what appeared to be successful results.
• All components were combined into a higher level entity and simulated with the Max+plus2 waveform editor with what appeared to be successful results.
• The higher level entity was downloaded onto the hardware and did not perform any of its required functionality.
• The higher level entity was broken up and simplified to trace the source of the design flaw.
• No functionality was obtained from the simplified and modularized versions.

Note: Simulations of over 10ms were performed on the higher level entity to better check the design. This required many hours of simulation and no useful results.

With the failure of the initial system, the design was rethought and completely redone. The control path was broken up into three separate controller entities: the Decoder control, the top-level control, the address/display control and the transmitter control (as shown in the hierarchy diagram). Each control path was remodeled after Altera’s recommended state machine design (i.e. Moore Machine). The separating of sequential logic and combinational logic was a key difference in changing the design. A new and improved functionality for the training mode was developed and a more modularized testing approach was taken.

Due to the failures and time-consuming nature of the simulations a new testing strategy was developed. Each control path was designed to be capable of separately controlling a section of the system and was independently verifiable on the hardware. In this manner separate portions of the overall design could be tested and verified on the hardware before being integrated. As each component was tested successfully a new component would be added to the existing design. The design strategy was to build up the overall module from existing functioning components, eventually leading to a functioning overall system. This approach worked quite well and the overall IR module entity was completed with only one major problem along the way. While testing to see if correct data was being stored in the memory, (on the hardware) it became apparent that the system was not storing expected sequences in memory. This resulted in a long and time consuming hunt for the ‘bug’ causing the incorrect values to appear in memory. Eventually it was discovered that the system was being over-clocked. The module was modified to run at a slower clock and the system performed as intended.

Speech processing and recognition: 784 logic cells, 2 EABs

Input logic: 117 logic cells

Control path: 107 logic cells

Infrared Interface: 365 logic cells, 2 EABs

Total: 1373 logic cells

The total number of logic cells exceeds the capacity of the flex10K20 and thus we were forced to use the flex10K70.

Before the speech analysis and recognition algorithms were implemented in VHDL, they were written in Matlab for evaluation and to generate test vectors for VHDL simulations. The Matlab algorithms were written to produce the same results as the VHDL implementations, using methods such as truncating the decimal portion of the results of integer division. The Matlab code is provided in Appendix C.

Four different words were used to test the Matlab algorithms, channel, up, down, and TV. Each word was recorded four times, to provide enough data for training. The input speech data samples used by the Matlab simulations were recorded as wav files using Microsoft's Sound Recorder program. Speech samples were digitally recorded using 8-bit sampling at an 8 kHz frequency.

The word boundary detector was the first algorithm tested. This algorithm could accurately detect the start and end of a word, as illustrated by the graphs on the following pages. These graphs show the raw speech waveform, the speech waveform after word boundary detection, and the zero-crossings and energy analysis for several different words.

Using the results from the word boundary detector, the training and recognition algorithms were tested. The recognition algorithm was tested with the same data used for training the system. Two versions of the recognition and training algorithms were evaluated, one version performed the speech analysis over blocks of 256 samples of the input speech data, while the other version used 128 samples. The following tables

show the results of the recognition tests for the zero-crossings analysis, the energy analysis, and the combined analysis.

Recognition Results For Blocks of 256 Samples

Recognition Results For Blocks of 128 Samples

From these results, the two version have about the same performance. However, the method using 128 samples in a block requires more memory than the 256 sample method, since each word will require twice as many blocks to represent it.

As mentioned above, a new approach to verifying the design was undertaken and involved the hardware testing of small, simple components rather then the simulation of more complex entities and hardware verification of higher level entities. The module was tested in these stages:

• Display and addressing.
• Code sequence detection/storage
• Top level training mode and memory verification
• Transmission verification from ROM
• Integration of Transmission and detection entities
• Integration with higher-level system

Display and addressing:

This section allows the user to scroll through a list of IR commands and select which one they will ‘train’ for. This is the new functionality that was mentioned above (the initial design did not allow the user to select the IR command). This sub-system displays the command on the seven-segment displays and also outputs the appropriate address location of the command (the starting location in memory of the code). This section was tested by routing all ‘would be’ internal signals to observable/alterable I/O devices (such as LED, push buttons and dipswitches) to confirm the correct operation of the entity. All aspects of this sub-section worked on the hardware.

Code sequence detection/storage:

This section independently receives the count values from the IR decoder (the IR decoder module, called bit stream counter in diagram, was reused from the initial design) and performs the necessary incrementing of the address and output of data (count values) to the lpm_ram (also reused from initial design). The entity sends a signal to the Display and addressing control to inform it when a full code was received. The Display then lets the user know that a code was acknowledged (by displaying an acknowledged symbol on the seven-segment display). This sub-unit was tested in the same way as above. All the ‘would be’ internal signals were routed to observable/alterable I/O devices and the sub-system was tested as it should perform. The sub-system performed as required on the hardware. The sub-system was integrated with the Display and addressing sub-system and they were tested together using the same techniques. The integrated sub-systems performed as required on the hardware. Code for the entity used for testing the combination of these two sub-systems can be found in appendix A and is called 'detector_module.vhd'.

Top level training mode and memory verification:

To complete the full operation of the system for training mode, (i.e. grabbing codes and dumping them into the appropriate memory location) the sub-systems described above were integrated with a top-level control, which selects between a training and a transmitting mode, and the lpm memory. To verify that correct data was being stored in memory, the transmitting mode of the top-level controller was replaced (for modular verification purposes) with a "view memory" mode. This mode allowed for the data in memory to be output to 12 LEDs (the width of the data) showing the contents of the memory. As well a feature was built in allowing for all memory locations to be scrolled through one at a time. In this manner the full-training side of the IR module was tested on the hardware. Results are currently being verified but appear to show correct operation of the module thus far. Code for this entity and all the sub-entities and components can be found in appendix IR. Since no significant simulation was performed no waveforms are presented. A demonstration of the correct operation of the IR system thus far can be arrange to demonstrate that full testing and verification of the system has been accomplished successfully. The code for the current entity used to test the system on the hardware can be found in appendix A and is called 'frontmodule.vhd'.

Transmission verification from ROM:

The transmission module was first tested on the hardware by separating it from all other components and having it read values from a ROM[11]. This allowed for the verification of this entity independently of the rest of the module. The correct transmission of codes was verified by using the demodulator circuit to detect and observe the bit sequence on an oscilloscope. The oscilloscope waveform was compared to the actual output from a remote control and proved to be virtually identical. Below is the waveform of the transmitted output as seen on the oscilloscope.

Integration of Transmission and detection entities:

The frontmodule.vhd entity was integrated with the transmission module to form the IRmodule.vhd. This final entity was also tested independently of the other sub-systems before integration.

Overall our system performed according to our goals laid out at the beginning of the term. Our system became much more limited in its vocabulary and IR code compatibility than our initial plan. This was a result of the limitations of the flex10K20 board and the time constraints of having a smaller group. Our group lost its fourth member after the project proposal and we nonetheless attempted to complete the

initial functionality laid out in the project proposal. The complexity of our system was perhaps incongruent with the size of our group and man-hours available for the project. The amount of documentation required for the project was the largest obstacle in hindering our efforts to complete the project. Invaluable knowledge was gained in the area of digital design, signal processing and recognition and IR coding.

[1] Asiedu-Ampem, P. et al. EE 552 Application Notes – Clock Divider. Internet: 1998.

www.ee.ualberta.ca/~elliot/ee552/studentAppNotes/98f/clk_div/clk_div.html

[2] Bensler, T. and E. Chan. Interfacing External SRAM. Internet: 1999.

www.ee.ualberta.ca/~elliot/ee552/studentAppNotes/99w/SRAM/

[3] Bo, N., K. Leung, and D. Ritter. FIR Filter Design. Internet: 1999.

www.ee.ualberta.ca/~elliot/ee552/studentAppNotes/99w/FIRFilter/

[4] Furui, S. Digital Speech Processing, Synthesis, and Recognition. New York: Marcel Dekker,

Inc., 1989.

[5] Gould, D., K. Grant, and A. Stanley-Jones. Voice-Activated RC Car. Internet: 1999.

www.ee.ualberta.ca/~elliot/ee552/projects/99w/voice_activated_rc_car/

[6] Robinson, T. Speech Analysis. Internet: 1998. svr-www.eng.cam.ac.uk/~ajr/SA95/

[7] Yadunandana, R. Speech Recognition Using Hidden Markov Models. Internet 1999.

www.angelfire.com/ny/yadunandana/report1.html

[8] Willmott, K. IR Remote Control Codes. Newsgroup Posting displayed via Internet 1999.

[9] University of Washington, Circuit Archive, Internet 1999

[10] University of Alberta, EE 480 gcd package VHDL code, Internet 1999

[11] Univerisity of Alberta, EE 552 lab 6 ROM VHDL code, Internet 1999

The Proto-A Voice-IR interface is an integrated system composed of an Altera Flex10K70 field programmable gate array mounted on the UP1 board, condenser microphone, dual 741 op-amp, 8-bit analog-to-digital converter, IR demodulator and IR-LED transmitter. The Proto-A has preprogrammed logic configuration downloaded onto the Flex10K20. This allows the FPGA to interact with the peripherals listed above to provide the user with superior voice control of consumer Audio/Visual electronics equipment. The system has four modes of operation giving it a wide range of user and A/V electronics interaction. The first mode performs a noise sample of the ambient noise to set the threshold for speech training and recognition. The second mode allows the user to train the system to their specific voice commands. The third mode allows for the user to train the Proto-A to transmit their A/V electronics IR codes. The fourth mode allows the user to utter voice commands and the Proto-A transmits the appropriate IR code, thus controlling the A/V device by voice. The Proto-A features zero crossing and energy analysis speech processing techniques for speaker dependent recognition of up to four voice commands. The IR demodulator allows the user to send the Proto-A codes for memory storage and subsequent transmission via the IR LEDs.

The device eliminates the need for tedious, repetitive, and wearing motions of continually pushing the buttons of A/V electronics remote controls. Easy, wireless and exclusive control is made possible with the Proto-A system. The design of the Proto-A has been optimized to include such calorie saving functions as ‘surf’, which allows a single IR command to be repeated every 3 seconds. These features make the Proto-A ideally suited for such applications as ‘couch-potato’ to ‘TV-hog’.

IR output

B

IR input

Vcc = 5V

D0 Pin 20

D1 Pin 21

D2 Pin 22

D3 Pin 23

D4 Pin 24

D5 Pin 25

D6 Pin 26

D7 Pin 27

SOC Pin 28

EOC Pin 29

A Pin 15

B Pin 17

This appendix contains all the VHDL code for the IR module including the code of individually tested components. Simulations of key entities are also contained in this appendix.

frontmodule.vhd – entity used for testing training mode of IR module

topview.vhd – top level control of IR module, modified for viewing the contents of memory during hardware verification.

Mux_1.vhd – 1 bit, 2 input multiplexer for control of address incrementing and clearing. (modified form EE480 web-site)[10]

Simplecontrol.vhd – control path for displaying and addressing

delay_counter – introduces a delay flag for portions of design requiring a delay

IR_decoder – two lpm counters for sending data on demodulated IR bit stream count

IRmodule – Top level IRmodule entity

Irransmit.vhd – top level entity for transmission

Mux2N_N.vhd – two input N-wide multiplexer Taken and modified from EE480 web-site [10]

Modulator.vhd – entity that outputs 38kHz 50% duty cycle signal.

Rom.vhd – used for independent test of testtrans2 – taken and modified from EE552 web-site[11]

Testtrans2.vhd – entity used for independent test of transmission module

Filter.vhd – used to display and increment code requested during training mode

Mother_control.vhd – this is a modified and final version of the detector control path

RegisterN.vhd – an N-bit register used for registering the count values from IR_decoder to mother_control

• taken from the EE480 web-site[10]

Subtestdet.vhd – this is the sub-control responsible for write-enable, increment address of mother-control.

Testdet.vhd – this is the main control for mother_control.

Topcontrol.vhd – this is the top level control of the IR module.

Trans_control – this controls the modulation of the out-going IR code.

Detector_module.vhd – entity used for testing the counting sequence of IR bit stream input