EEG Signal Processing

EEG Signal Processing andEmotiv’s Neuro Headset EEG-Signalverarbeitung mit dem Emotiv Epoc Headset Bachelor-Thesis von André Hoffmann 2010-09-28 Gruppe: Multimodal Interactive Systems Betreuer: Prof. Dr. Bernt Schiele DFKI Kaiserslautern Gruppe: Wissensmanagement Betreuer 1: Ralf Biedert Betreuer 2: Georg Buscher EEG Signal Processing and Emotiv’s Neuro Headset EEG-Signalverarbeitung mit dem Emotiv Epoc Headset Vorgelegte Bachelor-Thesis von André Hoffmann 1. Gutachten: Prof. Dr. Bernt Schiele 2. Gutachten: Dipl.-Inf. Ralf Biedert Tag der Einreichung: Erklärung zur Bachelor-Thesis Hiermit versichere ich, die vorliegende Bachelor-Thesis ohne Hilfe Dritter nur mit den angegebenen Quellen und Hilfsmitteln angefertigt zu haben. Alle Stellen, die aus Quellen entnommen wurden, sind als solche kenntlich gemacht. Diese Arbeit hat in gleicher oder ähnlicher Form noch keiner Prüfungsbehörde vorgelegen. Darmstadt, den (André Hoffmann) 1 Abstract Recently, there has been a growing interest in employing electroencephalographs(EEGs) for human-computer-interfaces as the prices for low-cost EEGs have been fallen to a level that makes them affordable for consumers. In this thesis a general introduction to the topic of EEG, data-acquisition and processing of the EEG-signal will be given. Special attention will be paid to artifact and noise. Two different experiments will be presented: In the first the Epoc’s applicability to the P300 speller(a method that allows users to spell by using their mind) and more generally event-related potentials was studied. The second experiment concerns event-related desynchronization and analyzes if the Epoc can be used for distinguishing if the user is relaxing or solving a math or logic question. This thesis targets to evaluate the Epoc’s overall usability for brain-computer-interface applications. Zusammenfassung In letzter Zeit ist das Interesse daran gestiegen, Elektroenzephalographen(EEGs) in Mensch-Maschine-Schnittstellen zu verwenden, da die Preise für kostengünstige EEGs auf ein für Endverbraucher bezahlbares Niveau gefallen sind. In dieser Arbeit geht es darum eine generelle Einführung in die Thematik des EEGs, der Datenerfassung und -verarbeitung des Signals zu geben. Insbesondere wird hierbei auf Störungen und Rauschen eingegangen. Weiterhin werden 2 verschiedene Experimente vorgestellt. Im ersten soll es darum gehen die Anwendbarkeit eines EEGs namens Epoc auf den P300 speller(ein Verfahren das es dem Benutzer ermöglicht mittels Gedanken zu buchstabieren) sowie auf ereigniskorrelierte Potentiale zu untersuchen. Das zweite Experiment beschäftigt sich mit ereigniskorrelierter Desynchronisation und betrachtet, ob mit dem Epoc unterschieden werden kann ob ein Benutzer sich entspannt oder ein Matherätsel beziehungsweise eine Logikaufgabe löst. Das Ziel dieser Arbeit soll es sein, die generelle Tauglichkeit des Epocs für Anwendungen der Hirn-Maschine-Schnittstelle zu untersuchen. 2 . . . . . . . . . . . . . . . . . . . . . . . . . .1. . . . . .3.1 Structure and function of the neuron 2. . . . . . . . . . . . . . . .3. . . . . . . .1.1. . . . 4. . . . . . . . 4. . . . . . . . . . . . . . . . . . . . . . . .2 Introduction . . . . . . . . . . . . .2. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .2 Hardware . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4. . . . . 5 Evaluation 7 7 7 7 8 9 9 9 10 11 11 11 12 12 12 13 13 14 14 15 15 15 16 20 . . . . . . . . . . . . . . 2.1 Electrode Locations . . . . . . . . . . . . . . . . . . . . . . . . . . . 4. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4. . . . . . .4. .1 Electrode Positioning . . . . . . . . . .2. . . . . . . . . . . . . . . .2 EOG . . . . . . . . . . . . . . . . . . . . . . . . . . 2. . . . . . . .3 Aliasing . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .3 Removing the Baseline . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .3. . . . . . 28 5. . . . . . . .1 Biomedical Background . . . . . . . . . . . . . . . . . . . . . . . . . . . . .3 EEGLab . . . . . .1 Motivation . . . . . . . . .6 P300 . . . . . . . . . . . . . . . . . . . 2. . . . . . 2. .3 Biological Artifacts . . . . . . 2.1 Import/Export . . . . 2. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .2 The EEG-Signal . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .1 P300 Speller . . . . . . . . . .4. . . . . . . . . . . .1.Contents 1 Introduction 5 2 Background 7 2. . . . . . . . .2 Validation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4. . . . . . . . . . . 2. . . . . .1. . . . . . . . . . . . . . . . . . . . . 4. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .1. . . . . . . . . . . . . . . . . . 2. . . . . . . . . . . . . . . . .2. . 2. . . . . . . . . . .1. 28 5. . . . . . .1. . . . . . . . . . . .2 Emotiv Epoc Headset . . . . 2. . . . . . . . . . . . . . . . . . . . . . . . . .3. .1. . . . . . . . . . . . . . . . . . . . . . . . . . . 28 3 . . . . . . . . . . . . . . . . . . . . . . .3 Averaging . . . . . . . . . . . . . . . . . . . . . .3. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .3. .2 Artifact Removal . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2. . . . . . . .2 Action potentials . . . . . . . . . . . . . . . . . . . . . . . . . .1. . . . . . . . . . . . . . . . .2. .2. . . . . . . .2. . . . .1 Creating the Java Wrapper . . . . . . . . .1. . . . . . . . . . . . . . . 4. .1 2nd -order IIR-Filter . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .1 50/60 Hz Artifact . . . . . . . .2 Bandpass Filtering . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .2. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .5 Additional Biological Artifacts . . . . . . . . . . . . . . . . . . . . . . . . .4. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .7 Event-related (De)synchronization . . .1. . . . . . . . .1. . . . . . . . . . . . . .1.2 Determination of the Sampling Rate 4. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4. . . . . . . . . . . . . . . . . . . . .1. . . . . . . . . .1. . . . . . 4. . . 2. . . . . . . . . . . . . . . . . . 3 Related Work 18 4 Signal Processing 4. . . . . . . . . . . . . .3 EMG . 2. . . . . . . . . . .1. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2. . . . . . . . . . . . . .2 Amplifier Noise . . . . . .9 Alpha Waves . . . . . .1. . . . . . . . . . 2. .2. . . . . . . . . . . . . . . . . . . . . . . . . . . . .1 The neuron . . . . . . . . .1. . . . . . . . . . . . . . 2. . . . . 2. . . .1. .4 ECG . . . . . . . . . . . . . . . . .4 Technical Artifacts . . . . . . . . . . .5 Event-related Potentials . . . 4. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2. . . . . . . . . . . . . . . .3. . . . . . . .1. . . . . . . . . . . . . . . . . . . . . .1 Blinks . . . . . . . . .1. .1. . . . . . . . . . . . . . . . . . 20 20 20 21 21 22 24 24 24 25 26 26 27 27 28 5. . . . . . . . . . . . . . .1 Overview . . .1. . . . . . . . 4. . . . . . .3. . . . . . . . . . . . . .1. . . . . . . . . . . . . . . . .1. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2. . . . . . . .2 Events & Signal . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .1. . . . . . .8 Frequency Bands . . . . . . . . . . . . . .2. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .1 Obtaining the Signal . .1. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .1 Tools . .2. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 5. . . . . . . . . . . . . .1. . . . . . . . . . . . . . . . . . . . . .3 Headset Placement . . . . . . . . . . . . . . . . . .2. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 5. . . . . . . . . . . . .5. . . . . . . . 46 46 46 46 46 4 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 5. . . A. . . .2. . . . . . . . . . . . . 5. . . . . . . . . . . . . . . . .2. . .1 Averaged Band Powers 5. . . . . . .1. . . . . . . . . . . . . . . . . . . . . . . . . . . . . .3 Experimental Setup . . . 28 29 29 30 31 33 33 33 33 34 34 34 35 6 Conclusion 39 7 Outlook 40 Bibliography 41 A Creating the Java Wrapper 46 A. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .7 Results . . . . . . . . . . . . . . . .2 ERD/ERS Maps . . . . . . . . . . . . . . . . .1. . . .1 JNI . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 5.2 Problems . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .1 Motivation . . . . . . . . . . . . . . . . . . . . . . . . .4 Experimental Setup . . . . . . . . .1. . . . . . . . . . . . .6 Preprocessing . . . . . . . . . . . . . . . . . . . 5. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .1. . . . . . . . . . .3 Solution . . . . . . . . . . . . . . . . . . .5 Results . . . . . . 5. . . . . . . . . 5. . . . . . . . . 5. . . . . . . . . . . . . . . . . 5. . . . . . . . . . . .1. 5. . .5. . . . . . . . . . . . . . .2 Introduction . . . . . . . . . . .2. . . . . . . . . . . . . . . . . . . . . . . . . . . . . .5 Tools . A. . .2 SWIG A. . . .1. . . . . . . .2 Alpha Waves Quiz . . . . . . . . . . . . . .2. . . . . . . . . . .5. . . . . . . . . . . . . . . . . . . . . . . . . .2. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .4 Headset Placement . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . A. . . . . . . . . . . . . . . . . . . . . . . . . . . . . : Low-cost EEG systems under $1000. that isgives changes the electrical the13 brain up very quickly in the Figure 2 shows the calculated scores for each device normalised to 10. Source[3] Fig. A 1. the Epoc scored best in terms of usability and was ranged in the middle Psychlab EEG1 Raw EEG USB price segment[3]. In their findings. the scores have little The EEG’s main drawback is its poor spatial resolution that is one can never measure a single neurons activity but instead meaning in themselves as different applications would require different device only the sum of hundreds of thousands of neurons[4].1) and rated In a recent study different low-cost EEG systems have been openEEG by usability(see figure 1. This a in total possibleactivity scoreofof forshow each device. Epoc Raw EEG/Classification Data Wireless Neurobit lite Classification Data Wireless IrDA Pocket Neurobics Raw EEG Wireless Pendant EEG Teunis van Beelen’s Rawcompared EEG by price and functionality(see RS232 table 1. The cheapest EEG acquisition hardware for on-line BCI Bluetooth systems $200 − $500 OCZ Tech. Ever since then the EEG and the human brain has been studied intensively by psychologists.$1000 Table 1.1. Also it is non-invasive and therefore relatively user-friendly. Today. Emotiv even announced future EEGs with dry sensor which will dramatically reduce the preparation time for using the device[5][6]. Specialised Hardware defines devices which provide little or no target application flexibility.1. Mindset Raw EEG/Classification Data Bluetooth Raw EEGData $200 . The authors would like to point out that although we intended these scores to give some idea of the usefulness of each device to research. they are already very affordable today(compared to $20. the German psychiatrist Hans Berger published the first of 20 papers with the title “Über das Elektroenkephalogram des Menschen”[1] where he describes his invention of the electroencephalogram[2]. the aEmotiv Epoc showed the highest score for this particular rating system... Sectioned into three categories. NIA Classification Data USB Emotiv.$500 Psychlab OCZ EEG1 Tech.$200 ModularEEG RS232 Teunis van Beelen’s openEEG Raw EEG RS232 Neurosky. Commercial. new neuroheadset such as the Emotiv Epoc attract hobbyists and may soon allow the use of EEGs for the consumer. We placed the OCZ’s NIA and the Neurobit Lite in . Hereto produce signal of the EEG[4]. NIA Classification USBUSB Price Range 0 − 200 $ Emotiv. the following information is a summary of a 5 table composed of extensive advantages and disadvantages of each of the devices4 . Epoc Raw EEG/Classification Data Wireless Price Range Device Data contents Interface $500 − $1000 Neurobit lite Classification Data Wireless IrDA (US$) from device Pocket Neurobics Pendant Raw EEG EEG Raw EEG Wireless . $500 .: tentative scale forofthe usabilityEEG score of 8 EEG acquisition devices Usability rating low-budget acquisition devices. It is impossible to find the exact location of a neuronal activity specifications. An EEG has ahydrated very high time-resolution.Figure 2.1 Introduction In 1929.000-250. neuroscientist and doctors around the globe. Towards Inexpensive BCI Control for Wheelchair Device Data contents from deviceNavigation Interface 341 ModularEEG Raw EEG RS232 Mindset Raw EEG/Classification Data Table 2.000 for a professional EEG system) and it is just a matter of time until a killer application emerges and justifies the initial cost of acquisition.Neurosky. In fact. portability. and support for dry or electrodes.1). Source[3] easy of electrode placement by the user. Though. currently expensive Table 1. there exist other methods that yield more reliable results. Technology Electrocorticogram (ECoG) Magneto-encephalography (MEG) Computed Tomography (CT) Single Photon Emission Computerized Tomography (SPECT) Positron Emission Tomography (PET) Magnetic Resonance Imaging (MRI) Functional Magnetic Resonance Imaging (fMRI) Event-Related Optical Signal / Functional Near-Infrared (EROS/fNIR) Primary Disadvantage Highly invasive. and the Epoc neuroheadset’s applicability to them analyzed.: Brain sensing technologies and their primary disadvantages in BCI research[7] In this thesis a general overview over the data-acquisition and processing of the EEG-signal will be given. the EEG is to this day the most used method in the research of brain-computer-interfaces(BCI). two different methods that are used in the work with EEGs will be studied. surgery Extremely expensive Only anatomical data Radiation exposure Radiation exposure Only anatomical data Extremely expensive Still in infancy.2.with an EEG. For a list of competing techniques in brain-research and the main reason they can currently not be used for BCI see table 1. That is due to the mostly extreme costs or high degree of invasion of other methods. 6 .2. B Axon (pre-synaptic) Dendrites Nucleus Cell body (soma) Time 0 mV Axon hillock Myelin sheath Axon (post-synaptic) C -50 mV A -70 mV D Figure 2. 20].1).2) before it then finally stabilizes at the resting potential[9. 20].1. 2. 20]) 2. the neuron becomes activated and sents this information(the action potential) to subsequent neurons.1.2. If this depolarization reaches −55 mV it causes the cell membrane to completely open up for Na+ ions that now enter the cell and make the potential momentarily positive which is referred to as action potential(see B. p.2)[8.1 Structure and function of the neuron A neuron consists of a cell body(also knows as soma). p. 18]. p. Based on the informations dendrites receive from other neurons. Temporarily delayed.1.2)[8.2 Background 2. 7 . the cell membrane also opens up for the K+ ions resting in the cell that now(see C.1.2) leave the cell causing the repolarization of the membrane’s potential[9. dendrites and an axon(see figure 2. figure 2. the neuron will be briefly explained. the neuron now has to make a decision that is then sent to other neurons’ dendrite over the axon[8. 18]. 20]. the most fundamental cell in neuropsychology. p.1 The neuron To understand the origin of the EEG signal. As the cell membrane only slowly loses its permeability for the K+ ions.1 Biomedical Background 2. figure 2.1. It is the action potential that is carried over by the axon to other neurons.: Action potential (based on[8.1. 19]) Figure 2.: Scheme of the neuron (illustration based on[8. p. p. p. 18]. see D in figure 2. the potential temporarily falls under −70 mV(this is called hyperpolarization. To summarize: If the summed electrical current from all incoming axons exceeds a certain threshold. p. Incoming electrical current from the dendrites makes the membrane potential less negative(see A.2 Action potentials A cell membrane surrounding the neuron keeps charged sodium(Na+ ) and potassium(K+ ) ions from floating into and out of the cell body that is therefore negatively charged to the outside with a resting potential of −70 mV[8. figure 2. A COGNITIVE FRAMEWORK 8 . last over 100 ms[13. Top: Enlarged detail of a section below the scalp. An electroencephalogram (EEG) electrode (shown on the right of the scalp) is therefore able to pic recordings in the “usual” way[19. special attention should be paid to the orientation of the horizontal propagated current and reflect it in the form of an electroencephalogram. 21].3. depending on their position in the folds of the cortex. 16]). ELECTROMAGNETIC MANIFESTATIONS OF MIND Figure 2. s meninges.: Cortical generators of electric currents(based on[16. 32]. p. 30]. thethat propagation the electrical dipoles of the pyramidal neurons other way around in other scientific fields.2. 29]). p. FIGURE 2 Simplified diagram of the cortical generators of electromagnetic fields and both intracer surface recordings of the signals derived from them. depicting a cerebral cortical convolution with one of its characteristic sulci formed by the fold cortical mantle. It is important to note here that the signal of each channel is always plotted as the difference to a reference electrode. part of the cerebral electrofunctional activation is not appropriately represented in the EEG.4. For an electrical signal to be strong enough to be detectable the following requirements have to be satisfied[8. 8] and constitute the main source of the EEG-signal. p. 4]. This is referred to as either excitatory postsynaptic potential(when positive) or inhibitory postsynaptic potential(when negative)[12. the smaller arrows indicate the direction of propagation of its magnetic counterpart. 3] and thereby influences the neuron’s potential difference. Only part of the electrical a the neurons arranged perpendicularly deep in the bottom of the sulci is picked up by the electrode. T axis when looking at EEG plots. Nor does it last long enough(0. p. p.1. When an action potential reaches the end of the axon(the axon terminal) the release of neurotransmitters from the axon terminal to the synapses is being triggered[8. Postsynaptic potentials vary in their amplitude between 50 and 100 mV(measurement on the scalp)[10. p. despite of it being the the dipole. 38]: • many neurons must fire synchronously • those neurons must be aligned in parallel so that they summate rather than cancel out(see figure 2. There exist two kinds of neurotransmitters: one causes an influx and the other an outflow of positive ions by changing the permeability of the postsynaptic neuron’s membrane[11. p. Thick arrows represent the propagation of electrical currents that can be picked up by the EEG electrode. These produce open fields.3 ms) to accumulate sufficient power together with other synchronously firing neurons[10. even though the name of the reference channel is often omitted. The same I. p.: A classical printed EEG recording(from: [15. Though. p. 10]. In the s areas of theauthors cortex. p. occurring at a rate of 200 Hz[4]. 2.2 The EEG-Signal A single neuron’s action potential can not be measured extracellularly as its amplitude is too small to be picked up by the EEG. With that said. there are some do notofagree to this convention and plot theiris radial to the b head.4) For further reading on the subject see: [10][14] 16 Figure 2. dark arrows represent the direction of propagation of the electrical coun Neuropsychologists have the odd convention of plotting negative upward and positive downward[18]. The electroencephalography is defined as a graphic representation of the potential difference between two different cerebral locations plotted over time[17]. The thick. The section also shows some pyramidal neurons with their apical dendritic prolongat Remark 1 (Conventions in EEG Plots) ented perpendicularly or horizontally. 1.1 Overview As can be seen in table 2.6) originates from a potential difference(electrical dipole). In the following.1.e.3.1 artifacts tend to have a much stronger amplitude than the EEG-signal itself. are considered “Biological Artifacts” or sometimes “Physiologic Artifacts”[20]. i. the eye is electrically charged. Signal EEG EEG(EP) EOG EMG ECG Amplitude Range from to 2 µV 100 µV 0.1.: Eye Dipole Model Figure 2.5 100 1 3.2 EOG Electrooculographic(EOG) artifacts arise from movements of the eye and blinks which cause changes in the eye’s electric fields[22]. One field(figure 2. positive at the cornea and negative at the retina. 2. it is therefore crucial to make special considerations to artifacts.3.8 show the electrode placement of an Electrooculograph which measures the voltages between the two electrodes on either side of the eye.7 and 2. the frequency ranges also overlap to some extent. that those properties refer to measurements being taken as close as possible to the origin of the electrical activity.[23].1 µV 20 µV 10 µV 5 mV 50 µV 5 mV 1 mV 10 mV Frequency Range(Hz) from to 0.05 100 Table 2. that is in the skin area closest to the skeletal muscle.2. Sclera Retina Lens Optic Nerve Iris Cornea - Figure 2. 000 0 100 2 500 0. 9 . It should be noted.: Overview over some artifact types and their amplitude/frequency ranges[21]. 2.1.6. Movements of the eye cause the positive side of the dipole(the cornea) to be closer to and the negative retina to be further away from one of the two electrodes and hence increases the voltages being measured for this and decreases it for the other electrode(10 µV − 40 µV per degree of rotation)[22].5. but do not arise from the brain itself. Unfortunately.3 Biological Artifacts Electrical activities that are caused by the patient.: Scheme of an Eye + Figure 2. Being a significant component of the signal obtained with an EEG. possible sources will be briefly explained. the first (dark) trough amplitude in lV. 15 min dark. Also EMG artifacts should be considered when designing an experiment: The patient should be seated comfortably and shouldn’t be required to move or turn his head during the experiment if it is not at all avoidable. All y-axis scalings are in µV . Recording electrode positions Figure 2.7. 123 Another electrical field is created by the eyelid gliding over the cornea during a blink[25]. fixation Measure the amplitude of the EOG Taking care to remove the effects of overshoot (see Fig. The report should also state - - Fig. Plot the change in the responses Plot the average amplitude of the EOG inAC/DC lV on a graph repre. ‘chin/head on rest/restraint in stimulator. Report As a basic factual report state the Arden ratio. 1 (a and b). Ignore any pointsAmplifier known to be incorrect (e. [21] 2. The EMG signal also influences the EEG recording as illustrated in figure 2. measure the EOG amplitude in lV.9.: How eye-movements and blinks(right) influence the EEG recording(left). 15 min light. but also influence the EEG signal.8.1. Calculate the Arden ratio This is the ratio between the peak and -trough of the underlying curve. 3 (lower + + + + panel).: EOG Electrode(green) Placement.AC/DC senting the 30 min of theAmplifier test. the pre-test light exposure should be the same for all test subjects.3 EMG The Electromyogram(EMG) measures electrical currents generated in muscles during its contraction representing neuromuscular activities[26]. not the maximum and minimum recorded values. Blinks and eye-movements do not only show up in an EOG recording.: Example of an EOG [24] Figure 2. either manually or by a computer algorithm.10.3.near as is practical. Calculate the average of the amplitudes in each 10 s trial. It is therefore important to inform the patient about it so that he can try to move as few as possible during the recording.g. Figure 2. Preparing the test subject Explain the procedure including. and identify the underlying curve (see notes) as in Fig. the time from start of the light phase to the peak (if present). subject did not follow the fixation lights properly). and the type of adapting light source.9. the pupil size. 2 and notes below) and stepped saccades. If possible pauses might be built into the experiment that the patient can use to move. as can be seen in figure 2. 10 . 771µV!. This correlation should be considered in the further analysis. A similar artifact occur when one of the EEG is placed upon a pulsating blood vessel[20]. E.Technical Report SIESTA-Task310 Artifact Detection and Denoising Figure 2. the covariance e. anshows EEG signal(blue. that are 1.: EMG artifacts in an EEG recording[20] EOG and ECG minimization 2.g. breathing.640 = 5.3.3. The second line is the corrected EEG1' and the third line shows the recorded ECG [3mV / unit ].8312 Similar to the EMG.4310 .1. Supplementary to the earlier described artifacts. This covariance matrix .EOG1 * 0.11. perspiration and glossokinetic(which is related to tongue EEG(1)' = EEG(1) + 1. Figure 1 gives an example for ECG reduction. The first eline the Figure 1: 2. The sampling rateelectrodes was 100Hz.5643 . in recording activity[28]. Technical Artifacts are sometimes also refered to as “Extraphysiologic Artifacts” or “Environmental Artifacts”.4ECG Technical Artifacts of the (second line) the QRS-complex in the EEG is attenuated. Furthermore can also be seen that EOG1 has a much bigger influence than EOG2.0160).: artifacts"EEG1 in an EEG recording[29]. A side-effect of the applied method is the covariance matrix obtained.g. In the first signal it can bee seen that the QRS-complex of the ECG contributes to the EEG signal.EOG2 * 0. The time axis is drawn in sample units. The ECG is also visible in the EEG recording(figure 2.8968 * ECG movements) artifacts can appear in the EEG-signal. 3 mV/sampl ). Furthermore many regression coefficients in Table 3 are much larger than zero. A segment of recording MR003S01 was taken and the recorded 2. 5050 corresponds to sample 475550 from the beginning of the might recording. between EEG1 and EOG1 is 1.8% 11 of the variance in EEG1 and can hardly be ignored. DISCUSSION As can be seen in Table 2.EDF is shown.1. different In EEG channels. After reduction 2.0844 .10. the Electrocardiogram(ECG) records potential differences in the cardiac muscle cell that occur during repolarization within each cycle of the heart other words it registers the heart’sMR001S03 electrical Indepolarization Table 2 can and be seen that the coefficients are different for beat[27]. 1 µV/sample ) and the EEG signal after ECG suppression(red. 65 60 55 50 45 40 35 30 25 20 15 5050 5100 5150 5200 5250 5300 5350 Figure TheECG channel C3-A2" of the recording MR003S01. That means that the recorded signals are correlated. The plot shows the ECG signal(green.771/30.5 Additional Biological Artifacts EEG1 was corrected using the ECG channel. original EEG in [µV].1. 1 µV/sampl e ).0.8312) than to channel EEG2 of a person’s neck[20].4 ECG EEG(1)' = EEG(1) . 0.ECG * 0. that the ECG influence to EEG 1 (C3-A2) is much bigger (0.11) with its influence depending on width and length (can be clearly seen how. 3 Aliasing Nyquist’s rule states that the sampling rate must be at greater than twice the fastest frequency that exists in the original waveform[30. the EEG can be assessed in almost any conscious Noise generated by state-of-the-art amplifiers has a very small amplitude compared to the EEG-signal(see figure 2. etc). Furthermore.4.7kOhm 10 Quantization noise -3 0 10 20 30 40 frequency f [Hz] 50 60 Figure 1: EEG and noise spectrum.5-30Hz) -1 Amplifier Noise (0. Additionally a notch-filter with a blockband at around 50 Hz . p. electrostatic noiseenvironments should be reduced by shielding the power source. coma) state. the Neuroscan SynAmps RT has a maximal sampling rate of 20. p. were recorded with a filter setting of 0.5-30Hz) Impedance noise R=10kOhm 10 -2 Impedance noise R=4. Both.4.e. 000 Hz1 ). 150 .2 Amplifiercontrol Noiseand artefact processing (Schlögl. .g. 88]. muscles in the head region (e.12) (awake) or unconscious (sleep. withratio a high impedance can causerange the wire running In this p. chewing.1. the Electrodes signal-to-noise is over a large frequency (0-35Hz) largerfrom these electrodes to function as antennae that picks up electrostatic noise[30. The electrical activity of these muscles have. the EEG and the noise. no large noise sources are in the data.025µV/Hz1/2 at any frequency above 1Hz 2.7kOhm. Both. 120]. 180 Hz. p. 120]. This undesired effect is known as aliasing.4. low-frequency electromagnetic fields can cause a If possible. the noise level of the from 0.12. Current state-of-the-art EEG-amplifiers do not have more than 0. speaking. 240 Hz.5 Hz to 30 Hz. the sampling rate was 128Hz. the disadvantages can be compensated.5 to 30Hz.: EEG and noise spectrum[31]. 2000). the EEG and the noise. If these factors are considered.cfm 12 . 1 http://www.com/synamps. With some effort in quality 2. quantisation noise is shown. with large.1. significant interference to EEG recordings.1.neuroscan. . If this rule is violated the resulting data may contain low-frequency components that are not present in the original data[30. Noise spectra of an EEG amplifier is compared with an EEG Figure 2. ) can be used to remove this artifact.2. . much large amplitude then the EEG. than 10. usually. 60 Hz and their harmonics( 100 Hz. However. were recorded with a bandpass filter(passband spectrum and the noise levels of 10kOhm and 4. Professional EEG-systems therefore come with a sampling rate that is high enough to accommodate to this(f. 10 Amplitude [µV/Hz1/2] 10 10 EEG and noise spectra 1 0 EEG (0. and 120 Hz. . 88]. 200 Hz . and is therefore negligible. Frequencies above the Nyquist frequency(one-half the sampling rate) will result in a set of frequencies between 0 and the Nyquist frequency[32]. TheHz second limitation is respectively the activity of . case.1 50/60 Hz Artifact A very common artifact is the interference of the power line with a frequency of 60 Hz in North America and 50 Hz in most other countries[30. . For the frequency ranges of the components measured by an EEG see table 2.1. than in parietal sites (including posterior cingulate gyrus. Accordingly. Although the propagation to the scalp of the P3aud P3b remains to be established. p. sMg). Data from Halgren et al. P3b is also elicited by infrequent events. The P3bP3a signal and measurement locations[12. 1. as t (msec) 2 1 200 300 400 was suggested independently by Kropotov et al. 1987.2 Typical P3a and P3b subcomponents of a P300 ERP signal and the measurement son. 2. or temporal sites (including paralocations gyrus. so that the sensory characteristic of each was. is a positive ERP that occurs 300ms after an event the subject has been instructed to generate some kind of response[12. P3a. (1995). and that post-lobectomy studies suggest200 that cingulate gyrus. Thus. The P3aud (not indicated) is generated in the superior temporal plane to auditory stimuli only. the negativity at 330 ms is termed P3aud (Fig. We alternated the tone pitch that was rare. sometimes also P3.2 shows typical P3a and P3b subcomponents.14). The P3a is evoked by rare stimuli. positive and al. overtly attended or unattended. a bigger MMN results in a dominant P3a. EEG Signal Processing can be also called “novelty ferences in sensory characteristics. (1995). The earliest potential was a large positivity superimposed on early components and peaking at 150 ms. 1995). Brain areas responsible for P3a andIt isP3b[34. It is associated with an electrodermal response and represents the cortical component of the orienting response. The ERPs are the responses to different stimuli. focal t (msec) 200 400 and/or inverting in polarity. aCg. (B) The depth P3b is later in the anterior hippocampus (aHC) and system for the orientation of attention. P3a identical when averaged across blocks. These were generated in the P3a superior temporal plane.1. barking. and is insensitive to attention. Its principle generators are in the hippocampus. 159] regardless of whether theyFigure are targets2. 3]. They P3b were insensitive to instructions to ignore the stimuli. 1).2. The P300 consists of two overlapping components: P3a which is related to novelty. and (probably) intraparietal sulcus. unlike P3a. At all sites. ventrolateral prefrontal cortex. novel or salient. the MMN is followed bythis thedifference P3a.2.5 Event-related Potentials Event-related Potentials(ERP) are small changes in the electrical activity of the brain recorded by an EEG and triggered by some internal(cognitive tasks) or external(stimuli) event[30. and Brodman’s area 46 in the dorsolateral prefrontal the generation of the P3 is partly modality-specific (Johncortex. Subsequent components could be large. / Electroencephalography and clinical Neurophysiology 106 (1998) 156–164 Fig. and usually included a positiv3 4 300 ity at 230 ms and a negativity at 330 ms.13. it is task relevant or involves a decision to evoke this component. The P3b (lighter areas) is evoked by attended visual or auditory stimuli that must be definitively processed. Halgren et al. p. All components in this area were specific to the auditory modality. N400 represents a negative peak 400 ms after the event. auditory or visual. the depth P3a is earlier than the Figure 2. superior temporal sulcus. on average. 1980. cortex However. Figure 3. Baudena et al.. p.e. 2.14. Note that positive is upwards and negative downscalp P3. 1995a. that is an event about that the subject has not been instructed prior to the experiment[12. Using these manipulations it was possible to demonstrate a series of potentials t (msec) 200 400 that were related to rarity per se (as opposed to sensory 300 differences or to habituation).: 3. It is important to distinguish between the ERPs when they are the response to novel or salient (i. Similarly. and supramarginal gyrus.13). sounds elicit larger P3a than identifiable sounds. and dorsolateral prefrontal A neural event is the frontal aspect of the novelty P300. Positive and negative potential changes are commonly labeled with either “P”(for positive) or “N”(for negative) and a number corresponding to the time of their appearance relative to the event[8.hippocampal 130]. 39]. frequent or distractor across blocks. the posterior parietal cortex (PsP) than at the scalp. habituation and subcomponents The P300’s latency(see figure 2.b.: or non-targets. p.b) and Baudena et this case are highly deviant environmental sounds such for as the a dog Nonidentifiable is a triphasic waveform with sharp negative. 159 E. Figure 3.1. Consequently. p. is small and insignificant for many depth sites. response earlier to distractor than to target stimuli. Latencies of P3a (A) and P3b (B) in different brain areas.isFor example. i.e. (A) The generated in the superior temporal plane propagate strongly t (msec) P3a has a significantly shorter latency in frontal sites (including anterior 300 400 to the vertex. and generally inverted across the Sylvian fissure. P3a. the i..1. a46). For all sites and wards(see remark 1)! inferior parietal. if the components. Summary of brain areas where P3s were found to be generated by simple rare stimuli in signal detection tasks. but. can be distinguished by their spatial location(see figure 2. pHg).e. a frontoparietocingulate system that has been associated with the orientation of attention (areas shaded dark in the figure). generated inp. Smith et al. Stapleton and Halgren. When only talking about the P300 most of the time it is referred to the P3b subcomponent while the P3a is that typical auditory oddball tasks confound rarity with dif130 P300"". one may note that other potentials Fig. Lateral (left) and medial (right) views are shown.. The eliciting events in The most widespread intracranial response to rare stimuli especially P3a. 1989). cingulate. 129] such as an unusual sound or image[33] and the P3b which is elicited after a task-related event in an oddball scenario(see remark 2). what 13 . 1990. the positivity at 150 ms may correspond to the mismatch negativity.6 P300 The P300. event is sufficiently deviant. pCg. Diagram based on recordings from about 4000 intracranial sites (Halgren et al. (1995a. 129]. &-()&+$" 6)K#+@ ?HTTOAJ a#+ #& )=B ?/03SA L)*KD.&*. !" #$%&'()*%'(+& +.'#==#.)=D*7#.=$%% #& )=B ?HTTPA ?L0SFAE &)*KD.+"@ />#"&D./01.)@+"@ $% 7$8#.'#==#.+@'& 7)"#=A 7.'#==#.#).$47-&# &'# &+4# . p.# &'#and #>#"&decreases $.9+" #& )=B ?HTTOAJ band due to a certain event and have been described by Pfurtscheller and Aranibar[37][38].>)= (#&8##" &8$ .*B ?HTTYAJ 1#%#(>.#* $% /012/03 4#)*-.$>#.1.E #*7#.' ). X.#**+"@ ?4$9+:#9 %. 1842].# 4#&'$9 >$=-"&).)*# $% ) %$.-#".+>)&+$"* .@#** )"9 5.-&+># #>#"&* *'$-=9 =)*& )& =#)*& *$4# *#.$"9* (#%$.# +">$=>#9E &'# +"&#. *)47=#* ).#%#.#=)&#9 7$8#. It has first been used in 1975[36].' #& )=B ?HTTPA \=$. p.-)"&+:.&*.=-9#* &'# %$==$8+"@ *&#7*M #& )=B ?HTTYAJ Z'-)"@ #& )=B ?HTTVAJ 6)@")"+ #& )=B ?HTTPA X)=.*# $% /01 _$47=#G 9#4$9-=)&+$" L#47$.)"*%$.'$" #& )=B ?HTTYA `$@)8) #& )=B ?HTVYA 3)=4#=+" #& )=B ?HTT[A done[39. bandpass filtering the frequency band that should be visualized L'# .D#>#"&D+"&#. -.+$9 ) %#8 synchronize) *#.#"&)@#A &$ &'# 7$8#.$ #& )=B ?HTTQAJ U%&)")* #& )=B fact that the band power increases when more neurons work in vice versa[39. synchrony(they ()*#=+"# 7#. 2 4#&'$9 2.%)7)DC ##E F#GGGH #IJKL#IMN L)(=# H R-)"&+:.$.$4S%-.< ()"9* Event-related synchronization(ERS)/event-related desynchronization(ERD) denote increase/decrease in a frequency ?HTVTAJ C$+&#" #& )=B ?HTTNAJ +* 9+*7=)<#9 .(6'77'+8 9":" . *)47=#*J OB )>#. squaring the ?L3/A amplitude samples( µV ) to obtain power samples( ) $% )== #>#"&D. averaging across trials ?HTTPA 5#.$-.+$9 .')"@#* +" $"@$+"@ //526/5 "##9 p.)&#* /01 )"9 )" +".all +".#**+># 4$9#=* )"9 of ERD/ERS in a frequency band $% )& =#)*& )77. 14 .$47$*+&+$" 1844]: .-^#=+#.$"*#. The subject is being told to execute an action(such as mentally counting the number of appearances of this event) whenever the target event occurs[35.4)&+$" (# #>#" =$"@#. $% &'# 1-W).$G+4)&#=< HI * +* . It is used to create a stimuli that elicits certain ERPs such as the P300.$"9*B F" &'# .#=)&+># ?)* 7#. ?HTTYAJ _=$.+7=# $% /01 ?=#%& 7)"#=A )"9 /03 ?..15.+".)-*# #& )=B ?HTTYAJ 3&#.+)= >).+"@ 8+&' /01 )"9 /03 7.. 4254].)= 9#.17 shows some of the frequency bands that have been named and scientifically examined since the invention of the EEG.#. 4#&'$9 S%-.# #& )=B ?HTTYAJ C#.. Those steps are9#.#@.#.15 NB *.#".#=)&#9 . The first category(the standards) appears with a probability of around 80% while the second category(the target) has only a chance of 20%.&*.&.=)**+.16).-).'#==#.#=)&#9 7$8#.)<'. #& )=BE HTTTA 2.< =+4( 4$>#4#"& *&-9+#*E )" +"&#.&*. %#)&-.+"@ &'# .$** )== &.)*# $% F"&#.+)" )"9 S%-.#* C)"9 7$8#.)= 7#.#)*# 3.9#9 9-. /03B `$&# &'# 9+%%#. +"9+./2 !"# $% &'# ()*+.. 2.#.1.#"& =)(*B ?6$9+:#9 %. )"9 U.$.#)*# also?L0S1A visualized in figure 2. HPQQ 3" 4-*+5.>)= To compute the time course the following steps are most commonly being U-&$.'#.)= #>$=-&+$" +".#. 8+&'+"the +9#"&+:#9 %. )"9 S%-.8 Frequency Bands Table 2.#)*# $% ()"9 7$8#.#"& &.$4 S%-.+)".)&+$" $% /012/03 +" &'# &+4# 9$4)+" -*#9 +" 9+%%#.& &.&.&.$.+)==< 8'#" )=7') ()"9 ..)-*# #>#"&D.&.# X=+4#*.#**+"@B U 9#..4)" &+4# &$ 9#>#=$7 )"9 &$ .)"+(). p.E HTTTAB Figure 2.'#==#. ?HTT[AJ X=+4#*.B 1.: ERD/ERS Processing[39./0-..+)=*J \+@B OB S.)&+$" 4#&'$9 0#%#. 1844] ERD/ERS is commonly being analyzed in ERD/ERS maps that show the band power of frequencies over time(see figure 2.&*.-.Remark 2 (Oddball Paradigm) The oddball paradigm is a technique where the subject is confronted with two categories of events that are presented in random order. $. The names arise from the *)4# //526/5 9#.$44#"9#9J +" &'# *7#.#".' #& )=B ?HTTQAJ L$. ?HTT[A C-.1 (& '(23 %&4 56%*3 !"#" $%&' ()*+.#=)&#9 *7#.7 Event-related (De)synchronization /01 .#7#.>)= *'$-=9 ]+=(#.#4#"&* +* &')& &'# //526/5 7$8#.)= 4#&'$9 &$ .+)=*J HB ()"97)** µV :=&#.+"@ $% &'# )47=+&-9# *)47=#* &$ $(&)+" 7$8#.#=)&#9 &.'<&'4* ).&+$" &+4# &)*KE &'# +"&#.+@@#. => 2%7?> 1 @7%A%(>7 B'*+)<6C.#)*# $% ()"9 7$8#.. 82 Fig. more Alpha data and/or larger differences are necessary to Beta gain sufficient statistical evidence.18. As shown in the color legend below the map. Bootstrap techniques are an alternative to assumptions cannot be made for ERD/ERS values. even ones on the right hemisphere. Here the center frequency of each band is denoted on the scale on the left of the map.16. asymptotic methods. In this setting each electrode is placed with a distance to neighbouring electrodes of either 10 or 20% of the whole saggital/coronal distance. values from !100% to +150% are scaled appropriately as various colors. basic idea by is simply to tions that are no longer approximated by GaussianHz). 1. and thus ignores a Statistical significance of ERD/ERS 2. That is. respectively. Therefore. Each position is uniquely identified by up to two letters representing the saggital.: Named Bands[30. Another powerful way to caldistributions of the values to be analyzed or they culate the confidence and significance of ERD/ require a large number of samples (or trials in the ERS values is the bootstrap (Graimann et al. The time courses on the left also show the corresponding confidence intervals that can be used to display only statistically significant values (see Section ‘‘Statistical significance of ERD/ERS’’). case of ERD/ERS analysis). Apart from its Hz). 1995). Properties of the gram of ERD/ERS values for exemplary alpha and processing and large amplitudes on the other hand reflect the inhibition and disengagement of task-irrelevant cortical estimator such as confidence intervals are then beta bands. whereas dates for theand statistical assessment processes of ERD/ERSrelated tant advantage is that there is no need to assume a values. A method for performing this statistical 8-12 Hz analysis is given below.2) and an index denoting its coronal location.2. (Davison 1997). 12-30 Hz 30-80 Hz Name Delta considerable Theta amount of information. The time course of the bandpower for each frequency band of interest is calculated and displayed as colors representing the instantaneous proportional change relative to the baseline average. Figure 2.1 Electrode Positioning The 10-20 electrode setting for 21 electrodes.2. almost any parameter of the the sign test reduces the numerical data to a qualbootstrap distribution may serve as a bootstrap itative form (plus or minus). (see table 2. resampling replace the unknown distribution with distributions.2 Hardware 2. the sign test is not very powerful. hence the name.: Abbreviations in Electrode Positioning[2] 15 . Name A C P F Fp O T Location Ear lobe central parietal frontal frontal polar occipital temporal Table 2. Alpha 1 ERD is Nonparametric thatdifferent are not based on alpha The alpha band can be separatedmethods into two bands: 1(8 and − 10Hinkley. data. visual stimuli the data. Assembly of event-related desynchronization and event-related synchronization (ERD/ERS) maps. Odd indices can be found on the left. Furthermore. (2005). both distributions areas[41].17. these 2. determined based on the empirical distribution are skewed toward the zero power.1. Figure the normalized histo-alpha and increased attentiveness[41]. It3isshows believed that small amplitudes are population an indicator for regions of active neuronal the known empirical distribution. depicted in figure 2. while the time in seconds appears under the map. p. Clearly. Gaussian or other parametric distribution on the by (Kalcher and Pfurtscheller. Hz) and alpha 2(10 − 12 general applicability and simplicity. however. This information (examples on the left) is assembled to form the map on the right.: Assembly of ERD/ERS maps[40] Frequency 0-4 Hz formulation of erroneous conclusions concerning 4-8 Hz the data. The bootstrap is a method The necessary nonlinearity (squaring) in the quanfor estimating the distribution of a test statistic by tification procedures yields highly skewed distribuAlpha waves are the oscillations in the alpha band(8-12 Its amplitude is The suppressed eye-opening. An efficient approach Gammafor calculating statistically significant ERD/ERS values by using parametric tests followed by Box–Cox transforms to approximately Figure 2. the bootstrap the Gaussianity assumptions are possible candisaid to represent attentional motivational alertness and is topographically has to a number of other advantages.an a simple sign testsensory-motor was proposed alpha 2 ERD is more localized may reflect processing and possibly semantic encoding[42]. The most impor-widespread. has been recommended by the International Federation of Societies for Electroencephalography and Clinical Neurophysiology[43].9 Alpha Waves 2002). Unfortunately.. was 230] normalize the samples suggested in ZygierMany statistical methods assume either GaussianFrequency ewicz et al. In fact. : 10-20 electrode setting for 75 electrodes[12] the reference electrodes: (a) and (b) represent the three-dimensional measures. There are also reference-free recording techniques.18.: Epoc’s Electrode Positiong[46] 16 . one or two reference electrodes are used.: 10-20 electrode setting for 21 electrodes[12] Figure Figure 2. increasing the sensitivity for the recording from basal subtemporal structures [43]. C7 . hand. In many applications such as brain–computer interfacing (BCI) and study of mental Two different modes of recordings. ipsilateral ear. F4 . In the activity. linked-ears. however. references. additional electrodes may be placed equidistantly in between already defined T3 T5 O1 20% 10% figure 2. The only difference between this system and the 10–20 conventional system is that the outer electrodes are slightly lowered to enable better capturing of the required signals. Also F3 . Other deviations from the international 10–20 system as used by researchers are found in References [44] and [45]. as has been suggested in the American EEG Society’s guidelines[44](see T6 T EEG Signal Processing Pg1 5 10% Preauricular point P3 OA1 Pz O1 Inion 10% 20% Fp1 Nasion 10% Fpz Fp2 20% 10% 20% T3 20% T5 F3 C3 Fz 20% 20% P3 O1 20% 10% F4 20% Pz 20% Oz F2 F9 C4 Cz 10% T6 T9 T7 TP9 TP7 P9 O2 F7 Fp1 Fpz Fp2 AF7 AF 3 AFz AF4 F5 F3 F1 Fz AF8 F6 F4 F2 F8 F10 FT9 FT FT10 7 FC5 FC FC6 FT8 3 FC1 FCz FC2 FC4 T4 A1 P4 20% Nz 20% F7 O2 Inion (b) (a) P4 C3 C5 P7 P5 PO7 10% Inion C1 Cz C2 C4 C8 T8 T10 A2 CP5 CP3 CP1 CPz CP2 CP4 CP6 TP 8 TP 10 P1 P3 Pz P2 P4 PO3 POz PO4 O1 Oz O2 P6 P8 P10 PO8 Iz Figure 1. the conventional 10–20 system has been modified to capture better the signals from epileptic foci in epileptic seizure recordings. linked-mastoids. contralateral ear. EEGSeveral signals during approximately seven seconds of normal adult brain activity. The choice of reference may produce topographic distortion if the reference is not relatively neutral. Nasion positions. Figure 1. often a small number of electrodes around the movement-related regions are differential mode two inputs each differential amplifier are from two electrodes.2. the choice of a reference does not play an important role in the measurement [41]. and tip of the nose [28]. In another similar setting. P3 .10 illustrates a typical set of 2. namely differential and referential.Nasion 10% Pg1 Pg2 Vertex 20% Fp1 Cz 20% Fz 20% Fz F4 F8 Pz P3 20% 20% F 16 Fz F3 F7 C3 F3 20% Fp2 20% A1 T3 C3 + Cz C4 20% T4 A2 7 10% If a larger number of electrodes is needed. C3 and C4 can be used to record the right and left finger movement related view of the electrode setup configuration signals respectively for brain–computer interfacing (BCI) applications. bipolarcontroller. and (c) indicates a two-dimensional For example.19). different reference electrode placements can be found in the literature. and P4 can be used for recording the ERP P300 signals.8 Conventional 10–20 EEG electrode positions for the placement of 21 electrodes (c) 1. The overall setting includes the active electrodes and the references. In modern instrumentation. The advantage of this system over the conventional one is that it provides a more extensive coverage of the lower part of the cerebral convexity. which actually use a common average reference. or leg electrodes may be used [42]. called the Maudsley electrode positioning system.19. In such systems other references such as FPz . are used. In used from the 10–20 setting system.2the Emotiv EpoctoHeadset selected and the referential mode. Physical references The Epoc is a wireless EEG system developed by Emotiv Systems[45] that aims to become the next generation game can be used as vertex (Cz ).20. Figure 2. on the other hand.9 A diagrammatic representation of 10–20 electrode settings for 75 electrodes including Figure 2. F8.: Cognitive Suite http://www. AF4. O1. FC5. FC6. the connection quality has to be checked for each sensor individually from within the Emotiv Control Panel(see figure 2. in order to access the raw data the routines of the supplied SDK that come in form of a C library(that currently only runs on windows) have to be used.22) allows to recognize the users’s facial expressions and visualizes them in an avatar. etc. T8. F3. Since the data being transmitted by the headset is encrypted[48].24. Prior to use. notch filter at 50 Hz and 60 Hz The resulting signal is then being made available through the API. low-pass filter with a cutoff at 85 Hz 2.22. P8. Finally. The headset also includes a gyroscope[46] which is intended to be used for controlling a player’s view in computer games. T7.16 Hz 3. O2(see figure 2. F4. high-pass filter with a cutoff at 0.: Expressiv Suite Figure 2. after a very short period of training for each action(at least 10 seconds). but could also be employed to detect artifacts caused by movements of the head. happiness. All of the applications are also scriptable and can be used without any knowledge about the EEG-signal. P7. The Epoc internally samples at a frequency of 2048 Hz which then gets downsampled to ≈ 128 Hz[49]. Afterwards the following preprocessing steps are being done in the hardware[49]: 1.: Affectiv Panel Figure 2.23) interprets the user’s conscious thoughts and intent such as long-term excitement. The “Cognitiv Suite”(see figure 2.: Emotiv Control Panel Figure 2. all felt pads on top of the sensors have to be moisturized with a saline solution2 .Emotiv’s Neuroheadset is equipped with 14 saline sensors: AF3.bausch-lomb. The Epoc’s SDK already includes 3 readily implemented applications: The “Expressiv Suite”(see figure 2.23.de/kontaktlinsen/renu/multiplus. 2 Figure 2. Its battery lasts for up to 12 hours and is therefore also suited for recording sleep behaviour. boredom. F7.24) shows a box in a three-dimensional room which can be moved by the player’s thoughts.htm 17 .21). The “Affectiv Suite”(see figure 2.21.20) and two additional sensors that serve as CMS/DRL reference channels(one for the left and the other for the right hemisphere of the head)[47]. 1.2. their phone number is automatically dialed. have been trying to pair an eye-tracker with the Epoc with the goal to help programmers find mistakes in source code easier and then resolve them automatically[50]..11 (2. They arranged the images of 6 phone contacts side by side. The 50s 77. EEG signals from the headset are transmitted wirelessly to an iPhone. this introduces delay cessing and classification algorithms running on the phone. Classification results under different conditions and different time ranges that were averaged are shown in table 3. meditation.67% headset transmits encrypted data wirelessly to a WindowsP7 based machine.: Neurophone UI[46] person whom the user wishes to dial. Unfortunately. the study seems to be at a very early stage as there have not been any publications from them that show any results or details on their approach.11).using as we state in our vision. they had to employ a Windows we plan to study the suitability of other neural signals as of on-going work.67% ates in the same frequency range as 802. frustration. 3. we discuss a number of design considersignals (Figure 5) are much more easily detectable in raw ations that relate to building a reliable and robust NeuroEEG data than P300 signals (Figure 4).89% after highlighting the different images for 100s in total.3 Design Considerations However. can besee extremely data over many trials.2). P300) from it through smart signal pronique in neuroscience [10]. detected a multivariate classifier with the component headset put on (ICA) in reverse(leading to headset can be easily deployed at large scale because of its A sensible approach to increase the SNR is to average the two channels low above 4.33% O2 ternational 10-20 system and are labeled as such [9].the andeyes.44% in33. NeuroPhone is the name of a study by Campbell et al.: NeuroPhone Contact Classification Results Source[46] (referred to as “Expressiv” by Emotiv).4Ghz).1. P300. The FC5 software that comes with the Emotiv headset provides the F3 following detection Table functionality: various facial expressions F7 3. SDK that can not be circumvented as the signals are being encrypted when sent from the PC to be able part to use the Epoc’s AF4 F8 device to the PC. Also built in the headset is a gyroscope Figure 3: Raw data from the headset that detects the change of orientation of subject’s head. 0 1 2 3 4 5 subject-specific training and detection of certain cognitive Time (second) neuro-activities such as “push”. and 1).3 Related Work Andrade et al. which natively runs a simple classifier to discriminate P300 signals from noise.82% 66. because we need 18 to average several trials before actually detecting the P300. 3. thus when his picture is flashed.78% roughly the 44. Naturally. they have not been given any insights on the channels they were using to detect the it is challenging to extract finer P300 signals from the EEGs headset But. despite them not knowing about the headset itself. Even though the iPhone principally is capable of connecting external devices via bluetooth. in the acquisition of a reliable P300 signal.1. the wireless chip is proprietary and oper100s 88. averaging and independent component analysis to extract the headset is not meant to be an extremely reliable device. Unfortunately. The Epoc seems to be perfectly suited for Second Life as it already comes with a programmable SDK that features the learning of certain actions which can be used to control movements. Though.handy if we can extract price. thus P300 component. The group around Hansen has been evaluating the Epoc as a possible controller for Second Life with the goal to make it easier for disabled people to socialize without the necessity to leave their house[51]. in wink mode we can avoid averaging because wink In what follows. Tim is automatically dialed.67% O1 66. Bayesian this tering [9] and independent analysis [14]. they have found that allowing the participants to put on the headset yielded better results. When a person’s contact-photo triggers a P300. and “lift” (“Cognitiv”) [2].89% 88. The electrodes are20s placed in77. “rotate”. we can still improve the EEG signals for better .2 Wireless EEG Headset F4 Electrode FC6 We use the Emotiv EPOC headset [2] which has 14 dataTime Sitting Music (Sitting) Standing T8 collecting electrodes and 2 reference electrodes (see Figures P8 6.g. Blinks this have beenproduces. which is also a commonly used techuseful signals (e. 3. and excitement (“Affectiv”). In this case. that uses the Epoc to dial a contact on the iPhone[46]. the user wants to dial Tim. then highlighted them one after the other and thereby created an oddball paradigm(see remark 2) that would elicit a P300 whenever the contact that the user wanted to call showed up.89% T7 66. Phone system. Signal Processing: Although we are averaging data for Signal to Noise Ratio (SNR): Since the Emotiv heada better SNR. The Their data processing steps included bandpass filtering. In the sitting condition they could identify the correct contact with a chance of 88. levels of engageAF3 ment. “pull”. Also it is capable of recognizing facial expression and emotions(see section 2. Figure 2: The Dial Tim application works on similar principles to P300-speller brain-computer interfaces: the phone flashes a sequence of photos of contacts from the address book and a P300 neural signal is elicited when the flashed photo matches the Figure 3. 3% Math vs. the EEG used in this study required conductive paste to be put on top of the electrodes which then later on had to be washed off of the proband’s head. Rotate 83.[7]. 3 tasks 68. Further.Lee et al. used a low-cost EEG for the classification of 3 different tasks: rest. 19 . albeit with $1.17).5% Rest vs. Rotate 82. To be able to use the bayesian network with time-series data they divided the dataset in 2-second windows overlapping by 1 s.2. mental Arithmetic and mental rotation.500 much more expensive than the Epoc.9% Table 3. Math 86.: Mean Task Classification Accuracies[7] For the classification they used a bayesian network trained with various features extracted from the six standard frequency bands(see table 2.8% Rest vs. A more precise estimate of the actual rate can be obtained by counting the recorded samples for a longer time period(algorithm 4. Consequently.1.1 FixTimeStamps(SAMPLING_RATE) miliSecondsPerSample ← 1000/SAMPLING_RATE referenceTime ← nil n←0 for all sample do if referenceTime = nil then referenceTime ← getTimeStamp(sample) end if setTimeStamp(referenceTime + (n ∗ miliSecondsPerSample)) n← n+1 end for To use this algorithm. many consecutive samples shared the same timestamp and at times there wasn’t any data assigned to time periods up to one second. Algorithm 4. the correct sampling rate of the device needs to be known.2 EstimateSamplingRate(MEASUREMENT_PERIOD) startTime ← now() elapsedTime ← 0 n←0 repeat sample ← getNextSample() if (sampl e) 6= nil then n← n+1 end if elapsedTime ← now() − startTime until elapsedTime > MEASUREMENT_PERIOD return n/elapsedTime 20 .1. the samples should be given new timestamps according to the sampling rate of the device as is explained by algorithm 4. Since the SDK for the Epoc came in form of a dynamic link library(dll) for Windows written in C. It should be made sure that the device stays within range of reception during the recording. Before discovering this problem.1.2 Determination of the Sampling Rate The timestamp that comes with each sample is being set in the Emotiv library using Windows’ time calls which get sent through the Windows queueing system and are therefore very unreliable[52]. The producer of the Epoc indicates that the sampling rate is roughly 128 samples per second but may vary slightly from device to device[52].4 Signal Processing 4.1 Obtaining the Signal 4. to prevent the loss of data packages.1 Creating the Java Wrapper One requirement of the project was to be able to work with the signal from within Java. 4. Algorithm 4. a way had to be found to make it accessible from Java.2). The solution is presented in Appendix A. 3 Removing the Baseline It is a common practice to remove the baseline of the dataset for each electrode individually.00347311534348034 sample s sample 28792594 100 · = 1 sample ⇔ t er r = ms = 287. As the experiments in this thesis will be only between 5 and 10 minutes in length.2 has been run with the Epoc device used throughout this thesis for 8 hours and yielded to a sampling rate sample . of ≈ 127. 3681940 samples have been counted: f1 = 3681940 28792594 · sample ms ≈ 127. 4. so that the values of the signal will be distributed around 0. In this recording period of 28792594 ms(≈ 8 hours). As the work in this thesis might be later extended to be used online a buffer has been used to remove the mean of the last n samples instead of the baseline of the whole recording. blinks) that can’t be done.1. that is to remove the mean of the recording.92594 s ≈ 4.878 s In contrast to the depicted algorithm.881496193084 28792594 ms s The maximal error in the sampling rate f1 is therefore: f er r = f2 − f1 = 100 28792594 · sample ms ≈ 0. t er r · 4. If more accuracy is needed. This leads to a maximal possible error of up to 100 samples in case the sample buffer has been fully re-filled while being flushed and read out.8780230777 sample s Assuming an error of 100 additional samples that should have been counted in the same interval: f2 = sample 3681940 + 100 sample · ≈ 127. but for some types(like f. the measurement could be slightly extended to up to 12h in total(before the Epoc’s battery 0 runs out of power) giving t er r ≈ 7. It is generally best to avoid the artifact in the first place.8 min 28792594 ms 100 After t er r = 4. since the buffer can not be used to retrieve a meaningful mean as long as it has not been filled completely. minimization and identification/rejection.1. this is an acceptable estimation. There were no incidents of a buffer overflow during this measurement and the buffer has been cleared at the beginning to avoid the inclusion of additional samples before the onset of the measurement. This chapter discusses some methods that allow the work with noisy EEG data. The first 64 samples have been removed. 100 samples have been retrieved at once instead of each one independently for performance reasons. 21 .3 BaselineRemoval(x) 1: size ← 64 2: initBuffer(buffer. 2 min.2 Artifact Removal There are 3 ways to deal with artifacts: prevention. x i ) if n ≥ size then yi−size ← x i − mean(buffer) 7: end if 8: end for 9: return y 5: 6: An example for the removal of the baseline is given in figure 4. Minimization of the artifact is mostly very expensive and involved so that the rejection of polluted samples remains as the most feasible alternative.1.e. The buffer has been chosen to have a width of 64 samples which equals half a second at a sampling rate of 128 Hz. size) 3: 4: for i = 1 to n do push(buffer.Algorithm 4. Algorithm 4.8 minutes the recording will be up to one sample off if f1 is used as a sampling rate for algorithm 4. 1 the Epoc has been mounted reversely leading to two electrodes positioned over the forehead: one over the left and the other over the right eye. mean. 1) for t = 1 to n do if y t > PEAK_LIMIT then if t − CLUSTERING_DISTANCE ≤ lastArtifact2 then {extend the last artifact} lastArtifact2 ← t else {add a new artifact} if lastArtifact 6= (1.1 Blinks In the setup of the first experiment that will be explained in section 5.Figure 4.4 findPeaks( y . stdDev) 1: PEAK_LIMIT ← 3.: Before(top) and after(bottom) the removal of the baseline 4. Those two electrodes will record very few brain activity and respond very well to movements of the eye and blinks and are therefore predestinated to detect EOG artifacts.5).2. lastArtifact ← (1. see algorithm 4. 1) then artifacts ← artifacts ∪ lastArtifact lastArtifact ← (t.5 · stdDev 2: CLUSTERING_DISTANCE ← 300 3: 4: 5: 6: 7: 8: 9: 10: 11: 12: 13: 14: 15: 16: 17: 18: artifacts ← .1. Algorithm 4. t) end if end if end if end for if lastArtifact 6= (1. To detect blinks the mean and standard deviation of each near-eye electrode has been computed and then scanned for samples exceeding a standard-deviation-related limit(see algorithm 4.4). 1) then artifacts ← artifacts ∪ lastArtifact 21: end if 19: 20: 22: return artifacts 22 . Those samples have been joined into nearby groups and finally fitted(extended to the left and right until the value has fallen below another standard-deviation-related limit. 5 · stdDev 2: 3: 4: 5: 6: 7: 8: for all artifact ∈ artifacts do t ← artifact1 while t ≥ 0 and y t ≥ REGULAR_LIMIT do {extend to the left} artifact1 ← t t ← t −1 end while 15: t ← artifact2 while t ≤ n and y t ≥ REGULAR_LIMIT do {extend to the right} artifact2 ← t t ← t +1 end while end for 16: return artifacts 9: 10: 11: 12: 13: 14: Figure 4. Figure 4. 1st row: right eye. artifacts. 2nd row: left eye Epochs that were contaminated with blinks have been marked and excluded from further processing.: Detected artifacts in the EEG-recording(yellow).Algorithm 4. stdDev) 1: REGULAR_LIMIT ← 0.2. 23 .5 fitArtifacts(y.2 shows the detected EOG artifacts in an EEG-recording. 2 Bandpass Filtering To cut off frequencies that are not related to the task. . Then the filter algorithm 4. This signal was then filtered using the algorithm 4. that is half the sampling rate: ≈ 128 Hz/2 = 64 Hz.html 24 . β ) 1: z1 ← 0 2: z2 ← 0 3: r ← 0 4: ∀i ∈ {1.1 2nd -order IIR-Filter For the filter design. The long time for the dataset has been semi-arbitrary chosen to be really sure that the filter has finished oscillating and reached a steady state at the end of the signal.2.3).dsptutor. 43]. For the detection of the P300 an IIR-filter of the second order with a passband between 2 Hz and 8 Hz was used which has been identified as the P300’s main power spectrum in the BCI Competition 2003[53]. 1 http://www. The z −1 block represents a unit delay.freeuk.4.4. a signal with combination of sine waves with different frequencies has been artificially generated.1 Hz up to 100 Hz) and a constant amplitude of 1 µV with a length of ≈ 22 min. α. This has been done already in the Epoc hardware itself(see chapter 2.. 9: 4.3. the code behind Ian Robin’s IIR Filter Design applet1 has been used to obtain the filter’s alpha and beta coefficients for given cut-off frequencies. n} yi ← 0 10: for i = 1 to n do r ← β 1 · z1 + β b2 · z2 yi ← α0 · (x i − r) + α1 · z1 + α2 · z2 z2 ← z1 z1 ← x i − r end for 11: return y 5: 6: 7: 8: α2 Figure 4.3 which is based on the following difference equation[54. The remaining differences can be explained by the low order of the filter. The results can be seen in figure 4.: Scheme of a 2nd -order IIR-Filter in direct form II[55.com/IIRFilterDesign/IIRFilterDesign. p.2.1.6 has been implemented using the scheme that can be seen in figure 4.4. 4.2. 1]) a bandpass filter might be used to cut out this band from the rest of the signal. Overall.6 and plotted along with another sine wave combination consisting only of the frequencies that should be remaining after applying a perfect filter.2. p.2. a bandpass filter has been employed.2) and therefore does not have to be considered here. p. the filtered signal(row 2) shows a close resemblance in both amplitude and wave form to the correct signal without the disturbance frequencies(row 3).6 iirFilter( x .. This filter(passband from 2 Hz to 8 Hz) further has been applied to different sine waves with varying frequencies(starting at 0 Hz going in steps of 0.2.2 Validation To test the bandpass filter. For each filtered signal the maximal amplitude of the last ≈ 8 s has been determined. the signal should be at least low-pass filtered with the cut-off set to the nyquist frequency(see chapter 2. To visualize the oscillations in certain frequency bands such as the alpha wave(8 − 14 Hz[41. Independent of the task. 313]: yn = 2 X βi x n−i − i=0 Π ∑ -β1 z-1 Π ∑ ∑ yn α1 Π Π Π ∑ z-1 -β2 α j yn− j j=1 α0 xn 2 X Algorithm 4. p.2 0 0 1 2 3 4 5 6 Frequency in Hz 7 8 9 10 Figure 4.4 1. 1. Alternatively. 25 . 4 Hz.4 0. 6 Hz and 35 Hz) and the second row the filtered(passband from 2 Hz to 8 Hz) result of the signal shown in row 1.6.: Gain of the 2 − 8 Hz bandpass filter from 0 to 100 Hz 4. that is the factor with which this frequency is being amplified/suppressed. Finally. In row 3 the sine wave combination(4 Hz and 6 Hz) is plotted. Since the amplitude of the unfiltered wave is 1.3 Averaging Especially event-related potentials(ERP) unfortunately have a very low amplitude compared to the noise in the signal. Signal-averaging is based on the assumption that the signal belonging to the ERP is invariant across epochs[30. That’s why most of the time the signal-average of multiple trials time-locked to a certain event is being considered instead.2. Amplitude] and in contrast that the noise is randomly distributed i. the amplitude at the end of the filtered dataset gives the gain of the particular frequency.5 Hz.6.2 1 1 0. 11. For the detection of ERPs such as the P300 this means that a single trial is not very significant as a peak may be caused by noise.5 and figure 4.4 0.: Gain of the 2 − 8 Hz bandpass filter from 0 to 10 Hz 0 0 20 40 60 Frequency in Hz 80 100 Figure 4.5.4 1.2 0.6 0.: Bandpass validation: the first row shows an the sine wave combination(0.6 0. the resulting gains have been plotted in figure 4.8 Gain Gain 0.2 1.Figure 4.4. the gain could have been determined by computing the norm of the filter’s transfer function in dependency of the frequency.e. but the chosen approach additionally considers the accuracy and correctness of the implementation.8 0. it varies from trial to trial and will therefore vanish in the process of averaging. 9.7. This event occured 41 times throughout the dataset.: The averaged signal of all epochs during the event “Column 0” An excerpt of the dataset has been visualized in figure 4.8 and 4.8. Figure 4.7.9 all trials of the event “Column 0” have been averaged out. Epoch-averaging. 4. an artificial dataset has been generated that consisted of gaussian noise with the mean at 0 µV and a standard deviation of 1 µV.3. In the latter case the signal has been bandpass-filtered(passband 2−8 Hz) before averaging the trials. It features the interactive plotting of EEG-recordings.: The averaged signal of all epochs during the event “Column 0” that additionally has been bandpass-filtered prior to averaging 4.: Excerpt of the generated dataset consisting of gaussian noise and during the event “Column 0”(green) a sinusoidal signal To validate the signal-averaging.Figure 4. the electrode placement and of course the raw signal in a format that EEGLab can import. In figure 4. During the event “Column 0” the signal of a sine wave with an amplitude of 1 µV and a frequency of 1 Hz has been added to the noise. Figure 4. Independent Component Analysis and Spectral Analysis. 26 .3 EEGLab EEGLab is an open-source MATLAB toolbox for electrophysiological research[56].1 Import/Export To export the data that has been recorded during the experiments to EEGLab a little tool has been created that exports the events. It can be seen that during the event the sinus-wave is very hard to be identified in the raw data of a single trial. 425391 0. CP3.725270 -32.389577 0. O2 .1. O1 . p. T7 .582877 37.725270 -34.788443 0.441968 0.455749 125.369777 0.. Each line is formatted as follows[57.492388 -47.321930 0. The signal have been saved in an ascii file where each sample is represented by one line listing all the raw channel values in the order that was specified in the location file.422641 0.. AF4.166427 64.045704 162.136735 -87. F4 . FC5..612754 47. FT7.627997 34.3.491221 0.: alternative electrode placement when the headset is put on reversely..663460 0.. CP4.369739 0. CP6..419583 -165.4.11). F7 . 12 of 12 electrode locations shown Figure 4.259532 0.654055 66.. T8 . C6 . F3 .986493 -116.253792 P4.226750 0.loc file format that contains one line for each channel in the order of their appearance in the signal file.398099 0. F8 .. P3. REYE LEYE AF7. CP5.286614 0.227868 0..430357 0.574506 -63..11.315310 0. P7 .315149 0. P8 .320078 0.344081 0. C5.329499 0.426297 87.354516 AF3.997081 116. Not seen here are the two channels(ID 7 and 8) that are located on the forehead over both eyes 4.. 27 .. FT8.105645 -38. 13]: <channelId> <polarAngle> <polarRadius> <label> The locations vary depending on the way the Epoc was put on: regular(figure 4.500972 -140.267805 21.687176 -34.1 Electrode Locations The electrode positions have been made available to EEGLab using the EEGLab polar .1.10) or in reverse(figure 4.424522 0.263866 -64.890370 -86.545549 -110. AF8.308052 0.140643 -128.10.232553 109.698621 86. Channel locations +X +Y AF3 F7 1 2 3 4 5 6 7 8 9 10 11 12 13 14 AF4 F8 F3 F4 FC5 FC6 T7 T8 P8 P7 O1 O2 -26..2 Events & Signal The events have been exported into an ascii tab delimited file containing both the latency at which the event occured and the name of the event.376674 0.. FC6.484866 0.3.004936 0. 14 of 14 electrode locations shown Figure 4.428948 0.: regular electrode placement when the headset is put on as intended Channel locations +X +Y AF7 AF8 FT7 FT8 C5 C6 CP5 CP3 P3 CP4 CP6 P4 1 2 3 4 5 6 7 8 9 10 11 12 13 14 136.788443 0. Each line hereby represents an event. ≈ 17%) as all possible selections(row/column) are being picked with an equal probability. The speller consists of a 6x6 grid of letters that is presented to the subject[59] who is then told to to averages samples from tobeing thehighlighted(see same stimulation silently pick aof letter and count how many epochs times the rowrelative or column was figure 5. Lines and rows are flashed 10 one at time that in do random and the intersection between the) than column The a selections not includeorder. In [2]. When the column/row containing the desired letter is flashed. therefore. the subjects (healthy and impaired ones) the headset was placed in reverse as it then covers P3 and P4 (see figure 4. then another round This very popular experiment has been chosen to serve as a benchmark to see just how good the quality of the Epoc’s of 12signal stimulations ismany repeated.1. This (same serves as row a stimulicolumn). which permits to spell words.[59] Fig. This can be used to provide locked-in patients with a new output channel independent of the motor system[60]. The visual stimulator for a P300-based BCI speller.1 Motivation columns is flashed exactly once in the first 12 stimulations. with different classification techniques.1. a virtual-reality system is presented where subjects operate objects selected 5. Classification is made by comparing the correlation of single responses with the averages of all target and non-target reThe optimal channel selection for the P300 speller is shown in figure 5. and P300 is recognized Krusienski[62][60]. and entire columns or rows are flashed one after the other 5. 1. control a cursor by choosing among four commands (up. and The P300 Speller an implementation the Oddball Discriminant Paradigm(see remark Analysis 2) first described by Farwell applied and classification is ismade throughofStepwise (SWDA) Donchin[58]. the target( P300 has been used in other BCI experiments. In [2]. as or same he then has to execute the task-related action of counting. called also Evaluation P3005 speller. The number of repetitions can be predetermined for each user to get the best trade-off between speed and accuracy. repeated stimuli are normally used to facilitate the detection of the correct stimulus. In [5]. right) via 28 the P300. Therefore through a genetic algorithm. Dal Seno. 1 for an example). flashing and columns done in a new is.2 letter.10) of the Epoc does not contain any of the selected channels.2 as has been identified by Dean J. Sportiello. Donchin and colleagues presented the first P300-based BCI.: The grid of the P300 Speller. independent component analysis (ICA) is used to decompose the EEG signals. Figure 5.1). A 6 × 6 grid of letters and symbols is presented to the user. epochs 1. a fuzzy classifier selects one of . Single-sweep detection is performed. In [3]. that elicits a P300(P3b) whenever the subject sees that his chosen letter(the target) is being highlighted[60]. a P300 is elicited. random order. the subject’s chosen letter can thus be identified. For example patients with amyotrophic lateral sclerosis(ALS) have been shown to be able to write using the P300 speller[61]. Either a row or a column is highlighted at the same time with an equal probability in a random order. 12 By analyzing which row and which column elicited the P300.11).1 P300 Speller in random order (see Fig. In [4] a wheelchair driven by a BCI is described. and this procedure is repeated for a predefined number of times for Introduction each 5.1. B.516 L. the subject’s letter hereby serve as the standards and arethe more row likely( and ≈ 83% 12 that elicit a2P300 indicates the selected letter. sponses. The regular electrode placement(see figure 4. Matteucci and.3 Headset Placement through the P300. Each one of the 6 rows and 6 5.1 s long are extracted around each stimulation. There have been studies aboutwith the P300 speller whichof is arows good basis to compare this device with others.1. and M. down. left. The predicted matrix symbol was deterDuring the experiment the instructor and theoftest personscores were for in the the respective room to keep the level mined asonly the maximum of the sum resultant row and col. The 8 electrodes selected for analyFigure Optimal channel selection for the P300 speller.1. resulting in approximately ·1500 = 2 100 ms 12 N 250 appearances of the target letter.5 Tools A tool named “P300 Recorder” has been written to: • show the grid and the intensifications to the proband as described earlier • record the Epoc’s raw data • record the intensifications in synchrony with the signal • save the recording in a way to be able to access it later on 29 . forming N scores. The resultant score for each intensification response was generated as a weighted have been informed that they should try not to make any facial and tongue movements. The proband’s chair was oriented such that he would only see the monitor in front of a wall and no umn intensifications. fier in the ensemble. Each intensification lasted 100 ms. but were told to give (1) a sign when they felt predicted ofrow maxletter w x ⋅ « n i » rows of the subjects row that they could not concentrate any longer. The ensemble classifiers were evaluated using the 4 subsequent To familiarize with the task. respectively: windows/doors were in the angle of vision.: the dotted circles. This experiment has been done with 10 subjects(between 18 and 27 years old) of which 3 were female. one for each classiThey have been instructed to get in a comfortable position so they would not move during the experiment. Johnson and D. none ended the recording prematurely.2. followed by another 100 º whatsoever. ª º = max predicted column wn ⋅ xicolumn they (2) stated what they After the experiment the subjects reported the number of «target appearances » had counted and columns thought would be the maximal difference to the real number. ¬icolumn n=1 ¼ They also have been asked whether they were able to concentrate throughout the experiment.4 Experimental dependently trained using SWLDA. They also sessions for each subject. each in5. J.D. where each segment was used to train an individual classino obvious signs of medical or psychological diseases. 2. i n = 1 ¼ ¬ row 1 5 · 60 · 1000 ms 2 In this 5 minutes there have been about · = 1500 intensifications. ª ms ofN no selection The subjects had to count the intensification the = target for 5 minutes. TheSetup ensemble classifiers used for this study consist of N linear classifiers. This session’s data was divided into N equal. each classifier in the ensemble made a short test run to make sure that they understood what they were expected to do and were not surprised by the independently evaluates the inner product of its respective SWLDA weights and the velocity of this experiment. ¦¦ ¦¦ 5. For a given set of test data. Though. For each subject.554 G. before the onset of the experiment. All of them had non-overlapping segments. The electrode montage used in the current study [8].1.of distraction to a minimum. sum of the N individual classifier scores. the first session of data was used to train the ensemble classifier. Also they fier. feature vector corresponding to each response. all subjects have been shown the interface. Krusienski FPZ FP1 AF7 F7 FT7 T9 T7 AF3 F5 FP2 AFZ F3 F1 FZ AF8 AF4 F2 F8 F6 F4 FT8 FC5 FC3 FC1 FCZ FC2 FC4 FC6 C5 C3 C1 CZ C2 C4 C6 CP5 CP3 CP1 CPZ CP2 CP4 CP6 TP7 P7 P5 P3 PO7 P1 PO3 O1 PZ P2 P4 PO4 POZ P6 T8 T10 TP8 P8 PO8 O2 OZ IZ Fig. From:[63] sis are indicated by5. the raw data was bandpass filtered(2 − 8 Hz) 3.: The “P300 Reader” To work with the records saved by this tool another program “P300 Reader”(see figure 5.6 Preprocessing The following preprocessing steps that have been described in chapter 4 were done in the following order: 1. the baseline has been removed 2.1.3. 30 . blinks were detected and the affected time ranges ignored in further processing steps 4.3) was created that allows to: • plot the raw signal • apply bandpass filtering • detect blinks • average selections(intensifications): – averaging per row/column – averaging per letter 5. trials containing the same letter have been removed if they were too close to each other(< 400 ms) to elicit a P300. epochs were averaged While averaging on a per letter basis.Figure 5. But there is still some hope for event-related synchronization.6).7 Results As a way to measure the ability to concentrate of each test person it was decided to compare the counted target appearances with the real number.1 there were people that came very close to the actual number(especially subject #3 and #6). As can be seen in table 5. although the peak in averaging group 2 has a smaller amplitude.7). unfortunately positive. the actual number of target appearances Unfortunately. Those two groups are then being averaged independently. 5.4 vs.4(which is very bad). In the following the split-epoch averages will be shown for the three test persons with the best counting score. but were not sure if they counted correctly some rare incidents when the target letter flashed a couple of times in a row. say 10. both averages would look almost the same in the area around 300 ms.1. split-epoch averages have been created by splitting the epochs that were being averaged before into two groups: the first group contains every 2n th and the second every 2n + 1 th trial. 5. Many test persons said that they generally did very well.1. Subject #1 reported that he had troubles counting above 100 and he therefore was very uncertain about the resulting count.5) as well as in channel P4 ( 5. Remark 3 (Signal-to-Noise Ratio) The signal-to-noise ratio is an indicator of a recording’s noisiness.1. in general only those epochs that were recognized by the subject should influence the average. This suggests a very low signal-to-noise ratio(see remark 3). Given the amplitude of a P300 wave of 20 µV and an EEG noise of 50 µV the according S/N ratio would be 20 : 50 or 0.9) as well as in channel P4 ( 5. A low S/N ratio(< 1) would make the two averages look completely different due to the randomness of the noise. p.: The counted number vs.1). 31 . the P300 could not be made significantly visible in any of the averages for the test person’s target letter using the preprocessing steps described earlier. in other words: to double the S/N ratio the number of averaged trials needs to be 4 times as high[30. p If n samples are being averaged the S/N ratio increases by a factor of n. # 1 2 3 4 5 6 7 8 9 10 Letter O O T 1 O A U H O L Count ≈ 200 207 224 244 199 261 242 237 205 247 Self-assessment of Error unsure ±10 ±2 ±10 ±6 ±10 ±10 ±5% ±10 unsure Real Count 258 249 223 257 218 257 259 259 242 261 Error 23% 17% 0% 5% 9% 2% 7% 5% 15% 5% Table 5. see table 2. 5. The average groups of subject #4 all have a little. The device’s signal-to-noise ratio does seem to be too low to confidently identify event-related potentials such as the P300. To quantify this. In subject #6’s splitted averages there is a positive peak that could arise from a P300 in both channel P3 ( 5. The averages for subject #3 clearly differ in channel P3 ( 5. peak around 300 ms.5. As those trials have been excluded from the averaging process(see section 5.10 vs.11) which raises hope that there might be more there than just noise. 31].8 vs. If the S/N ratio was reasonably high. 5. since ERPs have a very low amplitude to begin with(only up to 20 µV.6 vs. 7.9.: Subject #4.12.: Subject #6. Channel: P3 Figure 5. Averaging group 2. Averaging group 2. Channel: P4 Figure 5.: Subject #6. Channel: P3 Figure 5. Averaging group 2.: Subject #4. Averaging group 2.14. Averaging group 2.: Subject #6. Channel: P3 Figure 5. Channel: P4 Figure 5. Averaging group 1.: Subject #4. Channel: P4 Figure 5.4.8.6.10. Channel: P4 Figure 5. Channel: P3 Figure 5. Averaging group 1. Averaging group 1.13. Channel: P3 Figure 5. Averaging group 1.15.: Subject #3.Figure 5. Channel: P4 Figure 5.5. Averaging group 1. Averaging group 1.: Subject #3.: Subject #3. Channel: P3 Figure 5.11. Channel: P4 32 .: Subject #4.: Subject #6. Averaging group 2.: Subject #3. 16).17) and logic quizzes(figure 5.0 project[64] is to identify how focused a reader is on the text.2 Introduction In order to simulate mental concentration on a certain task is was decided to create a small intelligence test with breaks in between that serve as both a break for the proband as well as to obtain a negative sample of when the subject was not concentrating on a certain task. 5.3 Experimental Setup Figure 5. whether he is actually thinking about what he is reading and whether he has problems understanding the text.2. Every question appeared only once for a subject.17. They have been given 15 questions in total that were randomly picked from a question pool with equal probability and had 30 seconds to think about an answer. 5. 33 .1 Motivation A future goal of the Text 2.: Relaxation Example In this experiment the subjects were confronted with 2 types of questions: simple math exercises(figure 5.2 Alpha Waves Quiz 5.: Logic Quiz Example Figure 5.: Math Exercise Example Figure 5.5.2.2.16. As a first step.18. this experiment has been chosen to see if the attention/engagement of a person can be detected with the Epoc. The probands were given the same instructions as in the first experiment regarding movements to prevent biological artifacts. Alpha 1 ERD for the single-task condition beginning 3.2. relaxing) have been bandpass-filtered(passband 8 − 10Hz ). After time the for the questionperformance run out. For evaluation channels O1 and O2 were being used. the maps are shown beginning 3 s experiment plus 2 additional volunteers. if the felt the urge to do so. after auditory message onset and ending 0.3. 2. Error bars After answering the question the test person was told to relax.50 s Figure 5.10) of the Epoc as it closer resembles the sites that are shown in figure 5. In total the experiment took 13 − 15 min depending on how long they needed to communicate the answer. theplotted front of thetime. Consequently. upward.05 . head facing From:[42] Ž . Since some people can not just think about nothing. 3!5 represent the time course for the after the end of the auditory message. respectively. to who would thensessions note theŽgiven response.13 s after the start of the communication message and ending 0. they have been advised to move when telling the solution. in Žb. in the single task. 5.3.19.[65].1 Averaged Band Powers In the first approach each epoch(showing quiz. Each dot represents the position of an electrode site with a significant alpha 1 ERD P! 0. according Channels that show a significant alpha 1 ERD under increased 3. relaxing image was shown on the monitor(figure 5. 3.[42] andwas Sergeant al. Alpha 1 are shown in figure 5. Panel shows response tothat enter the important selected frequency answerstask rightduring and toeach makeofsure was actually thinkinginabout solution to testing the question.5 s Figs. a represent Tukey’s HSD values.5 Results For the presentation of the results two different approaches have been chosen that will be explained in the following two sections.4 Headset Placement electrode sites that showed significant alpha 1 ERD.. circlealpha represents the head 0. 5. workload conditions to studies by Fournier et al. communications thethat fourheworkload conditions eachaof the two sessions Žsessions 1 and 6. it was decided to use the regular setting(see figure 4. The resulting means where then plotted with respect to their category(quiz.19. 5.19..1.: Channels showed aEach significant 1 ERD during the at completion of a taskwith shown over after the communication eventthat terminated. relaxing). Because there no etsignifiSignificant ERD was found in the single task as Fig.Žathe . Additionally.125 intervals. Assuming that during the thinking 34 .2. alpha 2 ERD and theta ERS. was done give the subject thetimes illusion it was to get the for the each of the two testing sessions 1 and This 6.5. Single-task ERD has andbeen ERS canttheactivity during 3 s ofinthe This experiment done in the same environment and with same persons that the were first participating the communifirst cation event. a message appeared to communicate their answer to the instructor Fig.2.18). Panel composite Z scores for thetelling three them multi-task workload conditions medium and high. 3. shows Žlow. squared and finally all power values of an epoch have been averaged to obtain one mean power value for each epoch. 23.24.22. Channel: O1 Figure 5.2 ERD/ERS Maps The basics of ERD/ERS maps have already been covered in section 2. Channel: O2 Even though mean alpha 1 power values might be generally a bit lower.: Subject #1.5 6 3 5 5 Quiz 10 15 20 25 30 35 40 5 Relaxing Quiz 10 15 20 25 30 35 40 Relaxing Figure 5. Channel: O2 20 45 18 40 16 35 14 30 12 25 10 20 8 15 6 4 10 5 Quiz 10 15 20 25 30 35 40 45 5 Relaxing Quiz 10 15 20 25 30 35 40 45 40 45 Relaxing Figure 5. the mean alpha 1 power should be significantly higher for the relaxing epochs than the quiz epochs. Channel: O1 Figure 5.: Subject #3.28 in an ERD/ERS map.5 4 8 5 Quiz 10 15 20 25 30 35 Relaxing Figure 5. An alpha event-related desynchronization recorded with a professional EEG will look similar to figure 5.25.7.: Subject #6.2. 5.1.5 9 4 8 7 3. but overall far from being significant as can be seen by the results of the ANOVA test(see table 5.5 16 6 14 5. 35 .2 and 5. The resulting plots are shown below for 4 persons: 5. Alpha 1 inhibition could therefore not be shown with this approach.5.: Subject #6. Channel: O1 40 45 5 Quiz 10 15 20 25 30 35 Relaxing Figure 5. Channel: O2 8.5 22 8 20 7.: Subject #1.5 13 12 5 11 10 4.21.20.5 18 7 6.about the question the proband is more concentrated than during relaxation.: Subject #3.3).5 12 5 10 4. 9 < 2.5 7.09 3.11 15.12 < 4.57 0.0 15.99 < 2.82 1.89 2.94 10.98 21.: ANOVA test results for channel O1 .27. Channel: O1 Subject # 1( 5.94 5.75 1.23 0.5 14 5 12 4.89 Table 5.58 6.09 Var.5 18 6 16 5.18 < 4.50 4.5 22 7 20 6.9 < 2.5 0.26 0.06 4. Relax 3.48 0.21 2.18 < 4.99 Mean Relax 7.33 7.89 < 2.84 9.91 18.58 0.: Subject #9.89 < 2.37 14.18 < 4. Channel: O2 Var. Subject # 1( 5.87 1.91 < 2.23) 4 5 6( 5.94 1.03 7.45 16.22) 4 5 6( 5.18 10% Significance < 2.59 10.96 16.11 3.18 < 4.55 11.12 4.21 < 4.6 0.52 16.82 4.09 1.3.2 6.07 6.35 10.43 24.49 < 1.88 n 28 27 30 30 30 30 9 30 20 30 30 F 3.23 < 4.: Subject #9.34 4.6 5.89 < 2.27) 11 12 Mean Quiz 6.18 < 4.05 11.18 < 4.26) 11 12 Mean Quiz 4.15 Var.74 5.18 10% Significance < 2.18 < 4. Quiz 0.5 3 6 5 Quiz 10 15 20 25 30 5 Relaxing Quiz Figure 5.85 n 28 27 30 30 30 30 9 30 20 30 30 F 0.14 4.91 < 2.36 < 2.24 7.18 < 4.95 35.96 2.20) 2 3( 5.89 < 2.21 < 4.29 3.11 9.38 < 4.96 0.89 < 3.18 < 4.6 15.12 10.26. 36 .24) 7 8 9( 5.84 0.07 9.77 4.02 5% Significance < 4.73 11. For cells with a gray background the null hypothesis could not be disproved.94 0.71 0.22 9. Relax 0.15 7. For cells with a gray background the null hypothesis could not be disproved.40 2.36 < 2.42 5.89 < 3.35 0.68 0.1 0.23 < 4.18 < 4.15 1.31 0.09 11.89 < 2.99 < 2.21) 2 3( 5.18 < 5.6 14.18 < 5.25 5% Significance < 4.36 10.89 < 2.83 10.55 10 15 20 25 30 Relaxing Figure 5.23 6.4 0.34 2.98 12.: ANOVA test results for channel O2 .07 4.87 9.38 < 4.2.03 1.12 < 4.18 7.7 12.98 4.89 Table 5.99 3.7.28 4.25) 7 8 9( 5.01 15.61 Mean Relax 4.06 1.31 4.0 16.89 < 2.57 7.09 15.6 9.89 < 2. Quiz 0.21 0.07 3.0 0.5 10 4 8 3.57 Var.26 4.56 9.87 67.87 16.89 < 2.86 0. In the parietal cortex. the proband was looking at/thinking about the question. Modified. Modified.3.n In the following.: Subject #1. EEG a-frequency band amplitude. by suppressing task-irrelevant itude a oscillations [7]. Amplitude nd expressed relative to the baseline amplitude. RD. some exemplary ERD/ERS maps corresponding to the plots in section 5. with permission.x relaxation cycle.1 will be shown. four or six had to be memorized until the probe stimulus was presented and the subject indicated whether the ‘probe’ item was one of requency band amplitude is enhanced throughout the STM retention period (yellow and red colors correspond to large amplitudes).1. In fact. n arrow) pointing to the to-be-attended hemifield was followed by a ‘stimulus’ in the left or in the right hemifield. plitude dynamics in the human brain..30. large).xxx and the export tool described in section 4.: Subject #1. from Ref.29. Abbreviation: norm. [13].2. the intermediate (but not the small or large) prestimulus a-frequency band ortex facilitate stimulus detection. Channel: O2 Figure 5. Modified. (a) In the sensorimotor cortex. in this study. (d) The subjects’ finger tips were stimulated electrically at the threshold of ructed to react to perceived stimuli. from Ref. The subject was t stimuli in the attended hemifield. However. The whole time before that. [18]. uency band amplitude dependent on the number of memorized items. [23]. as recorded d by suprathreshold electrical stimuli to the median nerve. band amplitudes in the sensorimotor region led to equal psychophysical performance [23]. The time frame in which the solution was being communicated is marked by the red rectangle. [21]. the amplitude of ongoing a-frequency band oscillations. intermediate prestimulus a amplitudes were associated with facilitated detection and the fastest reaction times (Figure 1d). In MEG recordings. from Ref. Each map shows a full question-solutionTRENDS in Neurosciences No. These observations are not in line with the 37 . red. Channel: O2 The ERD/ERS maps unfortunately did not show more significant changes between the relaxing and attentional state. from Ref. is smaller in the occipital cortex al hemifield. averaged over trials. by contrast.28. small. with permission. After the cue. The remaining time after the rectangle marks the epoch where the proband was relaxing.: Alpha ERD(the big red stripe) in an ERD/ERS map[41] Figure 5. They were created using EEGLabVol. large amplitude oscillations are associated with best behavioural ission. (b) Subjects were presented visually with six . Amplitude of single trials is color coded (blue. with permission.5. Figure 5. reflects the ‘spotlight of ologically implemented indirectly relevant areas from inhibition. : Subject #3. Channel: O1 Figure 5.: Subject #3.: Subject #9.34. Channel: O1 38 . Channel: O2 Figure 5.35.31.Figure 5.: Subject #6.: Subject #6. Channel: O2 Figure 5.: Subject #9. Channel: O2 Figure 5.33.36.32. Channel: O2 Figure 5. the logic quiz also had questions with varying task difficulty such as f. since in the regular placement of the headset there are no electrodes above the eyes. During the study of ERD. employing the independent component analysis(ICA) still remains as an additional option.16) which can be either really easy or impossible to do depending on a proband’s tendency. this experiment might have had some flaws in its design: the math questions were too easy for some people and too hard for others depending on their mental arithmetic skills and nervousness. We also tried to quantify the overall quality of the signal. implemented. blink artifacts have not been considered. Biological and technical artifacts apparent in the EEG-signal have been analyzed and techniques to handle them including blink detection. bandpass filtering and averaging have been studied. However. In summary it should be noted that the Epoc is not built to be a very reliable device and it is therefore much more effort needed when trying to reproduce experiments that are usually done with professional EEGs. ICA has been briefly tested using EEGLab and generally seemed to yield better results. The fact that the experiments were not successful therefore only means that additional options need to be evaluated and more time has to be invested to make them work than what was possible in this thesis. but has not been investigated further due to time constraints.e. 39 . Though. validated and applied. even though they are much easier to detect than ERPs. way less time has been dedicated to the quiz experiment compared the P300 speller. ERD detection could also not be significantly shown with the Epoc. The results therefore should be considered as a first step. but does not justify a final verdict about the possible uses of the Epoc in the study of ERPs. the geometrical imaging task(seen in figure 5. the so called signal-to-noise ratio.6 Conclusion To recap: The Epoc has been investigated for event-related potentials and event-related desynchronization or more precisely the P300 component and the alpha ERD. The P300 could not have been made visible using the techniques described before. Additionally. 40 . be asked to sum the numbers from 1 to infinity until the end of the concentration phase.11). This would automatically scale to the abilities of the proband and be equally difficult for everyone. For both experiments the independent component analysis could show a real breakthrough. To make sure that the proband is being kept busy through the whole time. blinks or separate wanted components like the P300 from the rest as has been done in the study of the NeuroPhone[46]. since it allows to remove unwanted components such as f. which is only fulfilled when the Epoc is put on reversely(see figure4. he could f. there have been some flaws in the design of the Alpha Waves Quiz that should be considered in future experiments.e. Also an eye-tracker could be used for the detection of blinks as this would eliminate the need to have electrodes near the eyes.7 Outlook As has been pointed out earlier in section 6.e. Attention. [11] H Hallez. P Turner. Fisch and spehlmann’s eeg primer: basic principles of digital and analog eeg.com/books?id=vIuCV2IKwasC&printsec=frontcover. [7] Johnny Lee and Desney Tan. Archiv für Psychiatrie und Nervenkrankheiten. 2009. Jan 1995. http://www. Jan 1999. and J Cameron. [13] Bruce J. and R Grech. [15] Bernard S. T Vaughan. Review on solving the forward problem in eeg source analysis.org. and R Matthews.jneuroengrehab.com/en-us/um/redmond/groups/cue/publications/UIST2006-LowCostEEG. Jan 2002.il/articles/faces-e. UIST ’06: Proceedings of the 19th annual ACM symposium on User interface software and technology. http://books. Using a low-cost electroencephalograph for task classification in hci research.7385. Jan 2008.pdf. pages 336–345. Atlas of ambulatory eeg.sciencedirect. 41 . Informa Pharma Science.pdf. Wie entsteht das eeg? Das Neurophysiologie-Labor. Neurophysiologic basis of eeg. http://research.google.Bibliography [1] Über das elektrenkephalogramm des menschen. http://books. page 453. W Sarnacki.google. http://books.5577.com/content/pdf/1743-0003-4-46. http://books. http://www.ucdavis.sleep. [8] Jamie Ward.asp?route=3402.mada.pdf. Steven C. Fisch and Rainer Spehlmann.com/forum/messages/forum12/topic129/message500/#message500. http://mind.fi/readbinary. Hebrew University of Jerusalem. and Donald L. Chang. page 436. 2010. evolution. Journal of NeuroEngineering and Rehabilitation.pdf.google. [2] J Malmivuo. Journal of clinical neurophysiology. Jan 1998. B Vanrumste. [18] S Luck.com/books?id=MOcDuNt9kqwC&printsec=frontcover. Issue 1:527–570. Klinische elektroenzephalographie . page 109. Oct 2006. Novel Interaction Methods and Techniques.com/books?id=YMHsluy4QygC&printsec=frontcover. page 760. December 1929.com/books?id=swMsQgAACAAJ&printsec=frontcover. Bioelectromagnetism: principles and applications of bioelectric and biomagnetic fields. [3] K Stamps and Y Hamam. Jan 2003. [4] Elana Zion-Golumbic.7379. http://www. Brain Informatics. The cognitive electrophysiology of mind and brain. pages 623–631.google. T McManus.microsoft. 2006. page 289.edu/people/mlgamble/Luck/papers/Luck%201998%20Pashler%20Chapter. Towards inexpensive bci control for wheelchair navigation in the enabled environment–a hardware survey. Abitur training biologie 2 leistungskurs: Verhaltensbiologie.5611. A novel dry electrode for brain-computer interface. R Plonsey. 2010. Jan 2007. Jan 2005. [12] Saeid Sanei and Jonathon Chambers.bem. Eeg signal processing. http://brain.page iii. [14] Stephan Zschocke. Human-Computer Interaction. 23(3):186. [5] Emotiv forum thread about the epoc’s new dry sensors.fi/book/.google. Schomer. [10] T Kirschstein. page 278. http://emotiv.com/ science?_ob=ArticleURL&_udi=B8H16-4T5T4BB-4&_user=2415276&_coverDate=08%2F12%2F2008&_rdoc=1&_ fmt=high&_orig=search&_origin=search&_sort=d&_docanchor=&view=c&_acct=C000057219&_version=1&_ urlVersion=0&_userid=2415276&md5=c09612783887c2d9a9e9a807d1832755&searchtype=a. [6] E Sellers. The student’s guide to cognitive neuroscience. Jan 2007. 7. [9] Moisl Meinhard.com/books?id=s34_04ukPKkC&printsec=frontcover. Schachter. Jan 2010. http://www. What is eeg? The Department of Psychology and the Department of Cognitive Science. page 621. [17] P Olejniczak. [16] Alberto Zani and Alice Mado Proverbio. Neurophysiology of selective attention. http://books. processing. 2006.google&_acct=C000050221&_version=1&_ urlVersion=0&_userid=10&md5=bdbd9692722b66919b7c0b0e3d308a16&searchtype=a.[19] SJ Luck. and G Pfurtscheller. Der einfluß okularer artefakte auf das electroencephalogramm. 2005.com/article/1140247-overview.biomedsearch. 42 . http://emedicine.org/wiki/Basics.ist. Documenta ophthalmologica. . http://en.wisc. [28] Ivo van der Bilt. and M Bach._Der_ Einfluss_okularer_Artefakte_auf_das_Electroencephalogramm.4647&rep=rep1&type=pdf. M Brigell. M G Coles. [21] JJM Kierkels.sciencedirect. JA Kors. [31] A Schlögl.95. classification and applications. 1986.de/fileadmin/fb1/ein/PLA/Christ__S. http://www.springerlink.scientificcommons. [23] P Berg and M Scherg. and E Donchin.__2000_. Ecgpedia. 49(10):1109–1127. [29] A Schlögl and G Pfurtscheller. PhD thesis. [30] Todd C. http://citeseerx. Presence research and eeg. 2002. 17(3):637–650. the american college of cardiology foundation. http://en. Proceedings of the 5th International Workshop on Presence. Electroencephalogr Clin Neurophysiol. Event-related potentials: a methods handbook. Filtering and aliasing of muscle activity in eeg frequency analysis. Jan 2005. March 2010. A new method for off-line removal of ocular artifact. Online. Journal of the American College of Cardiology. council on clinical cardiology. http://www.com/attachments/00/16/79/96/16799694/bpo_v8_p11_m115.uni-trier. and E Donchin.pdf. Electroencephalography and Clinical Neurophysiology. JJ Bailey. en. http://www.ecgpedia. 1991. .com/science?_ob=MImg&_imagekey=B6T18-4N6Y471-2-1&_cdi=4884&_user= 2415276&_pii=S073510970700232X&_orig=search&_coverDate=03%252F13%252F2007&_sk=999509989&view= c&wchp=dGLzVlb-zSkWA&md5=245ac8366fda88e400a22fc65bb3307b&ie=/sdarticle. Localization of the event-related potential novelty response as defined by principal components analysis. report siesta-task310 artifact detection and denoising eog and ecg minimization eog and ecg minimization based on regres sion analysis. 2003.pdf. pages 154–160. http://www.psych. 2007.pdf. Coles & Donchin (1983). 8(1):11–35. http://www. [33] J Dien. Electroencephalography and clinical neurophysiology. [24] M Brown.tue. [22] S Christ.pdf. Iscev standard for clinical electro-oculography (eog) 2006. [27] P Kligfield. and F Mohd-Yasin. http://books. March 2010.pdf. 2007. P Macfarlane. 2006. unveröffentliche Diplomarbeit an der Universität Trier.edu/lit/Articles/DienJ2003a. page 404.pdf.1. [20] Diego Rielo Selim R Benbadis.psu. 113(3):205–212. Techniques of emg signal analysis: detection. http://www. Eine Evaluation des Augenkorrekturalgorithmus von Gratton. 55(4):468–484. Dipole models of eye movements and blinks.com/content/8q34wm77k63x1736/fulltext. Validating and improving the correction of ocular artifacts in electro-encephalography.1. http://dionysus. 64(1):77–80. KM Spencer. MS Hussain. Eeg artifacts.nl/extra2/200711857. Cognitive Brain Research. and the heart rhythm society endorsed by the international society for computerized electrocardiology.org. M Marmor. E Zrenner. LS Gettes.google. Apr 1983. http://alexandria. R Childers. D Bartussek. Biol.sciencedirect.scientificcommons. [32] BS Oken. and DM Mirvis.org/42786168.pdf.com/books?id=OQyZEfgEzRUC&printsec=frontcover.com/science?_ob=MImg&_imagekey=B6SYX-482YYTK-10N-1&_cdi=4846&_user= 2415276&_pii=001346949190154V&_orig=search&_coverDate=07%2F31%2F1991&_sk=999209998&view=c&wchp= dGLzVlb-zSkWA&md5=de5b532df94b6d540bde4c710b301c2a&ie=/sdarticle. 79(1):36–44.edu/viewdoc/download?doi=10. Technische Universiteit Eindhoven.com/science?_ob=ArticleURL&_udi=B6SYX-48303FF-2KB&_user=10&_ coverDate=07%2F31%2F1986&_rdoc=1&_fmt=high&_orig=search&_origin=search&_sort=d&_docanchor= &view=c&_searchStrId=1473580901&_rerunOrigin=scholar. Proced. EW Hancock. G Van Herpen.medscape. and E Naumann. [26] MBI Reaz. Jan 2007. 2000.sciencedirect. Handy. Recommendations for the standardization and interpretation of the electrocardiogram:: Part i: The electrocardiogram and its technology a scientific statement from the american heart association electrocardiography and arrhythmias committee. An introduction to the event-related potential technique. BJ Deal. [25] G Gratton. . M Slater. pdf.wisc. Two varieties of long-latency positive waves evoked by unpredictable auditory stimuli in man. 30(4):150–158. Electroencephalography and Clinical Neurophysiology.sciencedirect. [38] G Pfurtscheller. 38(4):387–401. [46] AT Campbell.jneurosci. . [40] B Graimann and G Pfurtscheller. . Quantification and visualization of event-related changes in oscillatory brain activity in the time-frequency domain. KC Squires. http://ml. Event-related cortical desynchronization detected by power measurements of scalp eeg désynchronization corticale liée à un évènement.pdf.pdf. ACM New York.sfu. USA. K Marinkovic. Systems. Event-related cortical . 43 .org/cgi/reprint/9/12/4253.ibms. 2007. http://emotiv. and DD Spencer.cingulate.ca/~jmcd/courses/925/papers/Squires(1975). GF Wilson. 106(2):156–164.sciencedirect. and P Chauvel.tw/ftpshare/Protocol/Internet%20Resources/E%20books/event% 20related%20dynamics%20of%20brain%20oscillation/8. Clinical Neurophysiology. T Choudhury. [39] G Pfurtscheller and FH Lopes da Silva.edu. http://www.pdf. 159:79–97. [41] S Palva and JM Palva. MK Mukerjee. Trends in neurosciences. and subjective indexes of workload when performing multiple tasks: manipulations of task difficulty and training.com/forum/forum10/topic697/.com/books?hl=en&lr=&id=HV2qE7BPEkgC&oi=fnd&pg=PR5&dq=International+ Federation+of+Societies+for+Electroencephalography+and+Clinical+Neurophysiology&ots= cyvLv90RBh&sig=pXJNDjDn3HFE6bnH7Gb9W1Nhgqg. Progress in Brain Research. [45] Emotiv’s homepage. . Jan 1977. 2010. Event-related synchronization (ers): an electrophysiological correlate of cortical areas at rest. and CR Swain. Event-related eeg/meg synchronization and desynchronization: basic principles. International Journal of Psychophysiology. [47] Emotiv forum thread about the epoc’s sensors. Apr 1991. H Lu.sinica. Task-dependent field potentials in human hippocampal formation. détectée par une mésure . 110(11):1842–1857. 8(2):200–2. CC Wood. 2010. . 31(2):129–145. 1998.com/science?_ob=MImg&_imagekey=B6T3M-3VGTJ2K-4-T&_ cdi=4950&_user=2415276&_pii=S016787609800049X&_origin=search&_coverDate=01%2F01%2F1999&_sk= 999689997&view=c&wchp=dGLbVtb-zSkzS&md5=0761f37740f628ece9644f2b9b251232&ie=/sdarticle.com/science?_ob=ArticleURL&_udi=B6SYX-482XHC3-155&_user= 2415276&_coverDate=06%2F30%2F1977&_rdoc=1&_fmt=high&_orig=search&_origin=search&_sort=d&_ docanchor=&view=c&_searchStrId=1475042127&_rerunOrigin=scholar.google&_acct=C000057219&_ version=1&_urlVersion=0&_userid=2415276&md5=ac507e3001d273444650167ab503dc3b&searchtype=a. Journal of Neuroscience. and RDS Raizada.pdf. [43] G Deuschl and A Eisen. PD Williamson. Neurophone: Brain-mobile phone interface using a wireless eeg headset. http://www. http://books.com/science?_ob=ArticleURL&_udi=B6SYX-482XNYX-2YW&_user=2415276&_ coverDate=07%2F31%2F1992&_rdoc=1&_fmt=high&_orig=search&_origin=search&_sort=d&_docanchor= &view=c&_searchStrId=1475067063&_rerunOrigin=scholar. http://www. and Applications on Mobile Handhelds (MobiHeld’10). 9(12):4253. 2006. 83(1):62–69.google&_acct=C000057219&_version=1&_ urlVersion=0&_userid=2415276&md5=5fc8dd517a9eae347e2bc1cbc519fc8d&searchtype=a. [37] G Pfurtscheller and A Aranibar. Recommendations for the practice of clinical neurophysiology: Guidelines of the international federation of clinical neurophysiology. and SA Hillyard. . [36] NK Squires. Electrophysiological.de/bbci/papers/PfuSil99. 1992.com. http://www. Electroencephalography and clinical Neurophysiology. http://www. J Clin Neurophysiol. Jan 1999. http://www. 1999.edu/postlab/readings/alpha_2007_tins. M Rabbi. Proceedings of The Second ACM SIGCOMM Workshop on Networking.tu-berlin. http://www.[34] E Halgren.google.com/science?_ob=MImg&_imagekey=B6SYX-3V4C19W-115-5&_cdi=4846&_user= 2415276&_pii=S0013469497001193&_origin=search&_coverDate=02%2F28%2F1998&_sk=998939997&view= c&wchp=dGLbVzz-zSkWb&md5=55c820d0c470c8bfb5d0fa3712682d15&ie=/sdarticle. 1989. [44] American electroencephalographic society guidelines for standard electrode position nomenclature.emotiv. [42] LR Fournier. Generators of the late cognitive potentials in auditory and visual oddball tasks. [35] G McCarthy. NY.cs. http://www. S Hu. 1999. books.google.sciencedirect. behavioral. .pdf. New vistas for [alpha]-frequency band oscillations. Electroencephalography and Clinical Neurophysiology. 2010.com/. http://psych.sciencedirect. 1975.pdf. S Bayoudh. TI Laakso. 1998.pdf. http://www.1. pages 515–524. http://www.text20. http://emotiv. and E Donchin.uma.text20framework.pdf.fi/spit/publications/1998j3. [50] F Andrade.dpmi. http://www. Eeglab tutorial.edu/viewdoc/download?doi=10. and M Pennisi. EW Sellers. 2010.com/docs/books/jni/html/intro. Jan 2007. EW Sellers. I Hartimo.tugraz.1. B Dal Seno. Journal of Neuroscience Methods. Talking off the top of your head: toward a mental prosthesis utilizing event-related brain potentials. Novel Interaction Methods and Techniques. A Davis.1111/j. 2008. Biomed.net/dl/final_uploads/200810L005. [56] A Delorme and S Makeig. DJ McFarland. and M Matteucci. A comparison of three methods with implicit features for automatic identification of p300s in a bci.google. S Schwarz. [63] G Johnson and D Krusienski. [49] Emotiv forum thread about the epoc’s internal preprocessing. Jan 1987. 2010. X Gao. Brain–computer interface research at the university of south florida cognitive psychophysiology laboratory: the p300 speller.1469-8986. DJ McFarland. 134(1):9–21. We know it before you do: Finding recognition patterns in brain activity. Journal of Neural Engineering. http://onlinelibrary. http://citeseerx. http://www. The text 2. Human-Computer Interaction. [55] J Kauraniemi.com/forum/messages/forum10/topic715/message4330/#message4330. P Davies. page 400.pdf. http://java. Eng. 2010. 2004. 51(4):647–656.pdf. and C Hansen. [61] EW Sellers.1987. 2009. 2008. A Kübler.pt/courses/ubicomp10S/projects/10_brain_eye.edu/busey/temp/eeglab4. G Buscher. and JR Wolpaw.pdf. [60] DJ Krusienski.pdf. 70(6):510–523. and JR Wolpaw. Swartz Center for Computational Neuroscience. Journal of neuroscience methods. Eeglab: an open source toolbox for analysis of single-trial eeg dynamics including independent component analysis. S Gao. 45(1):41–52. IADIS Multi Conference on Computer Science and Information Systems Amsterdam.etsu. Toward enhanced p300 speller performance. Delta operator realizations of direct-form iir filters.pdf. http://cognitrn. 2010. and W Winsum.pdf. http://emotiv.psych. X Miao.at/~schloegl/akmi/NengXu2004.indiana. pages 551–557. and SJ Ovaska. [64] R Biedert. F Cabestaing. [62] DJ Krusienski.gsu. 2006. [66] The java native interface programmer’s guide and specification.0 framework. TM Vaughan.1058&rep=rep1&type=pdf.net/documentation/paper. [54] Jürgen Reichardt and Bernd Schwarz.html#875. IEEE Transactions on Biomedical Engineering. R Geuze.[48] Emotiv’s wikipedia. Ensemble swlda classifiers for the p300 speller. 2004. http://emotiv. 1988. 2009. 2004. and F Yang. Event-related desynchronization and p300.hut. Netherlands.311. Psychophysiology.com/doi/10.wiley.com/books?id=v93aGAAACAAJ&printsec=frontcover. [65] J Sergeant.311/eeglabtutorial4. 167(1):15–21.wikia. B Hong. Vhdl-synthese: Entwurf digitaler schaltungen und systeme. [58] LA Farwell and E Donchin. H Serby.com/forum/messages/forum12/topic527/message2946/?phrase_id=108274#message2946. http://hci.sun. [51] S Hansen. http://books. [53] N Xu. Workshop on Eye Gaze in Intelligent Human Machine Interaction.edu/cas/bcilab/documents/Sellers_etalIEEE-2006. 51(6):1067. and A Dengel. [57] S Makeig A Delorme. T Lottermann.58. http://signal. Bci competition 2003—data set iib: enhancing p300 wave detection using ica-based subspace projections for bci applications. 44 . 3:299. IEEE Transactions on Circuits and Systems II Analog and Digital Signal Processing.iadis. 2004. TM Vaughan. 2010.tb00294. Artificial Neural Networks–ICANN 2009. M Möller.ist.cis. A comparison of classification techniques for the p300 speller. http://data. [52] Emotiv forum thread about the sample’s timestamps. Addressing accessibiltiy: Emerging user interfaces for second life communities.psu. [59] L Sportiello.x/pdf. Electroencephalography and clinical Neurophysiology.com/wiki/Emotiv_EPOC.edu/brainlab/papers/Farwell%20Donchin%2088%20-%20first%20P300%20BCI. Proceedings of the 4th conference on USENIX Tcl/Tk . [68] D Beazley. . 45 . Jan 1996.org/journal/jcsit/0202csit11. http://airccse.pdf. . Swig: An easy to use tool for integrating scripting languages with c and c++. International Journal of Computer Science.[67] VS Vairale and KN Honwadkar. . Wrapper generator using java native interface. 2. In contrast to JNA.dev. b o o l e n a b l e ) . Listing A.1) expects a pointer to an unsigned int as its second parameter which beforehand needed to be allocated. Additionally it makes it easy to append custom helper functions to the wrapper. i n t EE_EmoEngineEventGetUserId ( EmoEngineEventHandle hEvent . i n t E E _ D a t a A c q u i s i t i o n E n a b l e ( unsigned i n t u s e r I d .A Creating the Java Wrapper A. unsigned i n t * nSampleOut ) . For the Epoc’s wrapper dll. Amongst other things it simplifies the process of creating a Java interface for a given C header file. 1 https://jna. A.2 Problems We first tried to accomplish this task using Java Native Access(JNA1 ). • function EE_EmoEngineEventGetUserId(listing A.net/ 46 . This resolves the first issue and the latter has been eliminated as well by calling dataAcquisitionEnable instead of EE_DataAcquisitionEnable directly. A. SWIG has been used.1 JNI The Java Native Interface(JNI) allows to execute code written in other programming languages such as C from within Java[66][67]. To automatically generate the wrapper. SWIG takes as an input an interface definition file. which must at least include the dll’s header.2): The method createPUInt now allows to create a pointer to an unsigned int whose value can also be retrieved using pUIntToUInt and deleted with freePUInt. the Java Native Interface(JNI[66]) has been investigated.2 SWIG SWIG(Simplified Wrapper and Interface Generator[68]) is an interface compiler for C/C++ programs. additional user-defined functions have been supplied(some can be seen in listing A. JNI requires the compilation of a custom wrapper dll and therefore allows to integrate custom C functions into the library which made it possible to overcome the problems mentioned above.1. A. but there were 2 issues that did not allow it to be used with Emotiv’s dll: • function EE_DataGetNumberOfSample(listing A. For a library to be used by JNI. Later on. the value this pointer was pointing at had to be read out.3 Solution After abandoning JNA. an interface specification needs to be created and it needs to be compiled with the JNI library linked against it.1). unsigned i n t * pUserIdOut ) . It seems to be impossible to do this with JNA at the moment.1: some function signatures of the Emotiv SDK i n t EE_DataGetNumberOfSample ( DataHandle hData .1) returns a pointer to a user id which later on has to be passed by value to function EE_DataAcquisitionEnable(listing A.java.1.1 Tools A. sourceforge.Listing A. c^ −FeedkWrapper ."^ −MT^ −LD edkWrapper_wrap . } void f r e e P U I n t ( unsigned i n t * u I n t ) { free ( uInt ). } The Java classes that are needed to use the wrapper and the library can be created by executing the commands in listing A.3.google.com/p/text20/wiki/WrapperEmotiv http://code. * pUInt = u I n t . This file needs to be created manually.google. c l − I " C : \ Programme\ Java \ jdk1 . return pUInt . A more detailed explanation on how to create the wrapper has been made available to the Epoc community on the text 2. / l i b "^ /DEFAULTLIB : edk^ /MAP^ /DEF : edkWrapper . 6 .2: Additional helper functions unsigned i n t pUIntToUInt ( unsigned i n t * pUInt ) { return * pUInt .3: Commands to Create the Wrapper Library > > > > > > > > > > > > > > > > > swig . The definition file now maps each method name to the equivalent name with the appendix added to it and in this way lets JNI find the function in the dll. 0 \VC\ b i n \ v c v a r s 3 2 . 0 _20 \ i n c l u d e \ win32"^ − I ". b a t S e t t i n g environment f o r u s i n g M i c r o s o f t V i s u a l S t u d i o 2008 x86 t o o l s . that was used to create the wrapper dll. exe − j a v a edkWrapper . i C : \ Programme\ M i c r o s o f t V i s u a l S t u d i o 9 . d e f The definition file edkWrapper.0 project page2 . Listing A. b o o l e n a b l e ) { return E E _ D a t a A c q u i s i t i o n E n a b l e ( * u s e r I d . d l l^ / l i n k^ /OPT : NOREF^ /LIBPATH : " . This wrapper has been further extended to make the Epoc device accessible over the network and by this means also accessible from other operating systems using the Java Simple Plugin Framework3 and LipeRMI4 .net/ 47 . 0 _20 \ i n c l u d e"^ − I " C : \ Programme\ Java \ jdk1 . 6 . } i n t d a t a A c q u i s i t i o n E n a b l e ( unsigned i n t * u s e r I d .com/p/jspf/ http://lipermi. 2 3 4 http://code. appends the byte length of all function parameters to the function name – JNI however searches for the function without the appendix and thus fails to find it. } unsigned i n t * c r e a t e P U I n t ( unsigned i n t u I n t ) { unsigned i n t * pUInt = mall oc ( s i z e o f ( unsigned i n t ) ) .def is hereby worth mentioning as its need arises from the fact that Visual Studio’s linker. e n a b l e ) . but fortunately only once.