Chapter 2:Multimedia information representation 2.1 Introduction 2.2 Digitization principles 2.3 Text 2.4 Images 2.5 Audio 2.6 Video Introduction Analog data, such as the sounds made by a human voice, take on continuous values. When someone speaks, an analog wave is created in the air. This can be captured by a microphone and converted to an analog signal or sampled and converted to a digital signal. Periodic analog signals Periodic analog signals can be classified as simple or composite. A simple periodic analog signal, a sine wave, cannot be decomposed into simpler signals. A composite periodic analog signal is composed of multiple sine waves. Parameters A sine wave can be represented by three parameters: the peak amplitude, the frequency, and the phase. Amplitude The peak amplitude of a signal is the absolute value of its highest intensity, proportionalto the energy it carries. For electric signals, peak amplitude is normally measured in volts. Amplitude Frequency Period refers to the amount of time, in seconds, a signal needs to complete 1 cycle. Frequency refers to the number of periods in I s. Note that period and frequency are just one characteristic defined in two ways. Period is the inverse of frequency, and frequency is the inverse of period, Frequency Phase The term phase describes the position of the waveform relative to time 0. If we think of the wave as something that can be shifted backward or forward along the time axis,phase describes the amount of that shift. Phase The conversion of an analog signal into a digital form . is carried out by a an electric circuit called Signal encoder Sampling the amplitude of analog signal at repetitive time intervals and then converting amplitude of each sample into corresponding digital value The conversion of stored digitized samples into their corresponding time- varying analog form is carried out by an electric circuit called Signal decoder . Process Process 1. The analog signal is sampled. 2. The sampled signal is quantized. 3. The quantized values are encoded as streams of bits Multimedia Information Representation • Multimedia Information is stored and processed within a computer in a digital form • Codeword: Combination of a fixed number of bits that represents each character, in the case of textual information • analogue signal: Signal whose amplitude (magnitude of the sound/image intensity) varies continuously with time • Signal encoder: Electrical circuit used for the conversion of an analog signal into a digital form • Signal decoder: Electrical circuit that converts stored digitized samples into time-varying analog form Analog Signals •As mentioned earlier the amplitude of the signal varies continuously with time • The Fourier analysis can be used to show that any time varying signal is made up of infinite number of single-frequency sinusoidal components Analog Signals • The range of frequencies of the sinusoidal components that make up the signal is called the signal bandwidth • Speech bandwidth: 50Hz – 10kHz • Music Bandwidth: 15Hz – 20kHz Analog Signals –Signal Properties Analog Signals –Signal Properties Continued… • To transmit an analog signal through a network the bandwidth of the transmission channel should be equal to or greater than the signal bandwidth • If the bandwidth of the channel is less than the signal bandwidth than channel is called the bandlimiting channel Encoder Design Encoder Design Encoder Design The Encoder consists of bandlimiting filter and an analogue-to-digital converter (ADC) ( comprising sample and hold + quantizer) • Bandlimiting filter: Removes the selected higher frequency components from the source signal • Sample and hold Circuit: Samples amplitude of the filtered signal at regular intervals and holds the sampled amplitudes between samples • Quantizer: Converts the samples into their corresponding binary form Sampling Rate • Nyquist sampling theorem: To obtain an accurate representation of a time-varying analog signal, its amplitude must be sampled at a minimum that is equal to or greater than twice the highest sinusoidal frequency component that is present in the signal • Nyquist rate is represented in samples per seconds (sps) • Antialiasing filter: Another name for bandlimiting filter. Since it passes frequencies that are within the Nyquist rate Alias signal generation due to undersampling • In reality the transmission channel used often has a lower bandwidth • To avoid distortion the source signal is first passed through the BLF which is designed to pass only the frequency components that are within the channel bandwidth • This avoids alias signals caused by undersampling Quantization Intervals • Representation of the analogue samples require an infinite number of digits Quantization Intervals • Three bits are used to represent each sample ( 1 bit for the sign and two bits to represent the magnitude) • If Vmax is the maximum positive and negative signal amplitude and n is the number of binary bits used then the quantization interval, q, is defined as q = 2Vmax/ 2n • A signal anywhere within the quantization interval will be represented by the same binary codeword • Each codeword is at the centre of the corresponding quantization interval Quantization noise polarity • Quantization error is the difference between the actual signal amplitude and the corresponding nominal amplitude (also known as quantization noise since values vary randomly) Dynamic Range • With high-fidelity music it is important to be able to hear very quiet passages without any distortion created by quantization noise • Dynamic range is defined as the ratio of the maximum signal amplitude to the minimum. D = 20 log10 (Vmax/Vmin) dB Decoder Design • A signal decoder is an electronic circuit that performs the conversion prior to their output back again into their analog form through a digital-to-analogue converter and a low pass filter Encoder+decode= Codec Low-pass filter: Only passes those frequency components that were filtered through the bandlimiting filter in the encoder 2.3 Text Three types of text Unformatted text Formatted text hypertext Text • Unformatted text: Known as plain text; enables pages to be created which comprise strings of fixed-sized characters from a limited character set • Formatted Text: Known as richtext; enables pages to be created which comprise of strings of characters of different styles, sizes and shape with tables, graphics, and images inserted at appropriate points • Hypertext: Enables an integrated set of documents (Each comprising formatted text) to be created which have defined linkages between them Unformatted Text – The basic ASCII character set • Control characters (Back space, escape, delete, form feed etc) • Printable characters (alphabetic, numeric, and punctuation) •The American Standard Code for Information Interchange is one of the most widely used character sets and the table includes the binary codewords used to represent each character (7 bit Unformatted Text – Supplementary set of Mosaic characters The characters in columns 010/011 and 110/111 are replaced with the set of mosaic characters; and then used, together with the various uppercase characters illustrated, to create relatively simple graphical images Unformatted Text – Examples of Videotext/Teletext • Although in practice the total page is made up of a matrix of symbols and characters which all have the same size, some simple graphical symbols and text of larger sizes can be constructed by the use of groups of the basic symbols Formatted Text • It is produced by most word processing packages and used extensively in the publishing sector for the preparation of papers, books, magazines, journals and so on.. • Documents of mixed type (characters, different styles, fonts, shape etc) possible. •Format control characters are used Hypertext – Electronic Document in hypertext •Hypertext can be used to create an electronic version of documents with the index, descriptions of departments, courses on offer, library, and other facilities all written in hypertext as pages with various defined hyperlinks. Hypertext – Electronic Document in hypertext •An example of a hypertext language is HTML used to describe how the contents of a document are presented on a printer or a display; other mark-up languages are: Postscript, SGML (Standard Generalized Mark-up language, Tex, Latex. 2.4 Images Image are displayed in the form of a two-dimensional matrix of individual picture elements─known as pixels or pels Graphics Shape can be drawn using pencil or paint brush or mouse. Can change shape ,color , size can create gallery known as clip art. Also add shadows in case of three dimensional. VGA-video graphics array. Images has attributes. Color fill and rendering. 2.4.1 Graphics Two forms of representation of a computer graphic: a high-level version (similar to the source code of a high-level program) and the actual pixel-image of the graphic (similar to the byte-string corresponding to the low-level machine code─bitmap format) Standardized forms of representation such as GIF (graphical interchange format) and TIFF (tagged image file format) 2.4.2 Digitized documents A single binary digit to represent each pel, a 0 for a white pel and a 1 for a black pel. 2.4.3 Digitized pictures Color principles A whole spectrum of colors─known as a color gamut ─can be produced by using different proportions of red(R), green(G), and blue (B) Fig 2.12 Additive color mixing producing a color image on a black surface Subtractive color mixing for producing a color image on a white surface Fig 2.13 2.4.3 Digitized pictures Raster-scan principles Finely focused electron beam used to scan the raster over complete screen Discrete horizontal lines from top left corner to botom right corner. Progressive scanning Each complete set of horizontal scan is called a frame. Picture tube coated with light sensitive phosphor emits light when energized by electron beam. Brightness depends on power in electron beam. 2.4.3 Digitized pictures Refresh rate and flicker.-persistence very low. The number of bits per pixel is known as the pixel depth and determines the range of different colors Aspect ratio Both the number of pixels per scanned line and the number of lines per frame The ratio of the screen width to the screen height National Television Standards Committee (NTSC), PAL(UK), CCIR(Germany), SECAM (France) 2.4.3 Digitized pictures 2.4.3 Digitized pictures Digital cameras and scanners An image is captured within the camera/scanner using an image sensor Silicon chip consists of a two-dimensional grid of lightsensitive cells called photosites Stores the level of intensity of light falls on it. A widely-used image sensor is a charge-coupled device (CCD) Fig 2.16 2.5 Audio Typical Audio Types Speech signal for interpersonal application such as (video) telephony Music-quality audio such as CD synthesizer is an electronic instrument that utilizes multiple sound generators to create complex waveforms microphone Basics on Audio Signals 1. Human speech: 50Hz -10KHz (4Khz in a plain-old-telephone system) - 2 x 10K - ideally, 12 bits/sample 2. Human audible music: 15Hz - 20KHz - 2 x 20K - ideally, 16 bits/sample 53 2.5 Audio The bandwidth of a typical speech signal is from 50Hz through to 10kHz; music signal from 15 Hz through to 20kHz The sampling rate: 20ksps (2*10kHz) for speech and 40ksps (2*20kHz) for music Music stereophonic (stereo) results in a bit rate double that of a monaural(mono) signal Pulse code modulation Speech Previously analog to analog in original form. Introduced digital devices. Digitization procedure is known as PCM Speech signal-200hz -3.4 khz So 8khz sampling rate for easy transfer And if we have 8 bits per sample or 7 bits per sample We use 64kbps or 56 kbps bit rates. Human Voice over PSTN PCM Speech(1) Companding (compressing/expanding) Pure PCM signals Compander (compressor/expander) Enhanced PCM signals Non-linear (unequal) interval quantization & narrower intervals for smaller amplitude signals Equal (linear) interval quantization & same level of quantization error Irrespective of the magnitude of the input signal , the same error level for both low (quiet) signals and high (loud) signals is produced Why companding ? Because the human ears are more sensitive to noise on quiet signals than it is on loud signals. Hence the effect of quantization noise (error) can be reduced with companding 56 CD-Quality Audio Human audible bandwidth: 15Hz-20Khz 40Ksps In CD-ROMs, more higher, say, 44.1Ksps & 16-bit/sample used bit rate for channel = sampling rate x bits per sample = 44.1 x 103 x 16 = 705.6 Kbps total rate required for stereophonic music = 2 x 705.6 = 1.411 Mbps 57 Synthesized audio Used in multimedia applications which required to store digitized waveform of lesser magnitude. Midi-music interface digital interface. Three components-computer, keyboard and set of sound generators. Takes input from keyboard and generate digital from analog . 2.6 Video 2.6.1 Broadcast television Scanning sequence It is necessary to use a minimum refresh rate of 50 times per second to avoid flicker A refresh rate of 25 times per second is sufficient Field: the first comprising only the odd scan lines and the second the even scan lines . Each frame is refreshed at 30/25 frames per second. In PAL 625 lines and 50 refresh rate In NTSC 525 lines and 60 refresh rate. 2.6.1 Broadcast television The two field are then integrated together in the television receiver using a technique known as interlaced scanning Fig 2.19 The three main properties of a color source Brightness: represents the amount of energy varies on grayscale from black to white Hue: this represents the actual color of the source Saturation: this represents the strength or vividness of the color. 2.6.1 Broadcast television The term luminance is used to refer to the brightness of a source .amount of white light it contains The hue and saturation are referred to as its chrominance Ys 0.299 Rs 0.587 Gs 0.144 Bs Where Ys is the amplitude of the luminance signal and Rs,Gs and Bs are the magnitudes of the three color component signals 2.6.1 Broadcast television The blue chrominance (Cb), and the red chrominance (Cr) are then used to represent hue and saturation The two color difference signals: Cb Bs Ys Cr Rs Ys 2.6.1 Broadcast television In the PAL system, Cb and Cr are referred to as U and V respectively PAL : Y 0.299R 0.587G 0.114B U 0.493( B Y ) V 0.877( R Y ) The NTSC system form two different signals referred to as I and Q NTSC : Y 0.299R 0.587G 0.114B I 0.74( R Y ) 0.27( B Y ) Q 0.48( R Y ) 0.41( B Y ) 2.6.2 Digital video Eye have shown that the resolution of the eye is less sensitive for color than it is for luminance 4：2：2 format The original digitization format used in Recommendation CCIR-601 Digital Video For luminance we have 6MHz signal bandwidth and for two chrominance 3MHz. So sampling rate of 12 ksps and 6ksps is required. A line sampling rate of 13.5MHz for luminance and 6.75MHz for the two chrominance signals For 525 lines total sweep time is 63.56 micro seconds and for 11.56 microseconds beam is turned off . so 52 seconds in actual. Therefore scanning rate =702 samples per line. The number of samples per line is increased to 720 2.6.2 Digital video The corresponding number of samples for each of the two chrominance signals is 360 samples per active line This results in 4Y samples for every 2Cb, and 2Cr samples The numbers 480 and 576 being the number of active (visible) lines in the respective system Each line sampled at constant rate with fixed number of samples per line called as orthogonal and orthogonal sampling. Figure 2.21 Sample positions with 4:2:2 digitization format. 2.6.2 Digital video 4：2：0 format is used in digital video broadcast applications Interlaced scanning is used and the absence of chrominance samples in alternative lines The same luminance resolution but half the chrominance resolution Fig2.22 Figure 2.22 Sample positions in 4:2:0 digitization format. 2.6.2 Digital video 525-line system Y 720 480 Cb Cr 360 240 625-line system Y 720 576 Cb Cr 360 288 13.5 10 8 2 3.37510 8 162Mbps 6 6 2.6.2 Digital video HDTV formats: the resolution to the newer 16/9 wide- screen tubes can be up to 1920*1152 pixels The source intermediate format (SIF) give a picture quality comparable with video recorders(VCRs) 2.6.2 Digital video The common intermediate format (CIF) for use in videoconferencing applications Fig 2.23 The quarter CIF (QCIF) for use in video telephony applications Fig 2.24 Table 2.2 Figure 2.23 Sample positions for SIF and CIF. Figure 2.24 Sample positions for QCIF. 2.6.3 PC video