DAA Notes Module 1

VivekanandaCollege e of Engineering & Technolog ogy N Nehru nagar post, Puttur, D.K. 5742013 Lecture Notes on 15CS43 Design and Analysis lysis of Algorithms (CBCS Scheme) Prepared by Mr. Harivinod N Dept. of Computer C Science and Engineering, VCET Puttur Jan 2017 Module--1 : Introduction to Algorithms ms Contents 1. Introduction 4. Important Proroblem Types 1.1. What is an Algo gorithm? 4.1. Sorting 1.2. Algorithm Speci ecification 4.2. Searching ing 1.3. Analysis Framewo work 4.3. String pro processing 2. Performance Analysiysis 4.4. Graph Problems Pro 2.1. Space complexit xity 4.5. Combinat natorial Problems 2.2. Time complexityity 5. Fundamental al Data D Structures 3. Asymptotic Notation ons 5.1. Linear near Data Structures 3.1. Big-Oh notation on 5.2. Graphs 3.2. Omega notationon 5.3. Trees 3.3. Theta notation 5.4. Sets andd Dictionaries. D 3.4. Little-oh notatio tion 3.5. Mathematical ana nalysis Cours ourse website: www.techjourney.in Lecture Notes | 15CS43 CS43 – Design & Analysis of Algorithms | Module 1: Introdu troduction 1. Introduction 1.1. What is an Algorithm?? An algorithm is a finite setet of instructions to solve a particular proble lem. In addition, all algorithms must satisfy the following fol criteria: a. Input. Zero or morore quantities are externally supplied. b. Output. At least one on quantity is produced. c. Definiteness. Eachch instruction is clear and unambiguous. It mus ust be perfectly clear what should be donone. d. Finiteness. If wee trace t out the instruction of an algorithm, the hen for all cases, the algorithm terminate ates after a finite number of steps. e. Effectiveness. Ever very instruction must be very basic so that it ca can be carried out, in principle, by a person per using only pencil and paper. It is not ot enough that each operation be defini inite as in criterion c; it also must be feasible. Algorithm design and analy lysis process - We now briefly discuss a seq equence of steps one typically goes through in desig signing and analyzing an algorithm • Understanding the Prob oblem - From a practical perspective, the firstrst thing you need to do before designing an algorithm al is to understand completely the pr problem given. An input to an algorithm spec ecifies an instance of the problem the algorithm thm solves. It is very important to specify exactl ctly the set of instances the algorithm needs to handle. • Ascertaining the Capab bilities of the Computational Device - Onc nce you completely understand a problem, you ou need to ascertain the capabilities of the com computational device the algorithm is intended ded for. Select appropriate model from seq equential or parallel programming model. Prerpared by Harivinod N www.techjourney.in Page|1.2 i.in Page|1. problems for which tthere is no known satisfactory algorithm.. however. Another desirable characteristic of an algorithm is simplicity.Once an algorithm has beenn sspecified.The T next principal decision is to choose betw tween solving the problem exactly and solving ing it approximately. the sieve of Erat ratosthenes would run longer if we used a linke ked list instead of an ion. Prerpared by Harivinod N www. • Proving an Algorithm’s ’s Correctness .An algorithm design techniquee (or “strategy” or “paradigm”) is a generall approach a to solving problems algorithmically lly that is applicable to a variety of problemss from f different areas of computing. you have to prove that the algorithm hm yields a required result for every legitimate te input in a finite amount of time. al you need to specify it in some fashion.After correctness. especi ecially when they are used in their code optimiza ization mode. • Designing an Algorithm m and Data Structures . indicating how much extra memo mory it uses. In o algorithm efficiency: time efficiency.Most algorithms are destined to be ultimate ately implemented as computer programs. which can be precisely defined and inve vestigated with mathematical rigor. for others. Unlike efficieniency. you have to prove its correctness. like beauty.3 .techjourney. a proof of correctness is quite easy asy. Lecture Notes | 15CS43 CS43 – Design & Analysis of Algorithms | Module 1: Introdu troduction • Choosing between Exac act and Approximate Problem Solving .One should pay ay close attention to choosing data structuress appropriate a for the operations performed byy the algorithm. • Coding an Algorithm . is to a considerable degree in the he eye of the beholder. Modern compilers do provrovide a certain safety net in this regard. They provide pro guidance for designing algorithms foror new problems. A com ommon technique for proving correctness is to use mathematical induction because an alg lgorithm’s iterations provide a natural sequencece of steps needed for such proofs. That T is. Impleplementing an algorithm correctly is necessary ry but not sufficient: you would not like to dim iminish your algorithm’s power by an inefficie ient implementation. ty. Pseudocode is a mi mixture of a natural language and programmin ing language like constructs. simplicity. it can be quite complex.Once you have designed an algorithm. there are importa rtant problems that simply cannot be solvedd exactly e for most of their instances and somee of o the available algorithms for solving a prob oblem exactly can be unacceptably slow because use of the problem’s intrinsic complexity. and nd its usage often yields more succinct algorithm thm descriptions. Pseudocode is usually us more precise than natural language. and space ce efficiency. fas These are the two options that are re most widely used nowadays for specifyingg algorithms. Because. For some algorithms. by far the most importartant is efficiency. • Analyzing an Algorithm m .. the inherent ambiguity am of any natural language makes a concise and clear description of algorithmss surprisingly difficult. indic fact. there are two kindss of dicating how fast the algorithm runs. Algorithms + Data Structures = Program array in its implementation ams • Methods of Specifying an Algorithm.e. • Algorithm Design Tech chniques . For example. a Using a natural language hass an a obvious appeal. 4 . In C++ the same algorithm can be specified as follows Here Type is a basic or user defined de data type. Algorithm Specification tion An algorithm can be specified ed in 1) Simple English 2) Graphical representatio tion like flow chart 3) Programming language age like c++ / java 4) Combination of above ve methods. Example 1: Factorial computa tation n! = n * (n-1)! Example 2: Binomial coefficie cient computation Example 3: Tower of Hanoii problem pr Example 4: Permutation Gene nerator Prerpared by Harivinod N www. Lecture Notes | 15CS43 CS43 – Design & Analysis of Algorithms | Module 1: Introdu troduction 1. Using the combination of simple si English and C++.2.in Page|1.techjourney. Recursive algorithms An algorithm is said to be recursive re if the same algorithm is invokedd iin the body (direct recursive). the algorithm for selection sort is specified as follows. Algorithm A is said aid to be indirect recursive if it calls another er algorithm which in turn calls A. As another example. The choice of an appropriatete size metric can be influenced by operationss of the algorithm in question. the basic operation is a key com omparison. both resources time and space were w at a premium. Therefo efore..g. For Fo example. In the early days of electronic ic computing. e This approach is both excessivelyy difficult d and. it is logical to investigate an algorithm's efficiency ef as a function of some parameterter n indicating the algorithm's input size. multiply m larger matrices. usually unnecessaryry.techjourney. This metric usually gives a better ideaa aabout the efficiency of algorithms in question. Measuring an Input’s Size It is observed that almost all ll algorithms a run longer on larger inputs. for suc uch algorithms. the research exper perience has shown that for most problems. if it works by processing ng words. Therefore. it takes longer to sort larger arrays. followingg a well-established tradition of algorithm textbook oks. One possible approach is to count the numbber of times each of the algorithm's operations iss executed.. computer scientist b in the n's binary representation: log n 1. For example. For such ists prefer measuring size by the number b off bits algorithms.3. and compute the number of times the basic operation is execu ecuted. called the basasic operation.. In addition. The thing to do is to identify the most impportant operation of the algorithm. eff we would like to have a metric thahat does not depend on these extraneous factors. Lecture Notes | 15CS43 CS43 – Design & Analysis of Algorithms | Module 1: Introdu troduction 1. Units for Measuring Runnin ing lime To measure an algorithm's efficiency. the operation contributing the most to the total running time.5 . Analysis Framework General framework for analy lyzing the efficiency of algorithms is discuss ssed here. then we should measure the size by the numb mber of characters. where the choice of a parameter indicating an input ut size does matter. algorith rithms for matrix multiplication and polyn lynomial evaluation require two arithmetic operatio tions: multiplication and addition. how ow should we measure an input's size for a spell-checking algorithm? If the algorithm examines individual characters of its inpu put. we primarily concentrate on time efficienc ency. There are situations. we can achieve much more spectacular progress in speed than in space. We should make a special note no about measuring the size of inputs for algorithms alg involving properties of numbers (e. For example.in Page|1. and so on. space efficiency deals with the extra space s the algorithm requires. Now the amount of extra spac ace required by an algorithm is typically nott oof as much concern. most sorting algorithms al work by comparing elements (keys)(ke of a list being sorted with each other. checking whether a given integer n is prime).g e. as we shall see. Prerpared by Harivinod N www. we should count their number in the inpu put. Time efficie iciency indicates how fast an algorithm in questionn runs. There are two kinds of efficiency: timee efficiency and space efficiency. Best-Case. it is the function's n's order of growth that counts: just look at ta table which contains values of a few functions parti rticularly important for analysis of algorithms. and let C(n) be the number of tim imes this operation needs to be executed for this th algorithm.6 .in Page|1. Prerpared by Harivinod N www. Lecture Notes | 15CS43 CS43 – Design & Analysis of Algorithms | Module 1: Introdu troduction Let cop be the execution timee of an algorithm's basic operation on a partic ticular computer. Worst-Case. Orders of Growth Why this emphasis on the count's co order of growth for large input sizes? s? Because for large values of n. the formula can give a reasona nable estimate of the algorithm's running time. Then we can estimate the runningg time T(n) of a program implementing this is algorithm on that computer by the formula: unless n is extremely large or very small. and d Average-Case Efficiencies Definition: The worst-casee efficiency e of an algorithm is its efficiencyy for the worst-case input of size n.techjourney. Table: Values of several functions important for analysis of algorithms Algorithms that require an exponential ex number of operations are practica tical for solving only problems of very small sizes. It is for these reasons that the he efficiency analysis framework ignores multi ltiplicative constants and concentrates on the count nt's order of growth to within a constant mul ultiple for large-size inputs. Consider the algorithm for seq equential search. for which thee algorithm a runs the longest among all possible le inputs of that size. Thu hus it guarantees that for any instance of size n. Let us consider again sequenti ntial search. be Note that average-case efficiency can annot be obtained by taking the average of the wors rst-case and the best-case efficiencies. Lecture Notes | 15CS43 CS43 – Design & Analysis of Algorithms | Module 1: Introdu troduction The running time of above algorithm al can be quite different for the same me list size n. Investigation of the average-ccase efficiency is considerably more difficult ult than investigation of the worst-case and best-cacase efficiencies. The wors orst-case analysis provides algorithm's efficien ency by bounding its running time from above. In the case of an unsuccessful searc arch. among all possible inputs off size In general. In the case of a successfull search.7 . itss running ru time on the worst-case inputs. Prerpared by Harivinod N www. Definition: The best-case efficiency eff of an algorithm is its efficiency for or the best-case input of size n. best-case inp nputs are lists of size n with their first elements equ qual to a search key. when there are no matching elements or the first matching element el happens to be the last one on the list.techjourney. we analyze the algorithm alg to see what kind of inputs yield thee llargest value of the basic operation's count C(n) among a all possible inputs of size n and then compute c this worst- case value Cworst (n). for which the algorit rithm runs the fastest among all possible inputs uts of that size.in Page|1. Also. The standard assumptions are that at (a) the probability of a successful search is equ qual top (0 ≤ p ≤ 1) and (b) the probability ty of the first match th occurring in the i position of the list is the same for every i. Definition: the average-casese complexity of an algorithm is the amountt oof time used by the algorithm. We determine the kind of inputs inp for which the count C(n) will be thee ssmallest among all possible inputs of size n.n the running time will not exceed Cworst (n).p). neither the he worst-case analysis nor its best-case coun unterpart yields the necessary information about an a algorithm's behavior on a "typical" or "random" "ra input. The analysis of the best-case se efficiency is not nearly as important as tha hat of the worst-case efficiency. This information is provided by ave verage-case efficiency. averaged over alll possible p inputs. Therefo comparisons is n with the prob efore. Forr example. But there are many importortant algorithms for which the average case efficficiency is much better than the overly pess essimistic worst-case efficiency would lead us to believe. We can findd tthe average number of key comparisons Cavg (n) as follows. the number of robability of such a search being (1.. In the worst case. for sequential search. and the number of comparisons mad ade by the algorithm in such a situation is obviouiously i. s the probability of the first match ooccurring in the ith position of the list is p/n for every e i. the algorithm makes the largest number of key comparisons si n: Cworst(n) = n. Cbest(n) = 1. b and c. Space Sp efficiency is measured by counting the th number of extra memory units consumed by b the algorithm. the number of For the algorithm given here elements to be summed. Performance Analysis lysis 2. ( We call these parameters as instance ce characteristics) Space requirement S(P) of an algorithm P. Ssum(n) ≥ (n+3) (n+ n for a[ ]. and best-case efficiencies. variables. • A variable part that depends de on the input. re the problem instances are characterized by n. constants co and so on. The Space required by an algorithm al is the sum of following components • A fixed part that is independent ind of the input and output. 2.techjourney. Lecture Notes | 15CS43 CS43 – Design & Analysis of Algorithms | Module 1: Introdu troduction Summary of analysis framew ework • Both time and space efficie iciencies are measured as functions of the algororithm's input size. So the spa space complexity can be written as. i and s. Thee space s needed by a[ ] depends on n. For such algorithms.se.1 Space complexity Total amount of computer memory m required by an algorithm to comple plete its execution is called as space complexity of that algorithm. average-case.8 . wee need n to distinguish between the worst-case. Prerpared by Harivinod N www. • The framework's primary ry interest lies in the order of growth of thee algorithm's a running time (or extra memory uninits consumed) as its input size goes to infinity ity. Sp is the inst nstance characteristics Example-1: Consider followin wing algorithm abc() Here fixed component depend aracteristics Sp=0 nds on the size of a. • The efficiencies of somee algorithms a may differ significantly for input uts of the same size. This inclu cludes memory space for codes. • Time efficiency is measuasured by counting the number of times the algorithm's basic operation is executed. One each for n. output and recursion sta stack. S(P) = c + Sp where c is a constant depends on the fixed part.in Page|1. Also instance char Example-2: Let us considerr the th algorithm to find sum of array. The second method to determ rmine the step count of an algorithm is to bui uild a table in which we list the total number of step teps contributed by each statement. We also introduce state iate amount into the program. the execution time oro run-time of the program is refereed as its it time complexity denoted by tp (instance chara aracteristics). After the introduction of the Example-1: Consider the algo he count the program will be as follows. So for the estimation of the timee complexity c we count only the number of program pr steps.techjourney. So when each timee original program executes. An example ple is shown below. Also A different instructions will take differentt ttime to execute.9 .in Page|1. the count alsoo incremented i by the step count. timate that invocation of sum( ) executes tota From the above we can estim otal number of 2n+3 steps. Lecture Notes | 15CS43 CS43 – Design & Analysis of Algorithms | Module 1: Introdu troduction 2. oduce a new variable count to the program whhich is initialized to In the first method we introd atements to increment count by an appropriat zero. Exact estimation runtime iss a complex task. as the number of instru truction executed is dependent on the input data. We can determine the stepss needed n by a program to solve a particularr pproblem instance in two ways. This is the sum of the time ta taken to execute all instructions in the program. c For example comment has zero steps.2 Time complexity Usually. A program step is loosely defined de as syntactically or semantically meani aning segment of the program that has and execut cution time that is independent of instance characteristics. lgorithm sum( ). Prerpared by Harivinod N www. ste assignment statement has one step and so on. To compare and rank such orders of growth.techjourney. Θ (big theta) and o(lit (little oh) 3. and compute ththe number of times the basic operation is executed ted. Lecture Notes | 15CS43 CS43 – Design & Analysis of Algorithms | Module 1: Introdu troduction Example-2: matrix addition The above thod is both exces essively difficult and.in Page|1. computer scientists use three notations: O(bigO oh). that is. usually unnecessary. dependingdep on which algorithm one chooses and how one parameterizes it. the operation contributing the moost to the total running time. Big-Oh notation Definition: A function t(n) is said to be in O(g(n)). called the bas asic operation. One has to make a compromise c and to exchange computing time forfo memory consumption or vice versa. if t (n) is bounded above by some constant mul ultiple of g(n) for all large n.e.)). 3. i. Prerpared by Harivinod N www.10 . denoted t(n)∈O(g(n)). it can annot be solved with few computing time and loww memory consumption. Asymptotic Notations ions The efficiency analysis frame mework concentrates on the order of growth th of an algorithm’s basic operation count as the principal pr indicator of the algorithm’s efficienc ncy. Ω(big omega).1.. th. Th The thing to do is to identify the most important operation of the algorithm. Trade-off There is often a time-space-tr tradeoff involved in a problem. if there eexist some positive constant c and some nonnegati ative integer n0 such that t(n) ≤ c g(n) for all n ≥ n0. . let us for 100n + 5 ≤ 100nn + n (for all n ≥ 5) = 101n ≤ 101n2. Example: To prove n2 + n = O(n3) Strategies for Big-O Someti etimes the easiest way to prove that f(n) = O(g (g(n)) is to take c to be the sum of the positive ve coefficients of f(n). O(g(n)) is the sett of o all functions with a lower or same order of growth as g(n) Examples: ormally prove 100n + 5 ∈ O(n2) As another example. Lecture Notes | 15CS43 CS43 – Design & Analysis of Algorithms | Module 1: Introdu troduction Informally.techjourney. We can usually ig ignore the negative coefficients.in Page|1. n0=5) Note that the definition gives alues for constants c es us a lot of freedom in choosing specific val and n0.11 . (c=101. Prerpared by Harivinod N www. denoted t(n) ∈ Θ(g(n)). Prerpared by Harivinod N www. i. if t (n (n) is bounded both above and below by some pos ositive constant multiples of g(n) for all largee n. i... if t(n) is bounded below by some positive cons nstant multiple of g(n) for all large n.techjourney.3. n i.e. Here is an example of the formrmal proof that n3 ∈ Ω(n2): n3 ≥ n2 for all n ≥ 0. Example: Example: To prove n3 + 4n2 = Ω(n2) 3. Omega notation Definition: A function t(n) is said to be in Ω(g(n)). iif there exist some positive constant c and somee nonnegative n integer n0 such that t(n) ≥ c g(n) for all n ≥ n0. Theta notation A function t(n) is said to bee in Θ(g(n)). denoted t(n)∈ Ω(g(n)) )).2.12 .. we can select c = 1 and n0 = 0.e. Lecture Notes | 15CS43 CS43 – Design & Analysis of Algorithms | Module 1: Introdu troduction 3.e.. if there exist some positive constants c1 and nd c2 and some nonnegative integer n0 such thattha c2 g(n) g t(n) c1g(n) for all n n0 .in Page|1. in Page|1. Thus. – Being able to do a little algebra helps. – Quite often.techjourney. we have hav to pick c < 1.13 . – A good strategy is to pick a value of c which you think will w work. • The following theorem shows sho us that proving f(n) = Θ(g(n)) is nothingg new: Theorem: f(n) = Θ(g(n)) Θ if and only if f(n) = O(g(n)) and f(n)) = Ω(g(n)). and determine which value of n0 is i needed. we just apply the previous pre two strategies. Lecture Notes | 15CS43 CS43 – Design & Analysis of Algorithms | Module 1: Introdu troduction Example: n2 + 5n + 7 = Θ(n2) Strategies for Ω and Θ • Proving that a f(n) = Ω(g(n g(n)) often requires more thought. – We can sometim imes simplify by ignoring terms of f(n)) with the positive coefficients. Prerpared by Harivinod N www. 14 . Lecture Notes | 15CS43 CS43 – Design & Analysis of Algorithms | Module 1: Introdu troduction Prerpared by Harivinod N www.techjourney.in Page|1. 15 .techjourney. we represent it by little-oh. Little Oh The function f(n) f = o(g(n)) [ i.4.in Page|1.e f of n is a little oh of g off n ] if and only if lim 0 → Example: For comparing the order of growth gro limit is used If the case-1 holds good in the he above limit. Lecture Notes | 15CS43 CS43 – Design & Analysis of Algorithms | Module 1: Introdu troduction 3. Prerpared by Harivinod N www. techjourney.16 .in Page|1. Lecture Notes | 15CS43 CS43 – Design & Analysis of Algorithms | Module 1: Introdu troduction 3. Basic asymptotic Efficien ficiency Classes Class Name Comments Prerpared by Harivinod N www.5. Lecture Notes | 15CS43 CS43 – Design & Analysis of Algorithms | Module 1: Introdu troduction 3. average-case. Set up a sum express essing the number of times the algorithm’s ’s basic operation is executed. establish its order of growowth. If I it also depends on some additional proper erty. Therefore. Using standard formuulas and rules of sum manipulation. Th no need to distinguish worst. 2. Total number of basic operatio tions (comparison) are. the worst-case.) 3.techjourney. Decide on a parameterter (or parameters) indicating an input’s size.6. best-case efficiencies have to t be investigated separately. best and ave verage cases. Identify the algorithm’’s basic operation. if necessary. Note that number of comparisrisions will be same for all arrays of size n. 5. (As a rule. Example-2: To check whethe ther all the elements in the given array are distinct d Algorithm Prerpared by Harivinod N www. 4. Mathematical Analysis sis of Non-recursive & Recursive Algorithms ithms Analysis of Non-recursive Algorithms Al General Plan for Analyzing the th Time Efficiency of Nonrecursive Algorithm hms 1. Example-1: To find maximu um element in the given array Algorithm Here comparison is the basic ic operation. either er find a closedform formula for the countt or.. and.17 . it is located inn iinnermost loop.in Page|1. o at the very least. Check whether the number num of times the basic operation is execute ted depends only on the size of an input. all the elements in the rue). Algoritham also have same nu number of additions 3 A(n) = n Total running time: Prerpared by Harivinod N www.techjourney. The maximum no. ). Other than the worst case. Lecture Notes | 15CS43 CS43 – Design & Analysis of Algorithms | Module 1: Introdu troduction Here basic operation is compa parison.e. ( Forr example if the first two elements of the array are re equal. only one comparison is computed). So S in general C(n) 2 =O(n ) Example-3: To perform mat atrix multiplication Algorithm Number of basic opera erations (multiplications) is Total running time: Suppose if we take into accoun ount of addition. the he total comparisons are less than .18 . of comparisons hhappen in the worst th array are distinct and algorithms return tru case. (i.in Page|1. Total number of basic operatio tions (comparison) in the worst case are. the worst-case. with an appropriate initial cond ndition. if it can. for the number of times the basic operaeration is executed. of times es Analysis of Recursive Algori rithms General plan for analyzing the time efficiency of recursive algorithms 1. 2. and best-case efficiencies must bee investigated i separately. Lecture Notes | 15CS43 CS43 – Design & Analysis of Algorithms | Module 1: Introdu troduction Example-4: To count the bit its in the binary representation Algorithm The basic operation is count=c t=count + 1 repeats no. Identify the algorithm’ m’s basic operation. Set up a recurrence ce relation. Example-1 Algorithm Since the function F(n) is com mputed according to the formula ns M(n) needed to compute it must satisfy thee eequality The number of multiplications Prerpared by Harivinod N www.techjourney. ascertain the order of growth of its solution.in Page|1. Decide on a parameterter (or parameters) indicating an input’s size. 4. Solve the recurrence or.19 . or at least. average--case. 3. Check whether the number nu of times the basic operation is exec xecuted can vary on different inputs of thee same size. The goal is to move all the disks to the third peg.techjourney. There are n disks of different sizes that can slide onto any of three peg egs. which is illustrated in Figur ure. if necessary. Thus recur urrence relation and initial condition for the algorit rithm’s number of multiplications M(n) can be stated as We can use backward substitu itutions method to solve this …. o we first move recur cursively n-1 disks from peg 1 to peg 2 (with pe peg 3 as auxiliary). • To move n>1 disks from pegp 1 to peg 3 (with peg 2 as auxiliary).in Page|1.20 . all the disks are on the first pegg iin order of size. Figure: Rec ecursive solution to the Tower of Hanoi puzzle zle The number of moves M(n) depends de only on n. we move the singl ngle disk directly from the source peg to the des destination peg. We can move only ly one disk at a time. • If n = 1. o finally. and it is forbidden to place a larger la disk on top of a smaller one. and. o then move the large rgest disk directly from peg 1 to peg 3. Lecture Notes | 15CS43 CS43 – Design & Analysis of Algorithms | Module 1: Introdu troduction Such equations are called recu currence Relations Condition that makes the alg lgorithm stop if n = 0 return 1. Example-2: Tower of Hanoi oi puzzle. the largest on the bottom and thehe smallest on top. The recurrence equationn is i We have the following recurre rrence relation for the number of moves M(n): Prerpared by Harivinod N www. using the second one ass ana auxiliary. Initially. move recurursively n-1 disks from peg 2 to peg 3 (using peg p 1 as auxiliary). The problem has an elegant recursive re solution. In this puzzle. after i substitutions. we can get the total number of calls cal made by the Tower of Hanoi algorithm: Figure: Tree of recursivee calls c made by the recursive algorithm for thee T Tower of Hanoi puzzle.21 . we get Since the initial condition iss specified s for n = 1.techjourney.1. Prerpared by Harivinod N www. Lecture Notes | 15CS43 CS43 – Design & Analysis of Algorithms | Module 1: Introdu troduction We solve this recurrence by the th same method of backward substitutions: The pattern of the first three sums su on the left suggests that the next one will ill be 24 M(n − 4) + 23 + 22 + 2 + 1. by counting the th number of nodes in the tree obtained byy recursive calls. we get the following formula for the solulution to recurrence.in Page|1. Example-3 The recurrence relation can be written as . which is achieved for i = n . Also note that A(1) = 0. and generally. Alternatively. techjourney.in Page|1.22 . Lecture Notes | 15CS43 CS43 – Design & Analysis of Algorithms | Module 1: Introdu troduction The standard approach to solv fo n = 2k and then olving such a recurrence is to solve it only for take advantage of the theore rem called the smoothness rule which claim ims that under very k broad assumptions the orderr ofo growth observed for n = 2 gives a correcrect answer about the order of growth for all valuess of o n. Prerpared by Harivinod N www. String-processing algorithms have been impo portant for computer science in conjunctio tion with computer languages and compiling issueues. String processing. As a practical matter. called a searchh key. String Processing In recent decades. ed Graphs can be used for modeling a wide variety of applications. exc xcept. some are suitable only for listss rresiding in the fast memory. and so on. k in a given set. (or a multiset. The second notable feature of a sorting algorithm alg is the amount of extra memory thee algorithm requires. They range from the straightforward rd sequential search to a spectacularly efficient but bu limited binary search and algorithms bassed on representing the underlying set in a differe rent form more conducive to searching. Combinatorial problems. Two properties of sorting alglgorithms deserve special mention. the rapid proliferation pr of applications dealing with non--numerical data has intensified the interest off researchers and computing practitionerss in string-handling algorithms. a graph can be thought of as a collection co of points called vertices. 4.2. 4. there is no algori orithm that would be the best solution in all situatio tions.4. while others do better on almost-so sorted lists. 4. while others can bee adapted a for sorting large files stored on a disk isk. some of which w are connected by line segments called edges. There T are plenty of searching algorithms to choosose from. project scheduling. social and economi mic networks. we usuallyy need n to sort lists of numbers. A sortingg algorithm is called stable if it preserves the relat lative order of any two equal elements in its input. Although some algorithms are indeed better than others. Some of the algorithms are simple but relrelatively slow. Important Problem m Types Ty In this section. Sorting The sorting problem is to rear earrange the items of a given list in non-decre creasing order.techjourney. which permits its several elements to have the same value). Stud udying different technical and social aspects ts of the Internet in Prerpared by Harivinod N www. Graph problems. An algorithm is said to be in-pplace if it does not require extra memory.in Page|1. characters from fro an alphabet or character strings. Lecture Notes | 15CS43 CS43 – Design & Analysis of Algorithms | Module 1: Introdu troduction 4. Informally. including transpo sportation. while others are faster but more complex. for a few memory units. Searching The searching problem deals ls with finding a given value.. A string is a sequence s of characters from an alphabet. et. we are goin oing to introduce the most important proble blem types: Sorting. communication. Searching. possibly.23 . 4. co some work better on randomly ord rdered inputs.3.1. Graph Problems One of the oldest and most interesting int areas in algorithmics is graph algorit rithms. The latter la algorithms are of particular importance for real-world re applications because they are indisp ispensable for storing and retrieving information from rom large databases. and games. Second. i. Linear Data Structures res The two most important eleme mentary data structures are the array and the linked lin list. the number off combinatorial c objects typically grows extr tremely fast with a problem’s size.5. Another extension is i the structure called the doubly linked list. l contains pointers to both its successor and an its predecessor. First.24 .in Page|1. Prerpared by Harivinod N www. a collection of data items ar arranged in a certain linear order. Their difficulty stemss ffrom the following facts.1. Combinatorial Problems lems Generally speaking. except the first and thee last. A list is a finite sequence off data d items. each containing co two kinds of information: some data and nd one or more links called pointers to otherr nnodes of the linked list. Fundamental Data a Structures Stru Since the vast majority off algorithms of interest operate on data. there are no known wn algorithms for solving most such proble blems exactly in an acceptable amount of time.techjourney.e. economists. A data structure can be defined as a particularr scheme s of organizing related data items. In a singly linked list. from both a theoretical and practical p standpoint.. A linked list is a sequence of zero or more elements called nodes. combinat natorial problems are the most difficult proble blems in computing.li in which every node. The basic ope perations performed on this data structuree are searching for. reaching unimaginable un magnitudes even for modera rate-sized instances. 5. and social scientis tists. A (one-dimensional) array is a sequence of n items of the same data ty type that are stored contiguously in computer mem emory and made accessible by specifying a value v of the array’s index. 5. 4. each eac node except the last one contains a single le pointer to the next element. Lecture Notes | 15CS43 CS43 – Design & Analysis of Algorithms | Module 1: Introdu troduction particular is one of the acti ctive areas of current research involving co computer scientists. particular p ways of organizing data play a critical al role in the design and analysis of algorithmss. e). e). (b. b. deleting itss la largest element. This end is called the top because a stack ck is usually visualized not horizontally but ve vertically—akin to a stack of plates whose “operatio ations” it mimics very closely. d. and new elem lements are added to the other end. Lecture Notes | 15CS43 CS43 – Design & Analysis of Algorithms | Module 1: Introdu troduction inserting.. and adding a new element. and deleting an element. c. A priority queue is a collec lection of data items from a totally ordered univ niverse (most often.techjourney. f ). (b. is a list from which elements are deleted from fr one end of the structure. Directed grap aphs are also called digraphs. E = {(a.ns. } E = {(a. d. Graphs A graph is informally thoughght of as a collection of points in the plane ccalled “vertices” or nodes. (b. integer or real numbe bers). (c. c). The digraph depicted in Figure ure 1. c). A stack is a list in which insertions ins and deletions can be done only at the th end.in Page|1. called the rear rea (this operation is called enqueue). f )}. c).” some of them connecnected by line segments called “edges” or “arcs.2. stacks cks and queues. f }. e) e). (d.6b has six vertices and eight directed edge ges: V = {a. a queue operates in a “first-in–first-ou out” (FIFO) fashion—akin to a queue of cust stomers served by a single teller in a bank. c. A queue. d. c). Queu eues also have many important applications. are particularly important. (b. f ). (c. (a. Cons nsequently. on the other hand. (e. f }. (e. e. (d. e Two special types of lists. called the front (thi this operation is called dequeue). a). e). c). (e (e.25 . Graph Representations . (d. The graph depicted in Figuree (a) ( has six vertices and seven undirected edge ges: V = {a. A data structure that seeks to sa satisfy the needs of such applications is called a priority queue. A graph whose every ry edge is directed is called directed. The principal operations on a priority queue ue are finding its largest element.” “ar A graph G is called undirected if every edge ed in it is undirected. b. including several algorithms for graph problemsms. e. d). Many important applications ns require selection of an item of the highest est priority among a dynamically changing set off candidates. 5. in which the element in th th Prerpared by Harivinod N www.Graphs Gr for computer algorithms are usually represented rep in one of two ways: the adjacency matr trix and adjacency lists. The adjacency matrix of a graph g with n vertices is an n x n boolean matrix m with one row and one column for each of the the i row and the jth th graph’s vertices. f )}. that contain all the vertices adjacent ad to the list’s vertex (i. A graphph that has no cycles but is not necessarily connecte cted is called a forest: each of its connected co components is a tree. Among the many propertiess of o graphs. These numbers are called weights or costs. all the vertices ces connected to it by an edge).3. A path pat from vertex u to vertex v of a graph G can be b defined as a sequence of adjacent (conn nnected by an edge) vertices that starts with u andd ends e with v. Weighted Graphs: A weigh ghted graph (or weighted digraph) is a graphph (or digraph) with numbers assigned to its edges. Trees have several importantt properties p other graphs do not have. oone for each vertex. es.. a free fre tree) is a connected acyclic graph. Graphs with several cononnected components do happen in real-world rld applications. It is important to know for many ny applications whether or not a graph under er consideration has cycles. and equal to 0 if there is no such edge.in Page|1.techjourney. The adjacency lists of a graph aph or a digraph is a collection of linked lists. A graph is said to be connecte cted if for every pair of its vertices u and v there the is a path from u to v.26 . Both B are based on the notion of a path. In particu ticular.e. A cycle is a path of a positive length that starts and ends at the sam same vertex and does not traverse the same edge moore than once. the number of edges in a tree is always onee less le than the number of its vertices: |E| = |V| . Trees A tree (more accurately. two are important for a great numb mber of applications: connectivity and acyclicity.1 Prerpared by Harivinod N www.. 5. Lecture Notes | 15CS43 CS43 – Design & Analysis of Algorithms | Module 1: Introdu troduction i an edge from the ith vertex to the jth vertex column is equal to 1 if theree is tex. in Page|1. there always existss exactly e one simple path from one of these vertices ve to the other. Prerpared by Harivinod N www.. the vertices two edges apart from the root still below (level 2).27 .techjourney. Such trees are cal alled binary search trees. and so on. Ordered Trees. Lecture Notes | 15CS43 CS43 – Design & Analysis of Algorithms | Module 1: Introdu troduction Rooted Trees: Another veryy important i property of trees is the fact that for or every two vertices in a tree. The depth of a vertex v is the length of the simple path from the root to v. A binary tree can be defined de as an ordered tree in which every vertertex has no more than two children and each child is designated as either a left child or a rightt child ch of its parent. The height of a tree is the length of the longes est simple path from the root to a leaf.An orderedd treet is a rooted tree in which all the childrenen of each vertex are ordered. This property makes it possib sible to select an arbitrary vertex in a free treeee and consider it as the root of the so-called roote oted tree. Binary trees and binary ry search trees have a wide varie riety of applications in computer scien ience. If a number assigned to eachh parental vertex is larger than all the number ers in its left subtree and smaller than all the nummbers in its right subtree. It is convenient to assume as that in a tree’s diagram. a binary tree may also be emptyty. all the children ren are ordered left to right. the t vertices adjacent to the root below it (le (level 1). A rooted tree is usually depicted byy placing its root on the top (level 0 of the tree). The second and more common on way to represent a set for computing purpos oses is to use the list structure to indicate the set’s ’s elements. Sets can be implemented inn computer c applications in two ways. A data structure that th implements these three operations is call alled the dictionary. 3. An efficient implementation on of a dictionary has to strike a compro romise between the efficiency of searching and the th efficiencies of the other two operations.techjourney.g. which comprises all the elements in either or both of them..4.in Page|1. adding a new item. The first considers only sets that are subsets of some large ge set U. Note that if a set is represented by a list. and finding the intersection of twoo sets. The most important set opera rations are: checking membership of a givenn iitem in a given set. Sets and Dictionaries A set can be described as an unordered collection (possibly empty) of distinct di items called elements of the set. depending on the application ap at hand. theth operations we need to perform for a set et or a multiset most often are searching for a given giv item. finding the union of two sets. If set U has n elements. su After being initialized as a collection ion of n one-element subsets.g. called the universal set. and deleting ng an item from the collection. 7}) or by specif cifying a property that all the set’s elementss anda only they must satisfy (e. 5. in which the ith ented by a bit string of size n. The requirement for uniqueness is sometimes es circumvented by the introduction of a m multiset. tr A number of applications in computing require a dynamic partition off ssome n-element set into a collection of disjoint subsets. the collection is subje bjected to a sequence of intermixed union and nd search operations. Lecture Notes | 15CS43 CS43 – Design & Analysis of Algorithms | Module 1: Introdu troduction 5. S = {n: n is a prim rime number smaller than 10}).28 . Dictionary: In computing. S = {2.. or bag. A specificific set is defined either by an explicit listing of its elements (e. This is feasible only for finite sets ets. ***** Prerpared by Harivinod N www. s. This problem is called the set et union problem.. it might be worth maint intaining the list in a sorted order. then any subset S of U can be represen ctor. They T range from an unsophisticated use of arrays ys (sorted or not) to much more sophisticatedd techniques such as hashing and balanced search trees. which comprises all the common eleme ments in the sets. an unordered collection of itemss that are not necessarily distinct. called a bit vect element is 1 if and only if thee ith element of U is included in set S. Documents Similar To DAA Notes Module 1Skip carouselcarousel previouscarousel nextAnalysis of Algorithmsunit 1 ds04-sorting.pdf2 Alg Analysis 11taylorLimits.pdfch02Analysis of Algorithms IDAA JK - Asymptotic Notation - Minggu 4_16HN DAA 15CS43 LectureNotes 119. Algorithms and ComplexityDesign And Analysis of AlgorithmsComplexity Analysis of AlgorithmsAlgorithms Assignmenthw2soldownload_cs111_06 RepetitionContraction Analysislec1-2x2 - Recurrence AlgorithmsCh2 Complexity Growth WDToc 4pxc3873876Lecture 3Richard J. Lipton, Kenneth W. Regan-Quantum Algorithms via Linear Algebra_ a Primer-The MIT Press (2014)APGsDesigning the Algorithmch6Lecture Notes AlgorithmAnalyzing Human Solving MethodsADTScimakelatex.12018.Tyronne PowerMore From harivinodnSkip carouselcarousel previouscarousel nextHN DAA 15CS43 LectureNotes 1cssyll5DIP Unit 1 Handout 5 PropertiesOR NotesOR Notes.pdfFooter MenuBack To TopAboutAbout ScribdPressOur blogJoin our team!Contact UsJoin todayInvite FriendsGiftsLegalTermsPrivacyCopyrightSupportHelp / FAQAccessibilityPurchase helpAdChoicesPublishersSocial MediaCopyright © 2018 Scribd Inc. .Browse Books.Site Directory.Site Language: English中文EspañolالعربيةPortuguês日本語DeutschFrançaisTurkceРусский языкTiếng việtJęzyk polskiBahasa indonesiaSign up to vote on this titleUsefulNot usefulMaster your semester with Scribd & The New York TimesSpecial offer for students: Only $4.99/month.Master your semester with Scribd & The New York TimesRead Free for 30 DaysCancel anytime.Read Free for 30 DaysYou're Reading a Free PreviewDownloadClose DialogAre you sure?This action might not be possible to undo. Are you sure you want to continue?CANCELOK