SnapLogic® Tutorials Document Release: October 2013 SnapLogic, Inc. 2 West 5th Ave, Fourth Floor San Mateo, California 94402 U.S.A. www.snaplogic.com Table of Contents SnapLogic® Tutorials Table of Contents SnapLogic Overview Glossary The Tutorial Environment About the Tutorial Exercise 1: Working with Components Reviewing the Sample Leads to Prospects Pipeline Hands-on Exercise: Creating Components Creating a Writer Component Building the Read-Write Pipeline Exercise 2: Filtering Data Reviewing the California Prospects Pipeline Hands-on Exercise: Creating a Filter Component Duplicating Components Building the Read-Write-Filter Pipeline Exercise 3: Data Services Reviewing the Census Feed and Qualified California Pipelines Hands-On Exercise: Data Services Creating a Join Component Duplicating a Writer Component Building a Data Service Pipeline Exercise 4: Data Sorting Reviewing the Filtered Qualifying California Prospects Pipeline Hands-on Exercise: Sorting Data Exercise 5: Utilizing Pass-Through Reviewing the California Prospects Pipeline, Exercise 5 Hands-on Exercise: Implementing Pass-Through 1 2 4 4 4 5 6 6 8 11 12 16 16 17 19 20 22 22 25 27 28 29 32 32 34 38 38 39 -2- SnapLogic® Tutorials -3- 1 SnapLogic Overview SnapLogic is the only integration platform built on modern web standards and "containerized" Snaps, allowing you to easily connect any combination of Cloud, SaaS or On-premise applications and data sources. Like the web itself, SnapLogic standardizes the access protocols, access methods and data structure for every interaction. These RESTful1 interactions allow data to be exchanged between servers in an organized way that makes it easy for the data to be processed and interpreted. This creates a resource-oriented, data-services layer that provides transformation and integration services for all kinds of integrations. Glossary l Components: Components provide basic integration capabilities such as database queries, file read and write, aggregate, sort, filter, join, and others. Components are configured with specific properties. l Pipelines: Pipelines can be assembled into hierarchies that can implement complex logic and transformations on data for sophisticated integrations. l Repository: Resource definitions are stored locally in a SnapLogic Server’s Repository. The Tutorial Environment To illustrate how to build a data services infrastructure layer with SnapLogic, this tutorial presents four simple exercises. Each exercise builds upon the previous one to demonstrate how existing Resources can be reused as well as introduces new SnapLogic Components and capabilities. Because exercises build on the previous exercise, they should be completed in order as they depend on the successful completion of earlier work. Once you complete this tutorial you will know how to use the SnapLogic Designer interface to: l Configure Resources for data sources and targets l Configure transformation Resources l Link Components to form Pipelines l Specify field mappings between Components l Execute Pipelines from the Designer l Reuse Components in other Pipelines This tutorial is only an introduction to the power of SnapLogic. There are more Components available that can be referenced in the Documentation section located at https://www.snaplogic.com/component-reference/component-reference.htm -4- csv and joins (looks up) that data with the filtered data from Exercise 2. where fields not needed for a specific Component are passed through for use downstream in the Pipeline. Note: The tutorial is based on the version of the software at the time of the release date on the front page. phone (home/work/cell) and address (street. city. $15-25K. The Writer Resource is reused from Exercise 1 and a runtime parameter is set to use a new filename to preserve the previous results. $10-$15K.csv). About the Tutorial This tutorial is based on a hypothetical example that requires transforming a generic list of sales leads into a list of qualified sales prospects. -5- . By default. We begin by processing an input file consisting of leads that might be generated or purchased from some list provider (leads. The objective is to identify which of these leads are sales prospects by filtering them based on location and household income level.irs. For this tutorial. It then writes to a new output file. your experience may vary. where `n` represents the exercise number (for example. If you are using a different version. these output files will be overwritten each time the Pipeline is executed.csv.csv file and writing it out to a file where the output file omits the home and cell phone numbers from the leads. The file includes first name. l Exercise 2 uses the Resources from Exercise 1 and creates another Pipeline that filters the leads to include only the leads that live in California.csv).csv is generated in Exercise 1). This requires selecting only those leads within a specific territory (California) and matching them against household income data for the lead's zip code. The individual tutorials consist of: l Exercise 1 begins by reading the leads.SnapLogic® Tutorials to help you understand how each Component functions. so Resources don't have to be edited. (Parameters are values that can be specified at runtime. Household income information is available at www. l Exercise 4 filters and sorts the joined data based on zip codes where the percentage of households that have an income greater than $50K is above a specified threshold. state. This is done by creating a Pipeline with two Resources: A file Reader and a file Writer.csv file. last name.) l Exercise 3 reads the household income data from the file CAIncomeByZip.gov and provides for each zip code the percentage of households that have income less than $10K. and are typically used for specifying things like file names at runtime. ex_1_prospects. $25K-50K and greater than $50K. Note: The Tutorial Exercise Pipelines generate sample output files like ex_n_prospects. data for the state of California is accessed by reading a local file (CAIncomeByZip. l Exercise 5 introduces you to the concept of pass-through. and zip/postal code). When you run your exercises. are described in the following table. To accomplish this. Your version of SnapLogic includes an example for Exercise 1 that illustrates all the steps you need to take to create the necessary Pipeline. First let’s look at the Exercise 1 Components. connect to SnapLogic Designer using the hostname and port number selected during the installation. l The Properties tab contains file delimiter and output file variables for this Component. you will overwrite these files. The output file will be named: ex_1_prospects. (for example. This is a Component icon.csv.csv).csv file and writes out a file which reorders the First Name and Last Name fields and leaves out the home and mobile phone numbers. Pipelines and Components are indicated by different icons. the delimiter and filename are specified as $?{SOMENAME}. The syntax (dollar sign. This is a folder icon and is used to organize Pipelines and Components on the main screen. Note: In this sample exercise. These icons. To avoid this. as well as the other icons used in the client. and braces) indicates to the Component that the values for these properties are parameters that will be set at runtime. question mark.com:8081 Double-click on the Ex1_Leads_to_Prospects Pipeline within the Library. -6- . Click on the Component named Leads to view its configuration.csv.snaplogic. you can rename your exercise output files to be unique. Or you can simply re-run the sample pipelines to recreate the output files if you need to examine them.2 Exercise 1: Working with Components Exercise 1 reads in the leads. which also indicate their status. For an example of this usage. Reviewing the Sample Leads to Prospects Pipeline After installing SnapLogic successfully. Exercise 1 constructs a Pipeline with two Components: A file Reader and a file Writer. see Hands-on Exercise 1: Creating Components. For example: http://snaplogic1. ex_2_prospects. ex_1_prospects. Note: The Tutorial Pipelines generate the sample output files ex_n_prospects. Icon Type This is a server icon and it appears on the Server List.csv. This is a Pipeline icon. then we’ll create it ourselves. You can also click Suggest to have fields with the same names and datatypes automatically linked. those which have no input views and only output views. l The Preview tab allows you to see the data produced by the Component. l The Input tab specifies the fields (name. and what the filename and path is. Note: The purple and green blocks on the Leads and Prospects Components are output/input views. and what the default values should be. Click on the Component named Prospects to view its configuration. This tab also provides optional descriptive information for this Writer Component. and description) that are to be read from the leads. For each input field you can select an output field from the list of values. Each input field must be linked before the Pipeline can be executed. datatype.csv file fields. the input for the Writer Prospects is being linked to the output from the Reader Leads. l The Run tab contains the Run button which starts the Pipeline processing. Click anywhere on the link to display the Field linking page.SnapLogic® Tutorials l The Output tab defines the input . The green block on the Prospects Component is an input view for this Writer Component to obtain the data from the Reader. Click the following tabs: l The General Info tab provides optional descriptive information for this Pipeline as well as buttons to access the Related Pipelines information and the Scheduler. l The Parameters tab assigns default values to the delimiter and input file name variables. Here. l The Parameters tab shows what parameters are used in the Pipeline. and descriptions. will have this tab present. l The Properties tab defines the output . l The Parameters tab assigns values to the delimiter and input file name variables. Any parameter which does not have a default value is considered a required parameter. data types. The link between Components is created by clicking on a purple output box at the bottom of one Component and dragging it to a green input box at the top of another. There is also a parameters table that lets you specify parameters at runtime that might be different from the default values indicated on the Parameters tab. The Field Linker lets you map fields from the source to the destination. what the delimiter is. -7- .csv file. There are two ways to make the connections. Not every output field must be used and you can use an output field more than once.csv file to be created (written out) by specifying if it has a header row. This tab presents only the information that a Pipeline user would need. In this case. Click on the Pipeline Settings button in the upper left hand corner of the Pipeline. the purple block on the Leads Component is an output view for the data that the Component has read in. Only Components which are "Producers". which must be provided when executing the Pipeline. csv file. but have not saved them. leave the properties unchanged except for the following. We will specify the actual name of the file at runtime.) l Specify the delimiter as the parameter $?{DELIMITER}.csv contains a header row. Since leads. 3. Click Save. you will start to recreate the Leads to Prospects Pipeline that is used through several of the exercises 1. open the ex_1_prospects. you can discard the changes by simply closing the Pipeline’s tab. Double-click on the CSV Reader Component in the Foundry pane. Hands-on Exercise: Creating Components In this exercise. This setting allows us to skip the specified number of lines when reading an input file that might contain column headers. In the URI field enter: SnapLogic/User/Exercise_1/Leads where User is the username you wish to use. Note: You can click on the Component name in the General tab to access the online help for that Component. l Specify the input filename as $?{INPUTFILE}. To confirm that the operation was successful. we will use the name User. Saving is optional until all the edits are made. On the Properties tab. and creates the ex_1_prospects. (For details about these properties you can look at the CSV Read Component Reference documentation. maps it into the Writer input. we want to skip this row. 2. We will specify the actual delimiter character at runtime.csv file and see that the fields you mapped are the field contained in the file. The Leads Component tab opens. -8- . 4. If you have made changes to a Component. be sure to use the same path throughout these exercises so the Components show up in the same folders. and for all of these exercises.Exercise 1: Working with Components When you click the Run button. Your screen should look like the following figure. the Pipeline processes the Reader output. In this example. For each subsequent Component. All changes are stored in the client browser until they are saved to the Data Server. Note: The URI that you specified has generated a folder hierarchy in the Data Server. l Specify the skip_lines value as 1. Substitute your name for User everywhere it is specified in these exercises if desired. then the Suggest button will be enabled at the bottom of the Component tab. . Select the General Info tab. 8. You should see the two paramaters DELIMITER and INPUTFILE added. For the DELIMITER parameter. Select the Parameters tab. given a filename. (comma) as the default value. If you need help configuring the Component. This section contains fields for setting the Author and Description of the Component. These properties are informational only and are optional. 7. 6. 5. derive the number of output fields. 9. open the version of the Leads Component in the "Ex1_Leads_to_Prospects" Pipeline. the Component type is always displayed in the General Info tab and in the Properties tab as shown in the preceding figure. If not. click Add Row twice and name them. It may also be able to -9- .csv. specify the Default Value column as the input path and file name file://tutorial/data/leads. For INPUTFILE. You will not need to set any of these properties for the Tutorial exercises. Click Save. At this point we have specified enough information to allow the Suggest feature to derive the remaining information. enter . This points to a file in the /snaplogic/data/tutorial/data directory on the SnapLogicServer. The CSV Reader Component can. Your screen should look like the following figure.SnapLogic® Tutorials To verify what type of Component you are working with. If a Component supports auto-fill. 10 - . City 8. If you choose not to use Suggest. A sample output definitions screen is shown in the following figure. A dialog opens with the suggested output view and a suggested quote character. . 11. First Name 2.and so on. quote character. Cell Phone 5. You should see the view Output1 with all the field names correctly derived. Keep in mind that the fields in leads.csv file. Street Address 7.Exercise 1: Working with Components recommend the correct delimiter. State 9. Field002. Work Phone 6. and specify their data type as string. 12. Last Name 3. 10. enter meaningful names. and even field names for the output view. Use underscore characters if you want to concatenate words such as Net Profit becomes Net_Profit. Home Phone 4. The output view of the Leads Reader Component should match the fields in the leads. Accept the suggestions by clicking on the Apply All Changes button. All of the fields of the Leads Component should be defined as strings. Note that field names cannot contain spaces. you can create the output view manually by clicking the Add Output button and click Finish to accept the default view name (Output1). Zip Code 13. Click each name Field001. Click Suggest. Click the Output tab.csv contain the following types of information: 1. Here are several things you should notice: l In our output result. Select the Properties tab. Click Save. Navigate to the SnapLogic > User > Exercise_1 folder in the Library pane. We will specify the actual name of the file at runtime. 13. The Reader is now defined. . and description for Field001 as we defined it in the Reader Output field definitions. Now we need to tell the Writer to only write out the last name. Creating a Writer Component 1. Click Validate. Specify the File name as the parameter $?{OUTPUTFILE}. Click Add Row and enter the field name. 4. and provide a description. 5. the New Component dialog will automatically include this path in your URI. The Prospects Writer Component icon displays. Click OK. Note: By navigating to the intended location for a new Component. 11. These fields are optional. 8. Select Create a new (blank) view and click Finish to display the Input1 view. add Prospects to the end so it reads: SnapLogic/User/Exercise_ 1/Prospects. Double-click the CSV Writer Component in the Component Library pane. Validation checks that your Component is well-formed and performs checks such as all required properties are set and the necessary input/output views are defined. Accept the default name and click Next. Add your name as the author. Specify the Delimiter as $?{DELIMITER}. and work phone number from the leads. zip code. Repeat this for last name. Click Save. 7. city. 12. data type. 6. state. a.csv file. 9. we do not want home phone or cell phone. Select the Input tab and click the Add Input button. street address. city. For Header Present keep the default false (as we do not need a header row in the output file). 3. Click Save.11 - . In the URI field. 2. zip. Select the General Info tab. state. and the Prospects Component opens for editing. 10. address. so those fields are not added to the input view definition. and work phone.SnapLogic® Tutorials 14. first name. b. 15. Drag the Leads and Prospects icons onto the grid field (also known as the Pipeline canvas). for these exercises. Also. 14.Exercise 1: Working with Components l We have specified last name first. This allows us to reorder the fields from upstream Components. Click on the canvas. 18. 1. . 8. In the Name column. Click Add Row and repeat for the DELIMITER parameter. so do not define fields you do not want. l Some of the field names do not match exactly with those used by the Leads Component (for example. This is intentional and will be used to illustrate that exact matches are not necessary when linking Components together. Click the Pipeline tab within the Pipeline Properties. 3. Building the Read-Write Pipeline Next we need to build a Pipeline with the Reader and Writer Components to perform a data integration. In the Default Value column. 16. 6. The Writer Component is now defined. Enter a creator and description for this Component. 5. Note: Every field you define in an input view has to be mapped from upstream views. 17. Click OK. 15. file://tutorial/data/ex_1_prospects. Click Save. change PARAM_001 to be the output file parameter name we introduced in the Properties tab. Click Save. add the name Leads_to_Prospects to the path SnapLogic/User/Exercise_1/. Click the Parameters tab and click Add Row. 7. Click the New Pipeline button. Zip vs. The Leads_to_Prospects Pipeline is created and is opened for editing. all fields have to be specified using string as the datatype. Zip_Code). These values are optional.csv. The order of the input fields defines the order in which they will be written out. OUTPUTFILE.12 - . Click the General Info tab. specify the output file path and name. 4. In the URI field. 19. it is important to match the names used here with the ones used in the Parameter Map. 2. Note: While naming the Components as we have done is optional. In this case. By default. As some field names do not match. we can now use different delimiters if we need to. If you do not wish to expose a Component's parameter. link the remaining unlinked fields. The Pipeline will read in the data specified in the Reader Component and write it to the file specified in the Writer Component (ex_1_prospects. you can click the Add button to enter a new parameter. 19. To add a parameter. you can delete it by selecting the parameter and clicking the Remove button. specify a Pipeline parameter name and select the Component parameter to which it maps. 18. By default. Click Pipeline Properties on the canvas toolbar. the Component's parameters are automatically mapped as Pipeline parameters. Double-click on a parameter to edit it's name or value. We are now going to define the Pipeline parameters and how they map to the Pipeline's Components. enter the value to be used when running the Pipeline. and a default value if any. Again.13 - . The Run tab contains three sub-tabs. Click the Parameters tab. Click Save. Click Apply. You can also using a single pipeline parameter and map it to multiple Component parameters. Click Save. Click the Run button. the Field Linker dialog displays automatically whenever you connect two Components. Click the Leads output view box (purple box) and drag to link to the Prospects input view box (green box). 14. 17. For each input view field. You can also click the B button. we could change the values right here on the Run tab and run it. The status of the Pipe- . 15. When you drag a Component onto the Pipeline canvas. Run. For example. Preview Data and Runtime Information. By defining the DELIMITER parameters as INPUT_DELIMITER and OUTPUT_DELIMITER. If we wanted to. which will link any fields whose names and data types match. 11. Edit each parameter and change the default parameter names. Notice that the default values appear in the table. if you know that all input files use comma as the delimiter. 20. 13. select one of the available output view fields. In the Default Value field. Click the Run tab. 10. rather than simply DELIMITER as was defined for both Components.csv). you could map a single parameter FILE_DELIMITER to both Component DELIMITER parameters. 12. The Parameters table in the Run sub-tab is populated with the parameters to be used in this Pipeline. Smart Link will be invoked automatically when you connect two Components together. To add a parameter. we will leave both delimiters as a comma. Map the fields. this behavior can be controlled in the Pipeline Options. 16.SnapLogic® Tutorials 9. . .Exercise 1: Working with Components line's execution is displayed in the Runtime Information sub-tab. 21. The output file also contains 1.000 records.. Notice that we have achieved the goal of creating a prospects output file from the original leads input file where the First Name and Last Name columns have been reordered and which contains just the work phone numbers for all the leads.csv file does not appear as expected.14 - . Last. Verify that the Pipeline performed this task by opening the output file and verifying that the First. You can access detailed information by clicking on the row to access the Show Log menu.000 records.. Also notice that the input file contains 1. Note: If the ex_1_prospects. City fields display properly. open the Show Log menu for the run to see if there was an error. SnapLogic® Tutorials .15 - . ex_1_prospects. .csv. This exercise will show you how to create these two Components.csv). Click the Pipeline Properties button for the Ex2_CA_Prospects Pipeline to see what the Pipeline includes. Reviewing the California Prospects Pipeline Double-click the pipeline Ex2_CA_Prospects and it will open up in the Canvas. To accomplish this. Or you can simply re-run the sample pipelines to recreate the output files if you need to examine them. there are no input views. This data is optional. Your version of SnapLogic includes an example for Exercise 2 that illustrates the steps to create the necessary Components and link them with the Pipeline.csv file for the value equal to CA. Since there is no upstream Component feeding the Pipeline. then we’ll go create it ourselves. ex_2_prospects.3 Exercise 2: Filtering Data Exercise 2 duplicates the Components from Exercise 1 and creates another Pipeline that filters the leads to include only those in California. the Pipeline gets its data from a CSV Reader Component that reads it from a local file. In this exercise. and combine the FilterLeads Component with the Leads and Prospects Components in a new Pipeline that will filter out all the non-California leads. you can rename your exercise output files to be unique. That is different from data that enters a Pipeline from one of its input views. but we will specify a runtime parameter for the Prospects Component so that it can write to a different file. you will overwrite these files. the Pipeline has a CSV Writer Component that writes the data to a local file. Since no data is exiting the Pipeline directly. The filtered output is written to the file named ex_2_prospects. In this exercise. l The Input tab defines the structure of the data that is provided by the upstream Components into the Pipeline. We will reuse the Leads Component exactly as it was configured in Exercise 1.csv. To avoid this. the FilterLeads Component examines the State column in the leads.16 - .csv. how to duplicate and modify the Leads and Prospects Components from Exercise 1. Note: The Tutorial Pipelines generate the sample output files ex_n_prospects. Click the following tabs: l The General tab provides descriptive information for this Component. The Components used in this exercise will be the same ones used for Exercise 1. it has no output views. l The Output tab defines data exiting the Pipeline. When you run your exercises. First let’s look at the Exercise 2 components. (for example. Now double-click the FilterLeads Component: l The Properties tab defines the field to be used to filter data.csv file. Double-click the Filter Component in the Foundry pane. Click anywhere on the link between the FilterLeads Component to the Prospects Component to display the Link menu and select Field Linking to display the Field Linking page. You should see that the output result contains 675 records. its value. You cannot link Prospects to FilterLeads because the Writer Component does not provide output. Filter output view should match its input view. l The Input tab defines the structure of the data that is provided by the upstream Components into the Filter. In this example. Click the link between the Leads Component and the FilterLeads Component to display the Field Linker. The fields coming in from the FilterLeads are mapped here to fields in the Writer Prospects by clicking on the fields in the Incoming fields (available to map) panel and dragging them to the From panel. Now let’s create the Components in Exercise 2. to find all the California leads. 2. The purple block on the Leads Component is an output view and the green block on the FilterLeads Component is an input view. . passes it through the Filter and into the Writer input. When you click Run. and the condition to be met. you click Leads to draw a link to FilterLeads. In this case. The fields coming in from the Reader Leads are mapped here to fields in the FilterLeads input view by clicking on the fields in the Incoming fields (available to map) panel and dragging them to the From panel. the Pipeline processes the Reader output.SnapLogic® Tutorials l The Parameters tab shows what parameters are used in the Pipeline. In this exercise. we need to specify that the Column be State. Then these are matched to the Writer fields by dragging Unmapped outgoing fields to the corresponding To panel such that From fields are on the same line as the fields there are going To. Hands-on Exercise: Creating a Filter Component 1.17 - . Specify the URI as SnapLogic/User/Exercise_2/FilterLeads. l The Output tab defines the structure of the data from the FilterLeads Component. l The Run tab contains the Run button which starts the Pipeline processing. the value be CA. Then these are matched to the FilterLeads fields by dragging Unmapped outgoing fields to the corresponding To panel such that From fields are on the same line as the fields there are going To. and what the default values are if any. then creates the ex_2_prospects. and the condition be ==. There is also a parameters table that lets you specify parameters at runtime that might be different from the default values indicated on the Parameters tab. The link between them is created by clicking on a Component that can generate output and dragging to another Component. 13. Click Save. Repeat the steps you performed to define the input view Input1 to create the output view Output1. Click Close. you will get a brief summary of the fields in that view. This creates a FilterLeads Component that will search the State column of the incoming data stream and select all records that have CA as the state name. The list of values will be empty if no input view has been defined yet. Select the output view Output1 and click Finish. Select the Input tab. just like we did in Exercise 1. 9. Click OK. 14. but instead of copying the view from the Leads Component. The output view definition will be copied from your Leads Component. 5. Click the General Info tab and enter an author name.18 - . These values are optional. Click Save. 6. the wizard will display a list of any input or output views it has. Navigate to the Exercise 1 Leads Component you created in the first tutorial. just copy the input view of your Filter Component. We do not require the Phone_h and Phone_m fields in this Component so click the "X" beside those rows to delete the fields from this view. 15. When you select a Component. 10. . and description then click Save. or copying an existing view from another Component. 8. You must define the input view first before attempting to enter a value for the property. click the Add Input button. 7. 12. Specify the properties to be: l Field: State l Operator: == l Value: CA l View: Output1 Any Component which defines a property as being an input view field name will populate the list of values for that property with values derived from any input view it finds at the time. Here you will be able to choose between creating a new view. Click the Output tab. Select the Copy an Existing view option and then click Next. This restriction applies to properties which reference input or output view names or field names. and click Next on the View Name pop-up to start the view creation wizard. Click the Properties tab. When you place your mouse over a view.Exercise 2: Filtering Data 3. 11. 4. You can also CTRL+Drag items within the tree to create duplicates. The duplicated Leads Component appears in the Library. Click OK. pop-up. the instructions in this Hands-on section assume that you duplicate the Leads Component.19 - . The Filter is now defined. Click OK. From the Save As. Duplicating Components Duplicating a Reader Component The Leads Component created in your Exercise 1 folder is the same as what we need for a Reader in this exercise. specify the URI SnapLogic/User/Exercise_ 2/Leads.. This step diverges from the sample Tutorial version of Exercise 2 in that there is no Leads Component duplicated in the sample version. pop-up. Double-click the Leads Component icon from Tutorial Exercise 1 to open it.csv file) we can leave its Properties. In the sample version of Exercise 2. 3. 4. 2. Basic Properties. or use the "Copy" command from the "Edit" menu. Since we have not defined any parameters for this Component. Output....SnapLogic® Tutorials 16. 2. Basic Properties. Duplicating a Writer Component The Prospects Component created in your Exercise 1 folder can be used as the Writer Component in this exercise. do not make any changes to this tab. and Parameters unchanged. you simply drag the Leads icon from the Exercise 1 folder and drop it on the Exercise 2 Pipeline canvas. Both methods are valid. From the Save Component As. 1. So let’s copy that Writer Component. However. and because we have used parameters for the output file (which we can change at runtime from ex_1_prospects. Since the Leads Reader will function exactly as it did in Exercise 1 (that is. 1.csv to ex_2_prospects. Skip the Parameters tab. Duplicating a Component allows you to modify a Component without affecting the pipelines that reference the original Component. Output. specify the URI SnapLogic/User/Exercise_2/Prospects.csv) we can leave its Properties. 4. Select Save As from the Save menu button. reading in the fields from the leads. Select Save As from the Save menu button. Since the Prospects Writer will function as it did in Exercise 1. 3. . and Parameters unchanged. Click the Prospects Component icon from Tutorial Exercise 1. So let’s duplicate that version of the Reader and use it here. Drag your Leads and Prospects Components onto the Pipeline canvas. Link the FilterLeads Component to the Prospects Writer.csv file://tutorial/data/ex_2_prospects. By default. 8. 7. and all of the fields will be mapped automatically because a matching field name was found for each To field. 4. 2. Drag the FilterLeads Component onto the Pipeline canvas. 11. Pipeline Parameter PARAM_000 PARAM_001 PARAM_002 PARAM_003 13. and FilterLeads Components in a new Pipeline to process the data. Click Save. apply the filtering criteria to pass on the records with CA as the State value and write them to the file specified in the Prospects Writer Component (ex_2_prospects. 12.DELIMITER Leads_1 . The Pipeline Parameters table is now populated with the parameters to be used in this Pipeline. Click Run Pipeline Now. some of the To fields will remain unmapped because the From field names do not match exactly. Click the New Pipeline button. Specify the URI as SnapLogic/User/Exercise_2/CA_Prospects. we could change the values right here on the Run tab and run it.INPUTFILE Prospects_1. Mapped to Leads_1 .Exercise 2: Filtering Data Building the Read-Write-Filter Pipeline Next we need to link the Leads. 6. Click the Run tab. and the delimiters (commas) for both Components. Default Value . Click OK when finished.DELIMITER . Specify the Reader input file. click the Parameters tab.csv .csv). Edit the parameters and change the default names and values as follows. Click OK when finished. 9. Prospects.outPUTFILE Prospects_1 . file://tutorial/data/leads. When the Link dialog appears. Click OK. Map the From fields to the remaining unmapped To fields. Link the Leads Reader to the FilterLeads Component. 1. 14. the Field Linker dialog appears. 3. From the Pipeline. 5. The Pipeline will read in the data specified in the Leads Reader Component. 15. 10. If we wanted to. Notice that the default values appear in the table. the Writer output file.20 - . Open ex_2_prospects.csv to verify the filtered results. You should see only the 675 leads that have California addresses.21 - . .SnapLogic® Tutorials 16. but this time.csv. income information is added. This data is optional. Your version of SnapLogic includes an example for Exercise 3 that illustrates all the steps you need to take to create the necessary Components and link them with the Pipeline.csv. This data is optional. Reviewing the Census Feed and Qualified California Pipelines From the Data Server icon. (for example. First let’s look at the Exercise 3 components. This file still includes all of the records from ex_2_prospects.csv. you will overwrite these files. Or you can simply re-run the sample pipelines to recreate the output files if you need to examine them.22 - . A Data Service Pipeline is one that provides a feed of data that can be used by other Pipelines or other applications. Next. This tab also provides descriptive information for this Writer Component. We will create a simple Data Service Pipeline that provides a feed of income data taken from the file CAIncomeByZip. we will introduce a Join Component that reads the income data from the Data Service feed and joins it (looks up) with the filtered data from the leads Leads and FilterLeads Components. This exercise introduces the concept of a Data Service Pipeline. It then writes the output to a file named ex_3_prospects.csv. question .csv). It also provides descriptive information for this Reader Component. l The Output Views tab defines the CSV file fields.4 Exercise 3: Data Services Exercise 3 starts with all of the California-based leads and adds information about income levels by zip code. To avoid this. Double-click Prospects and the following tabs are available: l The Properties tab defines the output CSV file to be created (written out) by specifying if it has a header row. what the delimiter is. Data from these pipelines can be presented in different Representations when used by other applications. data types. ex_2_prospects. The delimiter and filename are specified as $?{SOMENAME}. you should see the Ex3_CensusFeed and Ex3_Qual_CA_Prospects Pipelines. Double-click Census and the following tabs are available: l The Properties tab defines the file delimiter and output file name. and what the filename and path is. you can rename your exercise output files to be unique. l The Parameters tab defines the values of the file delimiter and output file name. When you run your exercises. ex_1_prospects.csv. Note: The Tutorial Pipelines generate the sample output files ex_n_prospects. and descriptions. The dollar sign. then we’ll go create them ourselves. l The Parameters tab defines any parameters used by the Component. zip and Input2: name. (Input1. It also provides optional descriptive information for this Writer Component. when the Pipeline processes associated Components). This matches the fields defined in the Reader Output tab. In this example the Join Component does not use any parameters. any ambiguous output fields must be aliased. In this exercise. That is what Aliases specify. The Aliases dialog is used to disambiguate the output view field names. and description) that are to be written to the output file.name == Input2. This data is optional. The Join Expressions dialog contains the list of expressions used to join the input views. company. the Pipeline gets its data from a CSV Reader Component that reads it from a local file. Since the input views that are being joined often have fields of the same name. company.name = Input2. . l The Parameters tab defines the values of the file delimiter and output file name. datatype. Click the following tabs: l The General Info tab provides descriptive information for the Pipeline.name) And an output: name. there is an implicit AND between them. zip. there are no input views. For example. l The Input tabs specify the fields (name. and braces indicate to the Component that the values for these properties are parameters that will be set at runtime (that is. l The Output tab specifies the fields (name. That is different from data that enters a Pipeline from one of its input views.SnapLogic® Tutorials mark. which is how this field will be represented in the output. and description) that are to be read from the leads. in order to distinguish these fields. consider a Join Component with l two input views: Input1: name. A fully-qualified input field is given an alias.name). These will be joined by name.23 - . l The Input tab specifies the fields (name.csv file. l The Input tab defines the structure of the data that upstream Components provide to the Pipeline.csv and CAIncomeByZip. Double-click the CensusFeed Pipeline to see what the Pipeline includes. Double-click Join and then click: l The Properties tab defines the join expressions and the field aliasing properties. datatype. and description) that are to be output from the Join Component. Join needs to know which of these names should be used for the output. A join expression takes the following logical form: (Input1. datatype. If there is more than one such expression. Since there is no upstream Component feeding this Pipeline. you see how the filtered fields from FilterLeads are mapped to the Join Component with the fields in Census. Click the link from Leads to FilterLeads to display the Link menu and select Field Linking to display the Field Linking page. Then these are matched to the FilterLeads fields by dragging Unmapped outgoing fields to the corresponding To panel such that From fields are on the same line as the corresponding To field names. l The Run tab contains the Run button which starts the Pipeline processing. left click the link from Join to Prospects. There is also a parameters table that lets you specify parameters at runtime that might be different from the default values indicated on the Parameters tab. this Pipeline has a CSV Writer Component that writes the data to a local file. To understand how Join combines the FilterLeads and Census fields. it has no output views.Exercise 3: Data Services l The Output tab defines data exiting the Pipeline. like the CensusFeed Pipeline above. When you click the link from FilterLeads (the Filter Component) to Join (the Join Component) and select Field Linking to display the Field Linking page. In this exercise.24 - . l The Pipeline tab shows how the Components are linked. The CensusFeed Pipeline is directing the output from the Census Component's output view Output001 to the Pipeline's output view Output1. This data is optional. l The Parameters tab allows you to define the Pipeline parameters and map the parameters to the Pipeline's Components. l The Output tab defines data exiting the Pipeline. Select Field Linking to display the Field Linking page. l The Run tab contains the Run button which starts the Pipeline processing. Double-click the Qual_CA_Prospects Pipeline to see what the Pipeline includes. l The Parameters tab allows you to define the Pipeline parameters and map the parameters to the Pipeline's Components. Click the following tabs: l The General Info tab provides descriptive information for the Pipeline. l The Input tab. The fields coming in from the Leads are mapped here to fields in the FilterLeads by clicking on the names in the Incoming fields (available to map) panel and dragging them to the From panel. l The Pipeline tab shows the single Component. There is also a parameters table that lets you specify parameters at runtime that might be different from the default values indicated on the Parameters tab. Since no data is exiting the Pipeline directly. The fields coming in from CensusFeed (the Data Service Pipeline) are mapped here to fields in Join. Click the link from CensusFeed to Join and select Field Linking to display the Field Linking page. . defines no input views. which creates the ex_3_prospects. the Pipeline processes the data from the Leads and Census Components. Now let’s create the Components in Exercise 3. The fields are: l Zip l LessThan10K l MoreThan10K l MoreThan25K l MoreThan50K . Name the URI SnapLogic/User/Exercise_3/Census. 6. then join the two data sources to write out the desired file. 5. 2. The Writer Component in this exercise is very similar to the Writer Component we created in Exercise 2. Accept the default name and create the view based on a blank view. the input view of the Prospects Component has been changed to include the new field: MoreThan50K. We reuse the filtered data from Exercise 2. The datafile contains a header row that we want to skip. Set Skip lines to 1. 4. 7. Click the Parameters tab and confirm that the DELIMITER and INPUTFILE parameters are listed. Leave the other property settings unchanged. 11.csv and assign meaningful names. The output view for the Census Reader Component requires that you define the five columns in CAIncomebyZip. 9. Set the Delimiter to $?{DELIMITER}. Creating a Data Service Component 1. 3.csv file.25 - . so we can duplicate it to save time. Hands-On Exercise: Data Services This exercise uses two Reader Components to read data from two different source files. we use parameters that can be defined at runtime. Click the Properties tab. Click Save. 10. Select the Output tab. create a new Reader Component to read in data from another source file. Double-click the CSV Reader Component in the Component Library pane. click the Add Output. and sends the data to the Prospects Component. Set the File name to $?{INPUTFILE}. Rather than explicitly defining the delimiter and the input file name. When you click Run. 12. Click Save. Click OK.SnapLogic® Tutorials Notice that for this exercise. 8. 21. 16. . In the URI field.Output1. The CensusFeed Pipeline tab should open. Click Add Row five times and assign field names and data types as follows. Click the Run tab. then click Finish. Name the Pipeline output view Output001 and map it to Census. 22. 18. Click the New Pipeline button. l Pipeline Parameter: CENSUS l Mapped To: Census. Click Save. you can see that the Pipeline has an output view available to read from. 23.Exercise 3: Data Services 13.INPUTFILE l Default Value: file://tutorial/data/CAIncomeByZip. In the Output tab. Click Save. The Census Reader is now defined. In the Run sub-tab. Click Pipeline Properties. Click Save. Click Save. 17. Next we will create the Data Service Pipeline incorporating the Census Component. 20. enter SnapLogic/User/Exercise_3/CensusFeed. the data type for Zip must be string for this exercise to work correctly. 15. 24. naming it Census. The remaining fields must be number types. Field Name Zip LessThan10K MoreThan10K MoreThan25K MoreThan50K Type string number number number number Zip Code Income less than $10K per year Income between $10K and $25K Income between $25K and $50K Income greater than $50K Description For the purposes of this example.26 - .csv 25. click the Add button. 19. 26.DELIMITER l Default Value: . Drag the Census Component onto the Pipeline canvas. Click the Parameters tab and edit the parameter definitions to match the following: l Pipeline Parameter: INPUT_DELIMITER l Mapped To: Census. Click OK. 14. State. Double-click the Join Component in the Foundry pane. 5. copying the output view from your CensusFeed Pipeline. Open a new browser window and enter the URI as follows: http://servername:8088/feed/SnapLogic/User/Exercise_3/CensusFeed/Output001?sn. The browser's HTTP request will start the CensusFeed Pipeline and return the output view data from Output001 in a simple HTML representation.content_type=text/html where servername:8088 is replaced with the name and port of your SnapLogic Data Server. Output001. This will run the pipeline and send the Pipeline's output to the Preview sub-tab. Click Add row. City. Click the Preview Data sub-tab to see the data output by the data service. LessThan10K. You can also switch to the General tab and click on the Component hyperlink to access a list of available feeds. 9. 3. Click Save. For more information on available representations.SnapLogic® Tutorials 27. and Zip defined. Repeat these steps to create an Input2. MoreThan25K. 6. 2. MoreThan10K. Address. 7. Click Next to use the View Creation Wizard to copy the output view (Output1) from your FilterLeads Component from Exercise 2. Input2 matches the output view of the Census Reader Resource. Click the Preview button. 8. Last. Select the Output tab and copy your Input1 input view and manually add the MoreThan50K field after the view has been copied. Name the URI SnapLogic/User/Exercise_3/Join and click OK. see the section entitled "Output Data Representation Formats" in the SnapLogic User Guide. Another way to demonstrate the utility of the Data Service Pipeline is to view it directly from another browser window. and MoreThan50K fields. 4. 28. The /feed prefix is necessary to indicate to the server that a GET to this URI will trigger Pipeline execution. Select the Input tab and click the Add Input button. The view name suffix is how you select which output is being requested. For your convenience. 10. Click View/Edit to set the Aliases settings. . Phone_w. the Designer will launch these in a new window when clicked. You should have defined Zip. Select the Properties tab. You should have the fields First. Creating a Join Component 1.27 - . dialog.Zip.Zip.. The Join Component is now defined. 4. select Input1. click the '''X''' button to the left of the row you wish to remove. Again. Fill in the author and description fields as desired and click Save. 1. select the URI SnapLogic/User/Exercise_3/Prospects. Because we copied this Component. There are no parameters defined for this Component. 15. 16. since we copied this Component. From the Save Resource As.Exercise 3: Data Services 11. 13. Click Save. select Input2. If you wish to delete a row from the output view. Enter the following values: l In the Fully Qualified Field column. 17. The field type for this field must be number. Select the Parameters tab. we need to add a row to add the MoreThan50K field.Zip. l In the Join Field 2 column. 14.. These fields are optional. select Input1. 19. 12. Duplicating a Writer Component The Prospects Component created in your Exercise 1 folder is close to what we need for a Writer in this exercise. Click OK. 5. Select the Input tab. Click Close. Select the General Info tab. Click View/Edit to set the Join Expressions settings. Enter the following values: l In the Join Field 1 column. Click Save. Click Close.csv. Double-click on the Prospects Component from Tutorial Exercise 1. so let’s make a copy of that version of the Writer and modify it here. 18. 3. Do not use the Delete button for editing as this deletes the entire Component.28 - . Click Add row. select Zip. 6. 2. Your Parameters tab should contain the following information. the values here are almost correct for our needs. We just want to change the default value for OUTPUTFILE to be file://tutorial/data/ex_3_prospects. so you can leave it unchanged. l In the Output Field column. Select Save As from the Save menu button. . The Prospects Component is now defined. so manually map any remaining unlinked fields by selecting the appropriate field names from the list of values to the left of each field. Name the Components as you drop them onto the canvas to match their URI names. manually select Phone_w from the list. All fields should be linked in the Field Linker dialog. Building a Data Service Pipeline Next we need to build a Pipeline with the two Reader Components linked to the Join Components. 8. 6. select the Parameters tab. Click OK. 5. Link CensusFeed also to Join. enter SnapLogic/User/Exercise_3/EX3_Qual_CA_Prospects. then click OK. which is in turn linked to the Writer Component to perform a join. The Field Linker dialog appears and all fields should be automatically linked. for the Work_Phone field.29 - . Default Value file://tutorial/data/ex_3_prospects. In the URI field. Click Save. 9. When complete the fields should be linked. Select the Input1 input view. 1. Drag the Leads Component from Exercise 1. 10. Link Join to Prospects. . From the Pipeline. 7. 11.csv 7. the FilterLeads Component from Exercise 2. Click the New Pipeline button. Again all fields should be automatically linked in the Field Linker dialog. Link FilterLeads to Join. Select the Input2 input view and click OK. 2. Auto linking was not able to match all fields. Click OK.SnapLogic® Tutorials Parameter Name DELIMITER OUTPUTFILE . Click on the purple output view of Leads and drag to link to FilterLeads. and the Join and Prospects Components and the CensusFeed Pipeline created in this exercise onto the Pipeline canvas. 12. 3. For example. 4. INPUTFILE CensusFeed. Prospects. . all the delimiters are commas.csv file://tutorial/data/CAIncomeByZip. Click Run button. If necessary.DELIMITER 14. 17. Pipeline Parameter LEADS CENSUS PROSPECTS Mapped to Leads.csv INPUT1_DELIM-Leads. refining it to yield just the work phone numbers for all the leads and a new column has been populated with census data for each Zip code.30 - .csv and verifying that the MoreThan50K field has been added. 16. Edit the parameter definitions as follows. 18. which takes the data in the original leads input file. INPUT2_ DELIMITER. We have accomplished the goal of creating a Data Service Pipeline and using it as part of another Pipeline.INPUT_ ITER DELIMITER OUTPUT_ DELIMITER Prospects.DELIMITER ITER INPUT2_DELIM-CensusFeed. Select the Run tab. 15. Confirmation that the Pipeline is starting is provided on the status line in the Runtime Information sub-tab.CENSUS Default Value file://tutorial/data/leads.Exercise 3: Data Services 13.000 a year. .) .csv . Note that we have distinguished the input delimiters as INPUT1_DELIMITER. Notice that the default values appear in the table. we could change the values here. (This new column shows the percentage of people in a Zip Code who make more than $50. Select the Run sub-tab. For this exercise.OUTPUTFILE file://tutorial/data/ex_3_prospects. The Pipeline Parameters table is now populated with the parameters to be used in this Pipeline. and OUTPUT_DELIMITER to give us the option of defining them uniquely should the need arise. Verify that the Pipeline performed this task by opening the output file ex_3_prospects. Click Save. 31 - .SnapLogic® Tutorials . Let’s take a look at the Components used in this Pipeline. l The Properties tab is where you define the filter criteria used by this Component. These fields are optional. l The Parameters tab shows parameters used by this Component as well as their optional default values. These fields are optional. we filter for zip codes where at least 25% of households have incomes greater than $50. l The Input tab defines the fields consumed by this Component. Double-click the Ex4_Filtered_Qual_CA_Prospects Pipeline in the Library. l The Output tab specifies the fields output by this Component. l The Input tab defines the fields consumed by this Component. l The Output tab specifies the fields output by this Component. l The Properties tab is where you define the sorting criteria used by this Component. In this example. This will open the Pipeline in a new Canvas tab. l In these exercises. zip code is defined as a string datatype. we define a filter expression to check when the value of the MoreThan50K is greater than or equal to 0. we will be sorting based on the following fields: MoreThan50K (primary key). To do this.25. Zip_Code (secondary key).csv.000. you will see the Ex4_Filtered_Qual_CA_Prospects Pipeline.32 - . Double-click FilterProspects and then click: l The General Info tab provides descriptive information for this Component. you will see the FilterProspects and SortProspects Components. Reviewing the Filtered Qualifying California Prospects Pipeline In the Library.5 Exercise 4: Data Sorting Exercise 4 filters and sorts joined data based on zip codes where 25% or more of households have an income greater than $50. . then we’ll go create it ourselves. In this example. Double-click SortProspects. The results are written to a file named ex_4_prospects. In addition to Components used in previous exercises.000. l The General Info tab provides descriptive information for this Component. SnapLogic® Tutorials l The Parameters tab shows parameters used by this Component as well as their optional default values. These records exactly satisfy the criteria that the qualifying leads must live in Cal- .csv. FilterLeads. This displays the following field mappings: Now click the link between the FilterProspects Component and SortProspects Component. The Census. No Input Views are used by this Pipeline. l The Outputtab specifies the fields output by this Pipeline. You will see the following field mappings: Lastly. Click the link between the Join Component and FilterProspects Component. No Output Views are used by this Pipeline. and Join Components and used in this Pipeline and their field mappings are identical to how they are defined in Exercise 3. The following information focuses on what has been introduced in this exercise.33 - . This will show the following field mappings: Running this Pipeline results in list of 112 records written to a file named ex_4_prospects. Click the Pipeline Properties button. These fields are optional. click the link between the SortProspects Component and Prospects Component. l The Inputtab defines the fields consumed by this Pipeline. l The Parameters tab shows parameters used by this Component as well as their optional default values. l The General Info tab provides descriptive information for this Pipeline. Leads. In the Foundry. 7. type filter into the search text box. click the Parameters tab. Hands-on Exercise: Sorting Data This section explains how to build the Component and generate the output contained in the Tutorial/Exercise_4 folders. 4. 3. Click Finish. 3. Make sure that the Copy an existing view radio button is selected. 10. 14. Duplicating a Writer Component 1. Change the Default Value for the OUTPUTFILE parameter to ex_4_prospects. The Prospects Component is now defined for Exercise 4. 4. If you need additional information at any time during this hands-on portion of this exercise. Click Save. 6. In the new Prospects Component. let’s create this Pipeline from scratch. 2. Creating a Filter Component 1. 5. Click the Add Output button and copy the input view of the FilterProspects Component. Click the Properties tab and click the View/Edit link for Field Filters. 13. 8. Select the Output view of the Join Component created in Exercise 3. Duplicate (Save As) the Component as SnapLogic/User/Exercise_4/Prospects. Double-click the Filter Component Template. Click Next.csv. feel free to open and examine each of the Tutorial versions of these Components. Specify the relative URI as SnapLogic/User/Exercise_4/FilterProspects. Click the Input tab. 11. Click Next. 2. Click Save.Exercise 4: Data Sorting ifornia and reside in zip codes with at least 25% of households having an income greater than $50. Click the Output tab.000. Double-click the Prospects Component. Click Add Input to trigger the Add Input View Wizard. . 12. Now that you’ve had an opportunity to review the Components used in this exercise. Click Add Row. Open the Ex3_Qual_QA_Prospects Pipeline.34 - . Click OK. 5. 9. 6. Specify the Sort specs properties to be: l Sort field: MoreThan50K l Sort order: desc 8. Click Save. 6. 17.SnapLogic® Tutorials 15.35 - . connect to Join l Leads. type sort into the search text box. Click OK. 10. Click Save.25 l Comparison: >= l View: Output1 16. Select the Input tab and copy the input view of the Prospects Component. 1. Click Add Row. 7. Drag the required Components onto the Pipeline Canvas and link the Components as follows: l Census. Click the New Pipeline button . Click Add Row and specify the Sort spec properties to be: l Sort field: Zip_Cde l Sort order: asc 9. 5. 2. 4. 4. Building the Pipeline Next you can now link these Components together. Specify the URI as SnapLogic/User/Exercise_4/Ex4_Filtered_Qual_CA_Prospects. Select the Output tab and copy the input view of the SortProspects Component. connect to FilterLeads . 2. In the Foundry. Double-click the Sort Component Template and create a new Component with the relative URI to SnapLogic/User/Exercise_4/SortProspects. 3. 3. Click Close. Specify the Field Filters properties to be: l Field: MoreThan50K l Value: 0. Click the Properties tab and click the View/Edit link for Sort Specifications. Creating a Sort Component 1. Click Close. . all the delimiters are commas. connect to SortProspects l SortProspects. INPUT1_DELIMITER Leads. . 9. `INPUT2_DELIMITER`. We have distinguished the input delimiters as `INPUT1_DELIMITER`. Click Run. Define field mappings as described in "Reviewing the Filtered Qualifying California Prospects Pipeline" .csv file://tutorial/data/ex_4_prospects. connect to Prospects l Prospects 5. connect to Join l Join.36 - .DELIMITER INPUT2_DELIMITER Census.DELIMITER . For this exercise.INPUTFILE Prospects. 6.csv file://tutorial/data/CAIncomeByZip. these values could be changed right here before running the Pipeline.csv).csv file all have an percentage greater than 25% in the last column. connect to FilterProspects l FilterProspects. You can verify the results of this exercise by examining that 112 records contained in the ex_4_prospects.DELIMITER OUTPUT_DELIMITER Prospects.csv .OUTPUTFILE Default Value file://tutorial/data/leads. 7. The Pipeline will read the data specified in the Reader Census and Leads Components and write it to the file specified in the Prospects Component (ex_4_prospects. Click the Parameters tab and modify the Pipeline Parameter and Default fields follows: Pipeline Parameter LEADS CENSUS PROSPECTS Mapped to Leads. 8.INPUTFILE Census. If necessary. Notice that default values appear in the table. Click the Run tab.Exercise 4: Data Sorting l FilterLeads. and `OUTPUT_DELIMITER` to give us the option of defining them uniquely should the need arise. The Pipeline Parameters table is populated with the parameters to be used by this Pipeline. Click Save. SnapLogic® Tutorials .37 - . passes it through the Filter and into the Writer input. Again notice that the only field defined is the State field but also notice that the box for view Input1 is checked in the Pass-through Input Views section. (for example.csv. . This enables passing-through all the fields that aren't linked to a field of Input1. Click on: l The Input tab. you can rename your exercise output files to be unique. When you click the Run Pipeline button.csv file. State is the only column that it really needs as input. When you run your exercises. l The Output tab. FilterLeads only requires a single field. Reviewing the California Prospects Pipeline. The Pipeline is identical to the one in Exercise 2 with one exception. the Pipeline processes the Reader output. not only is the State field available but all the fields that were available to the input view of FilterLeads. See the Pass-through section in the User Guide for more information.csv.38 - .6 Exercise 5: Utilizing Pass-Through Exercise 5 is based on Exercise 2 and introduces you to the concept of pass-through. To avoid this. This Component is almost identical to the one used in Exercise 2. the State field. Note: The Tutorial Pipelines generate the sample output files ex_n_prospects. Exercise 5 From the Data Server icon (<my_machine_name>:8088). Click anywhere on the link between the FilterLeads Resource to the Prospects Resource to display the Link menu and select Field Linking to display the Field Linker page. Since this filter Component is applying a predicate to the State column. ex_2_prospects. The remaining fields "pass-through" the Component and are available for linking downstream. except for the way the input and output views are configured. then creates the ex_5_prospects. Now let’s create the Resources in Exercise 5. Back on the Ex5_CA_Prospects Pipeline. Double click FilterLeads. Since pass-through has been enabled in FilterLeads. In this exercise. you will overwrite these files.csv). double-click the Ex5_CA_Prospects folder. You should see that the output result contains 675 records. Notice that only the State field is defined. Or you can simply re-run the sample pipelines to recreate the output files if you need to examine them. click the link between the Leads Resource to the FilterLeads Resource to display the Field Linker page. ex_1_prospects. Components that support pass-through allow you to map only the fields required by the Component. Click the Properties tab for the FilterLeads Component. From the Save Resource As dialog. 9. 13. Double-click the Leads Resource icon from Tutorial Exercise 2 to open it. . Click OK. The datatype should be string and add a field description. 7. then add the row: l Field: Name of the CSV file column is State l Operator: == l Value: The value to filter on is CA l View: Output1 15.SnapLogic® Tutorials Hands-on Exercise: Implementing Pass-Through Creating a Filter Component with Pass-Through 1. 6. 2. Click Save.39 - . click the Add View button. 16. Click Save. Click the Output tab. 14. Double-click the Filter Component in the Component Library pane. Click Save. 8. 3. Check the box next to Input1 to enable pass-through on this output view. Click on View/Edit link of Field Filters. Duplicating the Reader and Writer Resources Duplicate the Leads and Prospects Resources created in your Exercise 2 folder. 11. Select Save As from the Save menu button drop-down list. or use the copy view wizard to copy the definition of Input1. 4. specify the URI as SnapLogic/User/Exercise_ 5/Leads. The Filter is now defined. Specify the properties exactly as you did in exercise 2. we will select the new blank view option and click Finish. Click Add Row to add a single row. 12. 2. Specify the URI as SnapLogic/User/Exercise_5/FilterLeads. click Next where the input can be copied from an existing view or can be created as a new view. Name the field State. 10. Since our Component only needs one field. 1. Click Close. Select the Input tab. 5. Repeat the steps you performed to define the input view Input1 to create the output view Output1. 3. reading in the fields from the leads. Click Apply and Save when finished.csv file) we can leave its Properties. 11. Click the Run Pipeline from the Pipeline menu. 5.csv and verify the filtered results. Output view. Click OK. Edit the parameters to change the default names. The duplicated Leads Resource appears in SnapLogic > User > Exercise_5. and Parameters unchanged. Click Save. Drag the FilterLeads Component onto the Pipeline canvas. Building the Pipeline Now we can link the Leads. Perform the same steps as above for duplicating the Prospects resource from Exercise 2. Open ex_5_prospects.40 - . Note that _1 is appended to the Component name. Link the Leads Reader to the FilterLeads Resource. 10.Exercise 5: Utilizing Pass-Through 4. The only difference is that the fields unneeded by the Filter resource are passed through to the "downstream" resource. 1. Specify the URI in the Save As dialog to be SnapLogic/User/Exercise_ 5/Prospects. 8. From the Pipeline Properties. the Field Linker dialog appears. 4. Name them Leads and Prospects respectively as you drop them onto the canvas. Rename the Component FilterLeads. 5. 9. 7. select the Pipeline Parameters tab. 2. 12. Click OK. 13. and FilterLeads Resources together in a Pipeline just as in Exercise 2 to process the data. Specify the URI as SnapLogic/User/Exercise_5/CA_Prospects. Prospects. Since the Leads Reader will function exactly as it did in Exercise 2 (that is. Click Apply and Save when finished. You should see only the 675 leads that have California addresses. 15. Drag your Leads and Prospects Resources onto the Pipeline canvas. Link the FilterLeads Resource to the Prospects Writer. and the State fields will be mapped automatically because a matching field name was found. 3. 6. The Pipeline will execute exactly as it did in Exercise 2. Click the New Pipeline button. By default. similar to the output from Exercise 2. . 14. SnapLogic® Tutorials .41 - . 35.42 - . 20. 29. 28. 16. 22. 38 P Pipelines building 12. 34 filter with pass-through 39 join 27 sort 35 writer 11 duplicating 19.Index C Components creating 8 data service 25 filter 17. 39 E exercises overview 6. 32. 34. 40 T tutorials environement 4 overview 5 . SnapLogic® Tutorials .43 - .