HANA Integration

1 HANA IntegrationHANA (High Performance Analytical Appliance) is a product based on New DB that allows the implementation of analytical application that processes a huge amount of data in real time. The Gateway HANA integration exposes HANA content as OData services. For this a generic Minimal Gateway application has been developed on the Gateway Hub, i.e. a local MGW application. The application is a prototype only. Additionally references between other Gateway content and HANA content are possible. Therefore the prototype allows federation between HANA content and Minimal Gateway Content. Gateway retrieves the data from HANA via “ABAP Database Connectivity” (ADBC). ADBC is an object-based ABAP API for programming relational database accesses, which in its class and model structure follows the JDBC (Java Database Connectivity) diction. For some of the following discussion the different table types in HANA should be known • • Replicated tables like for example the MM material table MARA Analytic Views are like cube model where Transaction Data is connected to attribute view o o • They can have multiple dimensions Data can be sliced and diced (e.g. Grouped by attributes) Calculation View are based on analytic views enriched with custom functions and calculations o They can be used to define a two dimensional list from a Analytic View 1.1 Prototype A prototype has been developed in system U3F. Architecture Concept for <Program/Project/Topic/Integration Scenario> 1. It is developed on the Gateway hub against interfaces not exposed to stakeholders. The Business Data Provider supports the common set of SAP Data query functionality (e. sorting).3 HANA Business Data Provider The HANA Business Data Provider delegates requests to HANA via the HANA Database Connector.1.1.g. The main building blocks are • • • "HANA Model Provider" (Class ZHANA_CL_HANA_MODEL_ADP) "HANA Business Data Provider" (Class ZHANA_CL_HANA_MGW_RUNTIME) “HANA Database Connector” (Class ZHANA_CL_HANA_DATABASE_READER) 1.1 Overview Figure 1 .Gateway HANA Integration The HANA integration is based on the SAP Data Channel (Minimal Gateway) for local applications. © Copyright SAP AG 2010 Internal/Confidential Page 2 of 11 . It implements BADI /IWFND/BD_MED_MDL_MODEL_LOAD and is triggered via the BADI filter EXTERNAL_MODEL_TYPE.1. Only read scenarios are supported. It implements BADI /IWFND/BD_MGW_RUNTIME and is triggered via the BADI filter SERVICE_DOCUMENT_NAME.2 HANA Model Provider The HANA Model Provider extracts metadata out the HANA repository via the HANA Database Connector and adapts it to the SAP Data Channel representation. filter. 1. 1. 1.5 Gateway Objects Several Gateway objects need to be created during design time: © Copyright SAP AG 2010 Internal/Confidential Page 3 of 11 .4.1 Connection to HANA To execute queries against HANA class CL_SQL_STATEMENT of package SDB_ADBC (part of BASIS) is used.1.4 HANA Database Connector The HANA Database Connector executes queries against the HANA database and provides the corresponding metadata.Architecture Concept for <Program/Project/Topic/Integration Scenario> 1.3 Metadata The metadata (entity description) is derived from the corresponding HANA metadata.1.1. These authority checks are filters that are merged to an executed query. However this requires user based connectivity between Gateway and HANA.2 Additional query information To execute the queries additional information is needed during runtime.4. More precisely it is read from a query result. 1. 1. The corresponding connection has been defined in table DBCON. A query requesting only one entry from the database is executed and the metadata of the result set is read.1. This information is merged with the data stored in table ZHANA_DBTAB_NAME and then adapted to the corresponding MGW format. This is stored in table ZHANA_DBTAB_NAME. 1. This is currently not available. This table has the following fields • • • • • • • MODEL_IDENTIFIER SCHEMA_NAME TABLE_NAME CON_NAME IS_ACTIVE COLUMNS_USED KEY_FIELDS MGW Model Identifier (key) HANA Database Schema HANA Database (View) name Logical name for a database connection Indicates that the GSDO Group is Active Names of the columns used from the HANA database Names of the columns that are key fields 1.4.4.5 Entities To retrieve an entity an ADBC query statement is composed and executed requesting only one entry. HANA supports user based authority checks.1.1.4 Feeds To retrieve a feed an ADBC query statement matching the OData request is composed and executed. 1.4. However this requires user based connectivity between Gateway and HANA.3 Scalability Only a limited fixed numbers (20?) of ADBC connections can be open in parallel from a SAP system.1 Limitations / Requirements The following limitations and requirements have been identified.Architecture Concept for <Program/Project/Topic/Integration Scenario> • • One Service Group One Object Model per HANA database table.2.1. Currently no connection pooling is available.2 Tools Connections are created by creating an entry in table DBCON via SE16.2.1.2.1 ADCB ADCB is the one big “question mark” for this integration approach. These authority checks are filters that are merged to an executed query.1.6 HANA specific meta-data on the Gateway No additional content specific metadata shall be maintained at Gateway to avoid Lifecycle dependencies between HANA content and Gateway content.2 Conclusions 1. 1. This is currently not available.1.2. 1. © Copyright SAP AG 2010 Internal/Confidential Page 4 of 11 . 1.1. 1. As the HANA metadata does currently not support semantic groups of entities that could be translated to SAP Data Service Groups.1.1. 1. The following tables need to be maintained o /IWFND/I_MED_OHD  o Defines the model identifier and set active /IWFND/I_MED_EMM  Specifies the external model identifier (used at metadata document) o • /IWFND/I_MED_INH  Object inheritance information All models are assigned to the same Service Group. additions to the current HANA repository are required: • • SAP Data Service Groups Language Dependent Column Labels 1.1.2. This is done via table o /IWFND/I_MED_SRG  Maps models to service groups • While the routing is not done via System Aliases still a GSDO Group is needed to which the Service Group is assigned.1 Authorization HANA supports user based authority checks. 1.4 SAP Server Operation System The Gateway retrieves the data from HANA via ADBC. 1. But as there are limitations on the MGW property naming (e.2. But even if that is possible no translation would be available. but not for example Solaris for ADBC.4 Labels The metadata retrieved from HANA does not contain label information.1. 1.1 • • Features needed on Gateway A method to calculate a MGW (OData) compatible name from a String A possibility for an admin to enhance the generically created “transient” model 1.3.6 • Features needed from BASIS The limitations listed above need to be overcome o o o o User based authorization Tool for connection setup ADBC scalability Supported operation systems (This is rather a limitation of the new DB database driver than ADBC) 1.1.Architecture Concept for <Program/Project/Topic/Integration Scenario> 1. Hence MGW properties are called as the HANA database table columns.2. 1.2.3 Hiding (showing) of fields The MGW metadata is created generically from HANA metadata.2.1.1.2 Renaming of fields The MGW metadata is created generically from HANA metadata.g. All other fields are not exposed 1.1.1 • Features needed on Gateway A possibility for an admin to enhance the generically created “transient” model © Copyright SAP AG 2010 Internal/Confidential Page 5 of 11 .2.2.1.1.2.1. Hence all HANA database table columns are exposed as Gateway properties.2.1.1 • Features needed on Gateway A possibility for an admin to enhance the generically created “transient” model o o Hide certain fields Define a “white-list” of fields to be exposed.2. It might be possible to read a descriptive XML from a HANA database table which contains label information.5 Future (To the author of this document) The future support and development of ADBC is not clear.4. HANA currently only supports Linux (and Windows?). no ‘/’ is allowed) certain columns cannot be used.1. 1.2. Hence the Gateway ABAP server needs to be running on Linux (or Windows?).1. 1. 1.1 • Workaround Two workarounds seem reasonable: The prototype uses database table ZHANA_DBTAB_NAME on the Gateway to store for each model (i. o o No library is available retrieve this metadata. • 1. Measures (Key figures) are numeric values or quantities.2. No field labels are available There is one specific table which contains a metadata XML tables and views. Corresponding problems are very likely. o o • This is only filled for replicated tables.2.1.1.2.2. Sales Representative ID.7 HANA Metadata There are currently 3 options how to retrieve metadata for a HANA database table • There is one specific table which contains column information for tables. such as Per Unit Sales Price.5 Key Fields HANA database tables do not support key fields. and Sales Revenue. each HANA database tables) the fields which shall be exposed as key fields Flag all characteristics as key.e. Unit of Measure.6 READ Entity HANA database tables do not support key fields. such as Customer ID. It needs to be checked if field labels are available • The ADBC libraries return for every Query result also the corresponding metadata.Architecture Concept for <Program/Project/Topic/Integration Scenario> 1.1.5. If the correct characteristics are supplied this is sufficient.6. A Gateway application would have to parse this XML itself. No information can be read for analytic and calculation views.1.8 Field length in ADBC libraries The ADBC libraries can only handle field (column) names with a length up to 30 characters. 1.1. Material Number. and Transaction Date. o o Attributes (Characteristics) are descriptions of key figures.2. © Copyright SAP AG 2010 Internal/Confidential Page 6 of 11 .1 • Workaround To retrieve an entity an ADBC query statement is composed and executed requesting only one entry. 1. Quantity Sold.1.2. o o o No field labels are available It needs to be checked if this returns metadata if the query did not find any record This can be fairly slow way when only the metadata is needed 1. Hence it is not possible to uniquely identify a table entry for a select single. HANA allows longer field names. 1.2.Architecture Concept for <Program/Project/Topic/Integration Scenario> 1.11 External Model Type Not surprisingly the Gateway has no External Model Type for HANA. 1.1 Features needed on Gateway • • • It should be possible to assign different models from different sources to one Service Group.1. To achieve this a new Model Provider class was needed.2. 1.1. For the prototype the value “B” was used as it was available. i.1 Features needed on Gateway • External Model Type should not be based on a domain but rather be customizing with a customer namespace. HANA as well as Enterprise Search or Analytics (BW) will most likely host hundreds of potential data sources. It should be possible to define associations (links) between models on a Service Group level. © Copyright SAP AG 2010 Internal/Confidential Page 7 of 11 .9 Dynamic Object Model assignment It is not possible on the Gateway to assign Object Models to Service Groups during runtime.2.1. It should be checked. 1. Once these can be loaded into Gateway dynamically some selection / registration mechanism is needed allowing an administrator to expose any of these data sources individually.2. 1. A related requirement for many generic Integration scenarios would be some kind of “WhiteList”. This class inherits from the original HANA Model Provider class to access the coding describing the Revenue Entity. within one Service Group (and for that Service Group only).e. Models are assigned to Service Groups during design time via SE80 (or SE16 as in the prototype).10 Composition The following iPhone application has been built for a Sapphire demo based on the prototype • • Search for customers (Entity Customer) o Based on a specific customer MGW application Display for a selected customer a list of the 5 products with the highest revenue (Entity Revenue) o Based on a model of the generic HANA application • Display some detail information for a selected product (Entity Product) o Based on a specific product MGW application To link the HANA model to the Product Details the generically generated Object Model description needs to be enriched with the association information (define the link).2.11.2. A similar concept was needed for the Business Data Provider. For a productive coding a specific model type is needed. 1.1. Specific coding describes the two other entities. if it is needed to allow the definition of associations (links) between models on a model level.9.10.1.1 • • Features needed on Gateway It should be possible to assign Object Models to Service Groups during runtime via APIs. 2 Other ideas Following is a list of some ideas collected before the prototype was designed • Use HANA as accelerator for QUERY operation on GW objects (for that matter the HANA object would need to have similar or identical structure compared to the GW object) o o o o This could be done in case the models are identical. o It needs to be checked if this is supported by OData or can be added This could be added by the Gateway.Architecture Concept for <Program/Project/Topic/Integration Scenario> 1. The idea of the prototype is to only provide a generic framework allowing a customer to easily expose HANA database tables which have   o Been shipped with a different life cycle Have been created by the customer But this merging of HANA and none HANA data is not trivial and hence rather a task for SAP development.2. 1. which of course is not needed for Gateway caching. An OData response and realistically also a mobile device can only handle 2 dimensional lists.12 Analytic Views An analytic view describes data in multiple dimensions. It would need to be checked if and how this could be developed and shipped • Use HANA as provider to particular GW object's properties (HANA can deliver KPI information based on analytical information to a Gateway object) o The idea of the prototype is to only provide a generic framework allowing a customer to easily expose HANA database tables which have   o Been shipped with a different life cycle Have been created by the customer But this merging of HANA and none HANA data is not trivial and hence rather a task for SAP development. o This has been done for the prototype. o © Copyright SAP AG 2010 Internal/Confidential Page 8 of 11 . HANA has been designed for multi dimensional mass data analysis. It would need to be checked if and how this could be developed and shipped • Use HANA (or more precisely the underlying newDB) may serve as offlining capability for GW caching o Simple two dimensional data (which is all what is needed for any reasonable Gateway caching) can be read from normal SAP database tables with at least similar speed as from a HANA database. o This would need some additional configuration on the Gateway This can be overcome by defining corresponding simplified Calculation Views on HANA. In case models are not identical this needs a rather extensive application logic or a even more complex mapping The HANA Business Data Provider would be made available for specific MGW applications.2. • • • This could be provided from the client.1. To retrieve such a list from an analytic view grouping information needs to be added to the SQL (ADBC) statement. This allows us to deliver the integration with a later Gateway content pack as well. (Decision pending) 1. 1. The current questions around ADBC are the one major obstacles to develop a Gateway quality product based on the ideas of the prototype Developing a generic tool to retrieve a SQL statement for an OData request would be a major part but can be accomplished fairly easy. An alternative would be to have one model containing all exposed tables as entities. For every exposed HANA database table one model has been defined.g. it should be possible to create and assign them to Service Groups during runtime) Composition and configuration © Copyright SAP AG 2010 Internal/Confidential Page 9 of 11 . 1.1 Gateway Software Component Layer Assignment The current analytics functionality is a part of IW_CBS.3 Design aspects • • One service group for all exposed HANA database tables has been created on the Gateway.2. should this be a repository object?) The definition of Object Models (E.2. The major lesson learned from the prototype for the Gateway is that following aspects should become more flexible or should be double checked: o o o The use of Service Groups (e. Consequently the integration of High Performance Analytical Appliance should be assigned to the same layer.Architecture Concept for <Program/Project/Topic/Integration Scenario> • Use HANA ideally also exposes itself via SAPDATA to allow homogenous access to all data sources of SAP and allow a consistent way for GW to access the information accordingly were needed/reasonable o The Prototype and this document are about the question how to expose HANA data via the Gateway.g.3 Summary • • • • From a design perspective the generic integration of HANA as described in the document is a very good fit for the MGW (OData Channel). The question what could be done on HANA has not been addressed.3. This is needed for example for the authority check. wdf.corp:50050/sap/opu/sdata/sap/ZHANA/MARACollection(mandt='800' .5.: ZHANA_BI_HANA_MGW_RUNTIME o MGW Adapter providing the runtime data(O-Data requests) ZHANA_CL_HANA_DATABASE_READER o Retrieves data and meta-data from HANA database © Copyright SAP AG 2010 Internal/Confidential Page 10 of 11 .5 will support Analytics.corp:50050/sap/opu/sdata/sap/ZHANA/MARACollection?sapclient=100&$format=xml&$filter=substringof(matnr.1.matnr='APO-FG')?sap-client=100&$format=xml Products per customer 1.5.wdf.sap.wdf. HANA 1.sap. So “with this one stone 2 birds would be killed.1. Instead of accessing HANA directly via ADBC rather integrate the Gateway with HANA implicitly by integrating with Analytics.: ZHANA_BI_HANA_MODEL_ADP o MWG Adapter providing the metadata (Model descriptions) ZHANA_CL_HANA_MGW_RUNTIME o Class for BAdI Impl.wdf.'AND') Entry http://ldai1u3f.5.werks='1000')/CopaCostingSheet?sap-client=100&$format=xml • 1.1.1 • Metadata http://ldai1u3f.3 • Feed http://ldai1u3f.matkl='002'.2 • • • Classes in U3F(100) ZHANA_CL_HANA_MODEL_ADP o Class for BAdI Impl.1.corp:50050/sap/opu/sdata/sap/ZHANA/COPA__ACT_PLCollec tion?sap-client=100&$format=xml&$filter=gjahr%20eq%202010%20and%20kunnr%20eq %20'0000002140'%20&%20&$orderby=revenue%20desc Costing Sheet http://ldai1u3f.1.1.2 • • Feed http://ldai1u3f.5 Appendix 1.5.5.1 Links to Prototype 1.matnr='M08'.sap.1.1 Links • Hana Input 1.4 Alternative There is at least one reasonable alternative architectural approach to integrate HANA.1. Advantages • • • ADBC will not be used Tool support like for example the Query Designer Analytics shall be integrated into Gateway anyway.sap.” 1.sap.kunnr='0000001360'.corp:50050/sap/opu/sdata/sap/ZHANA/$metadata?sapclient=100&$format=xml MARA 1.Architecture Concept for <Program/Project/Topic/Integration Scenario> 1.5.corp:50050/sap/opu/sdata/sap/ZHANAEXT/COPA__ACT_PLCollecti on(gjahr='1996'.wdf. 2 Gateway Objects • • Service Group ZHANA_SG_ALL_MODELS_0001 o Generic Hana Service Group .Architecture Concept for <Program/Project/Topic/Integration Scenario> 1.GW / Hana Integration Prototyping Object Models o ZHANA_OM_MARA_0001_BE o Creation / Assigment for new models  /IWFND/I_MED_OHD  Define the model indentifier and set active  /IWFND/I_MED_EMM  Specify the external model identifier (used at metadata document)  /IWFND/I_MED_INH  Object inheritance table not require -> dummy record  BASE_MDL_IDENT = space  /IWFND/I_MED_SRG  Maps service to models  SRV_IDENTIFIER = ZHANA_SG_ALL_MODELS_0001  ZHANA_DBTAB_NAME  map model id to hana schema and table name GSDO Group o ZHANA_ALL_OBJECT • © Copyright SAP AG 2010 Internal/Confidential Page 11 of 11 .5.