This blog is guest authored by Mark Goodwin, a regular follower of VL’s blog based in the UK. Virtual Logistics Inc. connected with Mark at the 2015 UK eCommerce Expo where we had an invigorating discussion comparing data integration in North America and in the UK. Mark’s industry expertise, technical knowledge, and unique perspective on the UK market led VL to ask Mark to write a guest blog inspired by our discussions. You can also learn more about Mark Goodwin at the end of this article.
A European Insight: Bridging the divide between B2B and B2C with Omni-Channel Data Integration
With increasing cross-channel development and e-commerce maturity across geographical markets, we have traversed an interesting juncture, the point of convergence’ between ‘traditional Business-to-Business (B2B) operations and the ever evolving Business-to-Consumer (B2C) requirements.
It is at this ‘point of convergence’ that a well executed Omni-Channel data integration strategy can bridge the divide between digital/web channels, enterprise systems, legacy systems and trading partner networks.
What is Omni-Channel Data Integration
Omni-Channel Data Integration is the ‘bi-directional’ integration of B2C (Digital/Web Channels, Storefronts, Marketplaces, Social) with B2B (Business Systems, Logistic Operations and Trading Partner Networks) across various touch points.
By employing Omni-Channel integration practices, businesses can focus on ‘fusing’ B2B and B2C operations in a scalable and adaptive manner for greater business agility in this competitive landscape.
Businesses at the ‘Point of Convergence’
Three distinct types of business are reaching this ‘point of convergence’ within the UK and Europe. They are coming at it from opposing sides of the spectrum:
- Purely web-based, now requiring Trading Partner Integration with Wholesalers, Manufactures and other types of Trading Partners.
- Traditional businesses with established Trading Partner Networks wanting to harness the power of Digital/Web Channels and Marketplaces.
- Business with existing Web Operations and Trading Partner Networks that require replacement and/or re-engineering due to antiquated technology and/or inefficient processes.
Moving ‘bi-directionally’ between B2B and B2C
Businesses requiring bi-directional integration between their B2B and B2C operations whilst maintaining business as usual (BAU) practices are approaching integrations in a number of different ways with mixed results due to technical, operational and resource complexities.
Some of the typical complexities faced are best summarised as:
Moving Between Data Standards (Non-Standards and Standards)
In the B2C arena there are ‘no set standards’ but rather a selection of key data elements that are ‘common’ across most platforms and systems. On the flip side in B2B things are heavily standards driven with the use of common message types ranging from but not limited to ANSI X12, EDIFACT, TRADACOMS, ebXML with various directories and subsets driving the standards.
Successfully moving bi-directionally between standards and non-standards can require high degree of skill to meet all the mandatory and optional/mandatory data requirements for full end-to-end compliance.
API’s – The Good, The Bad and The Plain Ugly
To access the vast majority of B2C data and some B2B data stored within various systems/ platforms, the API (Application Programming interface) will have to be accessed.
However no two API’s are identical, each having different requests/responses, data formats, data structures, authentication methods and endpoints. Each API now requires a change in programming ‘work-flow’ as each platform’s access is unique.
This now translates into the differences between a good or bad API as the design and level of complexity (authentication, message structures, data formats, call structure, call limits, endpoints, etc.) will have a direct impact on how easy it is to access.
The greater the level of its complexity, the greater the ‘resource’ requirements (skill, time and cost).
An example of a bad (painful) ‘API’ is authentication via OAuth 1.0a (Three Legged) with multiple endpoints requiring heavily modified calls for additional meta-data requests and then having to merge response data into intermediary files for further processing.
So although APIs are in most cases cleaner, they are intensive to integrate to.
Data Validation and Enrichment
When accessing data via API from Platforms, Systems and Marketplaces, validation is key.
Validation on ‘what’s been retrieved’ is now the highest priority prior to any form of transformation and enrichment. As the old adage goes ‘bad data in bad data out’ holds very true.
If partial data or ‘bad data’ is retrieved it needs to be ‘flagged’ and handled in a manner appropriate to its level of importance and in-line with preset operational handling procedures. The audit trail becomes key.
However when data has been validated, the role of enrichment now comes into play.
Enrichment when skillfully used in conjunction with ‘Data Transformation’ can take the ‘raw’ data and make it ‘purposeful’ and ‘meaningful’ for the target system, application and/or intended trading partner.
The level of skill and competence that the individual or team undertaking the data integration work has will have a direct bearing on whether it is a successful integration or not.
A deep understanding of integration ‘touch points’, formats and data structures, communication methods and prior experience of handling integrations across multiple systems, platforms and applications is critical.
As the majority of this work is perceived as integrating the web with back end business systems and operations, it is wrongly assumed by many UK and European companies that this work should be undertaken by Web Development Companies.
Unfortunately this practice is largely incorrect as the integration work is now handled by a web developer or programmer who’s main expertise is with a limited selection of web based platforms without the in-depth knowledge or experience required to deal with the complexities of bi-directional integration between web, enterprise systems, legacy platforms and trading partner networks.
These ‘custom’ integrations are typically ‘fixed’, ‘ridged’, and ‘not easily adaptable’ with limited logic handling – a far cry from an adaptive integration that can scale with future requirements.
So as this work straddles between both traditional B2Bi and Web Operations, knowledge of Data Mapping, EDI, Middleware, Messaging Standards, API’s, Web Services, Data Communications and systems such as ERP, OMS, WMS, Supply Chain Execution, TMS, CRM, Web Platforms and Transactional Websites is essential.
Custom business processes need to be ‘in-built’ with each integration. Specific business rules will be developed in-line with the businesses unique operational requirements.
Dependent on the size of the company and/or size of trading partner network, logistics infrastructure, this can be a complex and a time intensive task.
When dealing with a multitude of platform, systems, message formats, standards and trading partners there will be a prerequisite for well designed operational handling that factors in:
- Processing Methodology
- Business Rules
- Data Validation
- Recovery Logic
- Data Translation
- Data Enrichment
- Pattern Matching
- Partner Profiling
- Merging of Key related Files
- Communication’s Monitoring
- Error Handling
- Systems Monitoring
- Error & Network Logging
- Event Logging
- Code Conversions / Lookups
- File/ Message Archiving
Compliance will be dictated by the marketplace, hub or largest trading partner where specific MIG (Message Implementation Guidelines), API and Integration Guidelines will need to be adhered to.
Compliance requirements will differ for each marketplace, each hub and each trading partner with ‘mandatory’ and ‘optional/mandatory’ data requirements changing according tot heir business rules and logic.
Bridging the Disconnect with Omni-Channel Integration
In the UK and Europe there is currently a disconnect and much confusion over the real benefits omni-channel integrations verses more simplified point to point integrations and who are the best practitioners.
Data integration is still very fragmented between the Web and Traditional B2Bi. True Onmi-Channel Integrations bridge that divide by fusing the traditional with the ever evolving, allowing businesses the flexibility to adapt and scale no matter what the system, web/digital channel, communication protocol, data format or standard.
About Mark Goodwin
Mark has worked within the Supply Chain and Retail industry for over 15 years and heads up the Integration Practice at Graziashop, the Luxury Online Fashion Marketplace.
As Head of Integration, his remit spans Systems Integrations, Third Party Management, Partnership Development, Ecosystem Enablement through to Talent and Resource Management within B2B and B2C for High Fashion and Luxury Goods brands and retailers.
Prior to this Mark has worked with an EDI/ Data Transformation vendor developing SAAS Data Integration Solutions for Online Retailing and has also built, led and managed both Global and European Recruitment campaigns for numerous organizations focusing on Business to Business Integration (B2Bi), EAI, Middleware, EDI, Retail Merchandising and Supply Chain Management.
When he’s not involved with Omni-Channel Data Integration, you can find him researching new strategic, technological and commercial enablers to speed the processes of end-to-end integration. Mark is also an avid reader of VL’s blog.
Subscribe to VL’s Blog
Share this article!