EDI or Die – I thought it would be useful to look back at where EDI started and how things have changed over the years, so I reached out to Randy Smith and asked him to give me an EDI history lesson. Randy wrote SupplyTech’s STX12 which later became a Harbinger product. He also co-wrote the Pro_EDI translator, one of the first true EDI integration products. If you ever have the chance to meet and chew the fat with Randy do it. He’s one of a kind with a huge amount of experience in doing EDI, so you’ll learn a lot………
I’m an EDI guy. That’s all I do. I’ve watched this industry grow and mature. It’s been a fun trip and lately it seems to be coming around full circle.
The best definition of Electronic Data Interchange (EDI) that I ever heard in almost 28 years is “the exchange of business document data between business computer systems in a standardized format,” I heard that in 1984 and it is still spot on. I am not talking in this article about lower case “edi” – I’m talking about upper case “EDI”. Lower case “edi” is the exchange of data in any format between various platforms and environments. That could include putting files on a flash drive and loading them on another computer.
I’m talking about business transactions in a standardized ANSI or EDIFACT or XML format between ERP systems. No humans in the loop. I don’t’ get involved much in eBusiness or web retail and I will leave discussions about those to those who do.
I first learned to spell “EDI” back in 1984. The Detroit Tigers won the World Series that year, too. It turned out to be a good year for both of us. It was my first exposure to the methodology and arena I would work in for the rest of my professional career. I was coming off several successful projects in banking, oil and gas, defense, and aerospace as a programmer/analyst and took a job with a small start-up firm that was writing a PC based EDI translator. I knew a lot about programming, EDI – not so much.
Out of a strong survival instinct, I studied and researched the work being done in the ANSI committees developing the transactions and standards. My boss was actually the ANSI representative for the company, but it fell to me to do a lot of the committee work. Then when I formed my own software and consulting firm I became a full-fledged voting member and it was truly an enlightening experience. I gained some valuable insight into how these structures could be applied to business cases. I learned how to use the structures and the rules to close the gaps between the business case on one end and the business case on the other. Talking with those who submitted standards for approval and hearing the case they made for their proposals was an education in how business data flows and works.
In those days EDI was in fairly wide use, but not so widely known. It was a ‘best of times and worst of times’ thing to be involved in EDI software in those days. Over my first two years in the arena I identified less than 30 programmers who were working on EDI products and less than 15 who actually were knowledgeable of the EDI standards their software needed to support. Some of us used to get together at the annual shows and have a few beers and shoot pool. It was the “best of times’ for job hunting, but the “worst of times” for overloading your butt and living up to it – or not. (In case you didn’t know it – that’s the definition of consulting: overloading your butt and living up to it) Not many of us even knew what a transaction set was and surely had never written code to support the movement of data in and out of one.
Most of the programming efforts were in-house projects to support demands of EDI exchanges by trading partners. The majority of the EDI participation was not a strategic plan it was a reaction to someone else’s. The growth in EDI exchanges was being driven by the grocery, transportation, and automobile industries. The major players in those industries – particularly the automobile manufacturers in the late 80s – demanded that their trading partners ‘do EDI”, thus driving a growth that was fun and exciting. They were putting the proverbial gun to their suppliers’ heads and forcing them to invest in this new methodology and the software to support it. Bad news was that there wasn’t any software to be had. The good news was we were writing and selling it as fast as we could.
It was fun and exciting because the EDI software market was great, and the job was no longer mundane. The old adage about a programmer needing something new every few days or he gets bored is true and we were not bored. The race was on. There were about 10 companies bringing EDI translation software to market in those days. A few of them still exist , and there have been some really good new ones arise.
By today’s standards, the early products were rudimentary. They supported a limited range of business cases and formats, and so long as you did business the way their designers thought you should, they worked fine. Trying to color outside the lines resulted in a lot of frustration and support costs and a lot of money for consultants and support contracts.
EDI software, or enterprise messaging systems as they bill themselves today, evolved from those early days for two basic reasons. The fist is obvious, the second – not so much.
The biggest influence on the software design was customer requirements. This was new ground. Software was being written in a reactive mode. The more customers one got, the more functionality they wanted. In those days a ‘version’ of a software product was lucky to be viable and supported for more than a few weeks. It was like chasing the latest iPhone today. The software evolved to a more robust functionality and eventually into the tools we see in the more powerful products on the market today.
But, that second major reason for this amazing revolution was that as we gained customers and feedback it became obvious that there is a finite set of functions that must be performed by any successful EDI translator. We were defining that list as we wrote code and as the customers encountered more needs, but in the pure movement of data between non-EDI format and EDI format it became pretty obvious that the scope of functionality was not unlimited. Sure, we expanded the non-EDI formats we supported – XML, databases, flat files, etc – but the actual formulation and interpretation of standard transaction sets and interchanges is not rocket science.
The best system design and programming minds of the day were in a race to get the next generation of product on the market. Those of you who have worked as a programmer will understand that when good programmers attack a finite list of functionality with the aim to create the most efficient code the end result will be that all their products will eventually be very, very similar. Good functional code is good functional code – the rest is bells and whistles. The best products on the market today are almost indistinguishable except for variances in the user interfaces. I’ve worked in every major EDI translation product on the market today and most of the not-so-major and I’ve found the functionality is pretty much the same – the keystrokes to apply that functionality vary because of designer preference more than anything else. The learning curve for anyone that has an in-depth knowledge of EDI structures is short and not very steep.
But that is one of the problems in the industry I see today. Not so many people have a working knowledge of the mechanics of EDI as they used to. The electronic data interchange arena has almost come full circle with regard to why a company enters the market and the way his EDI solution is handled. Those making EDI policy and those tasked with making that policy work are not sufficiently educated in exactly what EDI is and the care and feeding of it. Too many policy makers see it as a task imposed on them instead of the strategic advantage it is. The rank and file who make it work on a daily basis have no idea how to do that.
As a consulting firm my business was involved in many educational efforts in the late 80s and through the 90s. We taught EDI 101, advanced mapping techniques, the EDI structures, the rules for using them, and how to apply them to the business case. We taught how to identify the business case and the data requirements of that case. By the late 90s the customer base was well informed and we were getting tough questions in the sales and support cycles. In the annual DISA conventions I saw the course attendance rise through the 90s – those responsible for making EDI policy and making policy work were hungry to understand. Then it started falling off. As companies established EDI programs that satisfied their needs the educational demand weakened. If it is established correctly and nothing changes EDI becomes a fairly transparent process.
Then between 2000 and 2010 I saw more and more companies looking for consulting services. This demand was in response to a normal attrition of knowledge. As the employees who had established the EDI programs were replaced or moved on the EDI programs had become “monkey see – monkey do” things. So long as nothing changed everything was OK. But, if a trading partner needed changes, or if the company changed or upgraded ERP systems that were interfaced with the EDI systems then it quickly became obvious that there was a serious deficit in the knowledge base. And, all of you know no one ever documents anything.
So, we’re almost back to full circle. Most of the companies I am involved with now are already EDI active. New to EDI or not, all of them are EDI re-active. They are faced with some changes in their business processes that require them to establish or revisit the EDI effort. It was purchased a long time ago and the guy who brought it online is retired or whatever. They need knowledge. This EDI thing is a new concept to many of them even though their businesses have been EDI active for years. To make things worse, decisions about how to change or affect the EDI solution are being made without any real knowledge or experience in the actual tasks that make EDI policy work.
Most of the projects I have been on in the last 5 years have been in trouble because the client had relied on “project management” as the primary skill set to design and manage their EDI project. It is a valuable skill set. But no project designed without the input of those who know the tasks involved and the skills required meeting the project goals can be well managed by anyone regardless of their project management experience. Too many times I’ve been asked how to accomplish a task that had already been scheduled and a completion date set. Too many times the political ownership of those decisions have outweighed my recommendations on how the project goals could be met in a more efficient and cost-effective way.
As a consultant with a lot of years under my belt I feel qualified to make an observation about EDI consultants that might not sit well with some of them. But, I’ve paid my dues and I’m too old to really care. Many of the EDI consultants I meet today have only a rudimentary knowledge of the EDI transaction sets and the power of the mapping tools on the market today. This includes some of the EDI software houses’ support staffs, too. They would rather make a change to some java script or BPML task, or suggest a business process change than simply use the power of the almost unlimited permutations of implementations of any one EDI transaction set. I am reminded of the tale about the guy who stated he had 20 years of experience on his resume when in reality he had one year 20 times. He did one task every day for 20 years. These consultants had been working for a company and been involved in EDI for that company for a couple of years or so. Then they left and billed themselves as EDI consultants. The fact is that EDI is designed to be different for every business case, for every transaction exchange within every business case. So most of these guys have one year of experience for a couple of years and it’s a very limited experience, limited to how that one business did things. I actually had one express pure amazement when I showed him a list of codes values for the N1_01 element. He thought the only N1 loops possible were bill to and ship to and had asked me how he could send the manufacturer name and address. EDI specs required the manufacturer. The IDOC was sending the manufacturer information but he didn’t think was possible in the EDI transaction. I was more amazed than him – he was the project EDI lead. For a day or two, anyway.
So, break out the bell bottom jeans and disco music. We’re back to the 80s. Few people know how to spell “EDI” and few businesses are in the EDI arena by choice. What the EDI industry needs more than anything else right now is good knowledge transfer. Back to the basics. The software is the best it has ever been. The power of the mapping tools is unbelievable – essentially ‘anything to anything’ which was a dream when I was writing code. Business boardrooms need to be re-educated on how it works and just how it can directly lower the costs of doing business. Then their IT people need to know how to make the stuff work with the existing business processes and practices. It’s a dirty job but somebody’s gotta do it. There are good consulting firms that have been through the wars and survived – use them.
Remember – if you have to change your business practices to do EDI you’re doing EDI wrong.