Saturday, May 5, 2012

Protocol Buffers for the mainframe


Protocol Buffers is a technology used internally at Google, which was made available as open source in 2008.

The idea is that XML and JSON are too verbose for communication-intensive systems.

XML and JSON bring two important benefits over binary protocols. They are:

  • Human readable (self documenting)
  • Resilient to changes (fields order is usually not imposed, fields can be missing or partially filled)

But these benefits come at a price:

  • Network load (the ratio of overhead metadata over actual data is quite high)
  • Parsing and writing relatively complex structures is CPU intensive

For systems that exchange isomorphic (same structure) data, millions of times a day, this price is too high.

Not surprisingly, that same diagnostic has stopped XML and JSON from being widely used on mainframes, although IBM introduced an XML parser quite early on z/OS.

Protocol Buffers is a binary protocol. In that sense, human readability and self description is lost. But it is resilient to changes. That second property turns out to be very important in heterogeneous, distributed systems.

Mainframes, and particularly COBOL-written applications, are by now completely immersed in heterogeneous, distributed systems. All IT departments, even those who claim to be very mainframe centric, run dozens of java or .Net applications alongside the mainframe (or sometimes even on the mainframe).

The rise of Enterprise mobile applications is bringing yet more heterogeneity in terms of Operating Systems and programming languages. Mainframe COBOL applications will necessarily have to interoperate with these newcomers too.

So after reading a lot about Protocol Buffers (and its competing sibling Thrift , developed originally at facebook then donated to the Apache Foundation), I came to the conclusion that such protocols might be exactly what COBOL on the mainframe need to better interoperate with Java, C++, ObjectiveC, etc.

To keep up with the spirit of LegStar, I started a new open source project called protobuf-cobol. I hope many of you will try it and let me know what you think.






1 comment:

  1. Hi, I'm a mainframe developer for 40 years and am happy to see there is a COBOL protobuf. Where can I find the COBOL source for this that doesn't require java and an old version of log4j?

    ReplyDelete