Tuesday, December 9, 2014

LegStar V2 is underway

LegStar has been around for a while now, 8 years since the first release in 2006.

Since then a lot of things have changed:

  • The java SDK is now in version 8. It was version 4 in 2006 and things like generics were unheard of
  • Programming patterns have become common practice
  • Multi-threading techniques have improved and are better understood
  • Use cases which used to be centered around remote procedure calls to CICS programs, now deal with massive imports of mainframe data for Big Data Analytics


Some parts of LegStar are showing their age and I have finally found some time to start rewriting some of the core features. The LegStar core project is the older in the product so this is where I started.

The legstar-core2 project is where I placed the new developments.

You should not consider this as a replacement to the current LegStar though:

  • There are far less functionalities in legstar-core2 at the moment than in LegStar
  • The API V2 will not be backward compatible 


That second point may come as a surprise for mainframe users but in the world of open source, breaking compatibility is an "art de vivre". A primary benefit is that the code you get is much cleaner and readable when it does not need to deal with legacy. When the project evolves though, we might want to work on a migration guide of some form.

The new legstar-core2 project contains a simplified version of the legstar-cob2xsd module which is the venerable COBOL to XML schema translator. The changes are minor in this module so far. Neither COBOL, nor the XML schema specs have changed much.

From an architecture standpoint the major change is that JAXB is no longer central to the conversion process. So far, we were always going through the JAXB layer even if the target was Talend, Pentaho, JSON, or any other format.

Now the conversion logic has been abstracted out in a legstar-base module. There is also an associated legstar-base-generator module that produces artifacts for legstar-base. The legstar-base module can be considered a complete low-level solution for mainframe to java conversions. This new legstar-base module has replaced the legstar-coxbapi and legstar-coxbrt modules.

JAXB is still supported of course with 2 new modules, legstar-jaxb and legstar-jaxb-generator which cover the old legstar-coxbgen features.

Besides the architectural changes, there are 2 important changes you need to be aware of:

  • The legstar-core2 project is hosted on Github, not on Google code. Therefore source control has moved from SVN to Git.
  • The licence is GNU Affero GPL which is not as business friendly as the LGPL used by LegStar


Again, this is just the beginning on this new project and its likely to make its way to the newest developments first (such as legstar-avro). Over time, I will describe the new features in more details. In the meanwhile please send any feedback.

Fady

4 comments:

  1. Thanks Fady.
    Could you please point me to some help on how to use legstar-base and legstar-base generator? I want to use it for some mainframe to Java conversion to have something generic job without having compilation step.

    ReplyDelete
  2. First of all, be aware that this is a very early stage for legstar v2. The project is likely to be very unstable for some time.

    If you still like to use it, then you should be able to download a distribution from here: http://search.maven.org/remotecontent?filepath=com/legsem/legstar/legstar-jaxb-generator/2.0.0/legstar-jaxb-generator-2.0.0.zip and unzip it to your file system.

    The samples folder contains an ant script that generates stuff for JAXB so I am assuming this is not what you want.

    If you want to run the legstar-base-generator standalone, you should be able to do that with an ant script such as:

    <project name="cob2java-sample" default="generate" basedir=".">

    <!-- Starting from COBOL copybooks in cobdir -->
    <!-- Generate java conversion classes in gendir -->

    <property name="libdir" location="../lib"/>
    <property name="cobdir" location="cobol"/>
    <property name="config" location="conf/cob2xsd.properties"/>
    <property name="logconfig" location="conf/log4j.properties"/>
    <property name="packageprefix" value="legstar.samples"/>
    <property name="gendir" location="gen/java"/>

    <!-- Classpath for generation -->
    <path id="project.class.path">
    <fileset dir="${libdir}">
    <include name="*.jar"/>
    </fileset>
    </path>
    <property name="project.classpath" refid="project.class.path"/>

    <target name="init">
    <mkdir dir="${gendir}"/>
    </target>

    <target name="generate" depends="init">
    <java classname="com.legstar.base.generator.Cob2CobolTypesGeneratorMain" fork="true" failonerror="true">
    <classpath path="${project.classpath}"/>
    <arg line="-i ${cobdir} -o ${gendir} -c ${config} -p ${packageprefix}"/>
    <jvmarg value="-Dlog4j.configuration=file:///${logconfig}"/>
    </java>
    </target>

    </project>

    This should produce java conversion support classes using COBOL copybooks in the cobol folder. For instance, in the case of the sample CUSTDAT cobol copybook, you get a class called legstar.samples.custdat.CobolCustomerData.

    Now in order to perform actual conversion of mainframe payloads, you will have to write some code similar to this:

    byte[] record = ...; // place your mainframe data here
    Cob2ObjectConverter visitor = new Cob2ObjectConverter(
    cobolContext, record, 0);
    visitor.visit(new CobolCustomerData());
    System.out.println(visitor.getLastObject().toString());

    let me know if that works for you.





    ReplyDelete
  3. Just released 2.0.1 of legstar-core2. There is now a distribution zip for the base generator along with sample java code and ant build.

    Checkout https://github.com/legsem/legstar-core2

    ReplyDelete
  4. Hi, I'm using legstar to convert from copybook to JSON. In the copybook definition used to generate the code we use camel case field names. We have a problem in a HosttoJSON transformer that the JSON keys generated are all in lower case. JSON is case sensitive. How could we make sure that fields camelcase name are preserved in the generated JSON ?

    Thanks

    ReplyDelete