Skip to end of metadata
Go to start of metadata

You are viewing an old version of this page. View the current version.

Compare with Current View Page History

« Previous Version 44 Current »

WORK IN PROGRESS

This page documents the existing Yang Parser used in ONAP and OpenDayLight and will investigate if it fulfill the needs of the C&PS.

Overview

The Yang parser used in ONAP (CCSDK / SDNC) was developed (and still is) a OpenDayLight Library.

There are 2 different usage patterns within CCSDK, though.

Most of CCSDK/SDNC is using Yang primarily to define their northbound interfaces. In that context, there’s a maven plugin (org.opendaylight.yangtools:yang-maven-plugin) that is used to generate source code from the Yang model. Our application code in that case doesn’t really do anything directly with the Yang, since all of the that is handled for us by the generated code.

In ccsdk/sli/plugins, there is a plugin called restconf-client which was contributed by Huawei. That code uses the yangtool parser more directly so that it can interpret the results being returned when it calls a restconf interface. 

Resources

Additional Resources (still to be examined)

*Although this documentation link is to the latest ODL doc revision, it is very outdated and the code examples need significant updates, see findings in Mini-PoC below (bug reported: https://jira.opendaylight.org/browse/DOCS-126)

Mini-PoC

To help this evaluation I will create a small sample project with the goal to pare a yang file using the ODL Yang Tools. I will report my findings here

Maven dependency

The documentation mentioned above lists many modules but the code examples do not clarify which exactly are needed to parse a Yang model and Yang data files in java code.


pom.xml 
    <properties>
        <maven.compiler.version>3.8.1</maven.compiler.version>
        <maven.compiler.release>11</maven.compiler.release>
        <odl.yangtools.version>5.0.5</odl.yangtools.version>
    </properties>

    <build>
        <plugins>
            <plugin>
                <groupId>org.apache.maven.plugins</groupId>
                <artifactId>maven-compiler-plugin</artifactId>
                <version>${maven.compiler.version}</version>
                <configuration>
                    <release>${maven.compiler.release}</release>
                </configuration>
            </plugin>
        </plugins>
    </build>

    <dependencies>
        <dependency>
            <groupId>org.opendaylight.yangtools</groupId>
            <artifactId>yang-parser-api</artifactId>
            <version>${odl.yangtools.version}</version>
        </dependency>
        <dependency>
            <groupId>org.opendaylight.yangtools</groupId>
            <artifactId>yang-parser-impl</artifactId>
            <version>${odl.yangtools.version}</version>
        </dependency>
        <dependency>
            <groupId>org.opendaylight.yangtools</groupId>
            <artifactId>yang-model-util</artifactId>
            <version>${odl.yangtools.version}</version>
        </dependency>

        <dependency>
            <groupId>org.opendaylight.yangtools</groupId>
            <artifactId>yang-data-codec-xml</artifactId>
            <version>${odl.yangtools.version}</version>
        </dependency>
        <dependency>
            <groupId>org.opendaylight.yangtools</groupId>
            <artifactId>yang-data-codec-gson</artifactId>
            <version>${odl.yangtools.version}</version>
        </dependency>
        <!-- SLF4J API -->
        <dependency>
            <groupId>org.slf4j</groupId>
            <artifactId>slf4j-api</artifactId>
            <version>1.6.1</version>
        </dependency>
        <!-- LOG4J -->
        <dependency>
            <groupId>org.slf4j</groupId>
            <artifactId>slf4j-log4j12</artifactId>
            <version>1.6.1</version>
        </dependency>
    </dependencies>

Note. SLF4J has also been added as the implementation requires an enabled logger

Documentation Code Updates

The sample code provided in the documentation is faulty (using == for assigning?!) and is using some long deprecated and even removed classes and methods.

Bug reported: https://jira.opendaylight.org/browse/DOCS-126

Corrected Code 
    static YangParserFactory PARSER_FACTORY;

    static {
        final Iterator<YangParserFactory> it = ServiceLoader.load(YangParserFactory.class).iterator();
        if (!it.hasNext()) {
            throw new IllegalStateException("No YangParserFactory found");
        }
        PARSER_FACTORY = it.next();
    }

    ...

        File file = new File(ClassLoader.getSystemClassLoader().getResource("example.yang").getFile());
        YangTextSchemaSource source = YangTextSchemaSource.forFile(file);
        
        final YangParser yangParser = PARSER_FACTORY.createParser(StatementParserMode.DEFAULT_MODE);
        yangParser.addSource(source);
        SchemaContext schemaContext = yangParser.buildEffectiveModel();

        schemaContext.getModules()

This is the kind of object (module) that gets created:

Generated Java Object Structures per Yang concept

Object Structure

The SchemaContext object generated by the Yang Parser in java has the following possible structures (java collections)

 

SchemaTree is implemented as a ImmutableMap<QName, SchemaTreeEffectiveStatement<?>> 

The SchemaTree can represent a tree of any Yang Leaf, List or Container. All leafs implement the  LeafStatement interface which provides methods for many Yang Language features including

  • getType()
  • getUnits()
  • getMandatory()
  • getWhenStatement()
  • getMustStatments()

Main data types

The package org.opendaylight.yangtools.yang.model.util.type contains classes for all the possible data types including:

  • BaseBinaryType
  • BaseBitsType
  • BaseBooleanType
  • BaseDecimalType
  • BaseEnumerationType
  • BaseInt8Type
  • BaseInt16Type
  • BaseInt32Type
  • BaseInt64Type
  • BaseStringType
  • BaseUint8Type
  • BaseUint16Type
  • BaseUint32Type
  • BaseUint64Type

There are also some special data types such as:

  • BaseEmptyType
  • BaseIdentityrefType
  • BaseInstanceIdentifierType
  • BaseLeafrefType
  • BaseUnionType

And also 'restricted' versions of the base types such as:

  • RestrictedStringType
  • RestrictedUint64Type


DescriptionYangJava Object ViewNotesXML Validation

JSON
Validation

Datatypes and basic constraints

Basic String

leaf response-code {
  type string;
}

TypeStatement TypeAwareDeclaredStatement.getType()YesYes
Mandatory Basic String

leaf response-code {925px
  type string;
    mandatory "true";
}

Optional<MandatoryStatement> MandatoryStatementAwareDeclaredStatement.getMandatory()

NoNo
Limited String leaf pnf-name {
  type string {
    length "0..256";
}

Optional<LengthConstraint> AbstractLengthRestrictedType.getLengthConstraint()

YesYes
typedef (String) with pattern

typedef dotted-quad {
  type string {
pattern
  '(([0-9] ...';
  }
}

leaf address {
   type dotted-quad;
   mandatory "true";
}

List<PatternConstraint> RestrictedStringType.getPatternConstraints()YesYes
Limited uint64 leaf cid {
  type uint64 {
     range "0..503";
  }
}

org.opendaylight.yangtools.yang.model.util.type.RestrictedUint64TypeYesYes
boolean with default value

leaf blacklisted {
  type boolean;
  default 1;
}

org.opendaylight.yangtools.yang.model.util.type.DerivedBooleanTypeN/AN/A

Unique

Unique list server {
  key "name";
  unique "ip port";
  leaf name {
    type string;
  }
  leaf ip {
    type dotted-quad;
  }
  leaf port {
    type uint32;
  }}

org.opendaylight.yangtools.yang.model.api.stmt.UniqueStatementNoNo

Choice

Choicechoice transfer-method {
  leaf transfer-interval {
    type uint64 { range "15..2880"; }
    units minutes; }
  leaf transfer-on-commit {
  type empty;
  }}

org.opendaylight.yangtools.yang.model.api.stmt.ChoiceStatementN/AN/A

Must

Must leaf ifType {
type enumeration {
enum ethernet;
enum atm;}}
leaf ifMTU {
type uint32;}
must "ifType != 'ethernet' or "
+ "(ifType = 'ethernet' and ifMTU = 1500)"
{error-message 466px"An ethernet MTU must be 1500";}

org.opendaylight.yangtools.yang.model.api.stmt.ErrorMessageStatementNoNo

When

When
leaf a {
    type boolean;
}
leaf b {
    type string;
    when "../a = 'true'";
}

org.opendaylight.yangtools.yang.model.api.stmt.WhenStatementNoNo

Extension

Extension declaration

extension store-state-ext {
argument duration;
description "An extension to enable
 state-storage for any attribute.
Use duration to specify how long: nnn<h|d|y>";
}


N/A
Extension usage leaf attribute-with-temporal-storage {
  type string;
  cm-notify-api:store-state-ext "3d";
// store state 3 days
}

org.opendaylight.yangtools.yang.model.api.stmt.UnrecognizedStatement

which extends UnknownStatement:

public interface UnknownStatement<A> extends DeclaredStatement<A> {
    default @Nullable A getArgument() {
        return argument();
    }
}
N/A

Augmentation


 augment "server" {
  when "port = '8787'";
    leaf enable-debug {
      type boolean;
    }
}

The additional leaf just appears in the SchemaTree (without any reference that it is an augmentation)N/A

RPC

rpc
rpc nbrlist-change-notification {
  description
    "RAN Neighbor List change notification to configure RuntimeDB";
     input {
       :
    }
    output {
      : 
    }
}


N/A
rpc input
 input {
     leaf fap-service-number-of-entries-changed {
       type uint64;
       description
         "Number of cells for which neighbor list has changed";
     }
     list fap-service {
       key "alias";
       leaf alias {
         type string {
           length "1..64";
         }
       }
       leaf cid {
         type string {
         length "0..52";
         }
       }
       uses x-0005b9-lte-g;
       leaf lte-cell-number-of-entries {
         type uint64;
         description
           "Number of cells in a neighbor list that was changed";
       }
       list lte-ran-neighbor-list-in-use-lte-cell-changed {
         key "plmnid cid";
         uses lte-ran-neighbor-list-in-use-lte-cell-g;
         description
           "Changed/Modified List of cells 
in a neighbor list for this fap service";
     }
   }
}


N/A
rpc output
output {
    uses cm-notification-response;
}


N/A

Yang Data Parsing and Validation

XML Parsing

XML Parsing Example 
		SchemaContext schemaContext = ... (see previous snippets)
        final Module module = schemaContext.findModules("ultraconfig-interfaces").iterator().next();
        QName qName = QName.create(module.getQNameModule(),"interfaces");
        final Optional<DataSchemaNode> node = module.findDataChildByName(qName);
        if (node.isPresent()) {
            final InputStream resourceAsStream = ClassLoader.getSystemClassLoader().getResourceAsStream("example2data.xml");
            final XMLStreamReader reader = UntrustedXML.createXMLStreamReader(resourceAsStream);
            final NormalizedNodeResult result = new NormalizedNodeResult();
            final NormalizedNodeStreamWriter streamWriter = ImmutableNormalizedNodeStreamWriter.from(result);
            final XmlParserStream xmlParser = XmlParserStream.create(streamWriter, schemaContext, node.get() );
            xmlParser.parse(reader);
            final NormalizedNode<?, ?> transformedInput = result.getResult();
        }

*Note: the DataSchemaNode being used when creating the XmlParserStream HAS to be the root node of the xml data!

XML Validation Findings

  • The XML Parser is found to do basic data type checks including range checks and (regex) pattern validation. If the dat input doesn't conform those a clear exception detailing the problem is thrown
  • Features such as 'mandatory' and 'unique' are to be validated
  • More advanced features such as 'must', 'when', 'choice' etc have not yet been tested

The table in the sections above has a column with the XML validation findings.

JSON Parsing

JSON Parsing Example 
		SchemaContext schemaContext = ... (see previous snippets)
        JSONCodecFactory jsonCodecFactory = JSONCodecFactorySupplier.DRAFT_LHOTKA_NETMOD_YANG_JSON_02.getShared(schemaContext);
        final NormalizedNodeResult result = new NormalizedNodeResult();
        final NormalizedNodeStreamWriter streamWriter = ImmutableNormalizedNodeStreamWriter.from(result);
        final JsonParserStream jsonParser = JsonParserStream.create(streamWriter, jsonCodecFactory);
        final InputStream resourceAsStream = ClassLoader.getSystemClassLoader().getResourceAsStream("example2data.json");
        final JsonReader jsonReader = new JsonReader(new InputStreamReader(resourceAsStream));
        jsonParser.parse(jsonReader);
        final NormalizedNode<?, ?> transformedInput = result.getResult();

JSON Validation Findings

  • As expected the parsing of string, originating form XML or JSON is done by the same code and the results are identical to those for XML Data Parsing

Conclusion

Pros

  1. The YangTools model parser is comprehensive and covers all possible Yang Language elements we might require
  2. Extensively used throughout ONAP and ODL projects
  3. Mature code, dates back to 2013 with contributions from many companies including Cisco and Redhat and Pantheon Tech.
  4. Parsing of JSON and XML data with validated for a a (parsed) model included

Cons

  1. Due to its completeness it also is a complicated piece of software which will take some time to get familiar with
  2. Documentation out of date, this page hopes to address that somewhat





  • No labels