The flat file strikes back: BizTalk 2004 parsing positional records


In a previous entry, I outlined how flat files with delimited records can be parsed by the flat file disassembler. Another post on microsoft.public.biztalk.general attracted my attention. Let’s help Norbert disassemble his file.

The document to disassemble is presented below. I have inserted blank lines to facilitate reading the data. Keep in mind that these blank lines are not in the original data. I strongly suggest to navigate to the original post to better understand the original format:

1 112300004321003901015632011234562003120120031231 2000158EUR2003123101

200000000000100000000000000 000000022650000000000000000000001955080311Rutten/Verbeek G.W.H. 1231SZ00058 NL10 0002265 3000000000002Schakel LOOSDRECHT 01015632 40000000000030000000000 00000002265 00000000 2003111000808001000100000000000000000000000001700D

200000000000400000000000000 000000627160000000000000000000001964031811Heese M.J. van 4566BB00021 NL10 0062716 3000000000005Magdalenastraat Heikant 01000112 40000000000060000000000 00000062716 01000112 2003111000808001000100000000000000000000000001700D

200000000000700000000000000 000000627170000000000000000000001966041821Heese K.E. van 4566BB00021 NL10 0062717 3000000000008Magdalenastraat Heikant 01005227 80000000000090001Betaler20062716 1M.J. van Heese Magdalenastraat 00021 4566BBHeikant 1803196400000000000 40000000000100000000000 00000062717 01005227 2003111000808007000100000000000000000000000004765D

200000000001100000000000000 000077777770000000000000000000001929072911Jans R. 4602PC00013boveNL10 7777777 3000000000012Algemenestraat OOSTERHOUT 01015632 80000000000130001Betaler27777777 1R. Jans Castricummerallee 00035 4607ADOOSTERHOUT 2907192900000000000 40000000000140000000000 00007777777 00000000 2003111000808001000100000000000000000000000001700D

9000140000009865D1

I have no knowledge of the data so there will be some guessing going on here. However, Norbert knows the format of the data so he should be able to adjust what I wrote here to his application. It looks like the file has a header (in blue), a body with two records of type A (in green) and two records of type B (in orange). Sharp readers have seen that the last two records have one more data than the first two. There is also a footer (in purple). But copying the data into a decent text editor, you can easily see that the file has positional records. This means that fields have a fixed width and they start always at the same offset from the beginning of the record. During this pre-investigation, I also noticed that every line in the file is exactly 160 characters.

In practice, for flat file with positional records, it is easier to start by building a flat file disassembling schema that takes a line of input and puts it into one element in the output XML. This approach simplifies the task of getting line terminators right. When you have got the line parsing right, you can then break each individual line down to individual fields without having to worry about line terminators (almost). Anyway, let’s get started. If you wish to download the whole schema, click here

1 – A strawman schema:

Assuming that the flat file posted was representative, we can already end up with a schema that breaks up data line by line: first a Header taking the whole header field (160 characters) to a string (there is exactly one line for the header). Then there are two records and each of them is composed of three lines. Then, two other records, but there are four lines long. And finally, a header which is one line long. Since lines are separated by <CR LF>, we can start building a strawman schema as shown below:

The Root node should be a delimited record, where Child Order appears in PostFix order. This is because the data of a field appears before the line terminator. The delimiter will be set to 0x0D 0x0A and its mode will be “Hexadecimal”. This way, we can get all lines of the files correctly parsed under Root.

Under the Root node, a first sequence is used to define that there will be exactly one “Header” record (minOccurs and maxOccurs on the sequence are set to 1). This element is a Positional record. By default, the BizTalk schema editor for flat file disassembler assumes delimited records. We need to switch the mode to “Postional” and this is easy: just change the “Structure” type to be “Positional” in the record’s properties page. For now, we will set the “Postional length to 160 to grab the whole header’s line into one attribute.

The second sequence is there only to specify that there are exactly two (minOccurs = maxOccurs = 2) records of type “ARecord”. Under this structure, I have broken the record into three lines called “ARecordA”, “ARecordB” and “ARecordC”. Under every record, I have put one single attribute for now with a positional length of 160 to grab the whole line in one field.

The same approach can be done with the second set of records: exactly two records of type BRecord. This record has four lines so I have four elements under the sequence. The trailer as its own sequence (minOccurs = maxOccurs = 1) and one item under it to grab the whole line.

2 – Testing the strawman schema:

One can use FFDasm,.exe (see a previous post) to see how well we did:

FFDAsm input.txt -bs FF.xsd -c

While the output is too large to be reproduced here (download the schema if you want to try it), we observe that indeed, every line was parsed and put into an attribute of the previously mentioned elements.

3 – Breaking down lines into specific fields:

We were successful in taking full lines of input into one attribute (or element) of the XML output. This is good but this is not enough. What we really want to do (especially if we are going to do more processing on this business document) is to break each individual field into an attribute (or element). This is in general a tricky process but I’ll try to give you a few tricks to achieve this quicker.

For the purpose of this example, I will only concentrate on “ARecordA” above. You can always download the whole schema here. The key here is to figure out exactly how long a field is. Remember that in our file, lines are 160 characters long. So if the sum of the size of all your fields under a record ends up being different from 160, then the disassembler will flag an error. So every time you change the positional length of a field, make sure that all the fields in a record (which maps to a line in our example) add up to a full line of 160 characters. I have found convenient to copy the field and paste them into word (with spaces) and use the “Count characters (with spaces)” feature to figure out the exact width of the field.

So here we go. For ARecordA, we have 7 fields. The first one is 32 characters long, the second one is 42 long, the third one is 25 long, the fourth one is 16 long, the fifth one is 15, the sixth one is 9 and the last one is 21. To check that I did not make any mistake, I can switch to the “Flat File” tab in the BizTalk 2004 schema editor and observe that the length of RecordA is indeed 160.

4 – So how about those white spaces in fields?:

Yes, this is true. There are white spaces at the end of Rutten/Verbeek in the third field or RecordA. The good news is that in this example, we do not have anything do to. By default, positional record parsing is setup to ignore the padding characters and the character to ignore is space.

Hopefully, this helped Norbert to get his issue resolved.

Comments (13)

  1. Norbert says:

    Thanks for the big help. I’m really new with flat files. But I tried to run the schema with the dat using FFDAsm, but it still generates an error. It is in Dutch but this what it says:

    Error

    Source: System.Xml

    Message: Data on root(?) level is invalid. Line 1 Position 1

    HRESULT: 80131940

    Thanx

  2. Norbert says:

    Forget the previous comment

    I got the rn error but solved this with:

    allow_early_termination="true"

    Getting another error:

    Message: Unexpected end of stream looking for: P0sitional Data (Length is 30)

    The current definition being parsed is ARecordB. The sream offset where the error occured is 1322.

    Thanx

  3. Norbert says:

    This is the schema I need to use on the flatfiles, but I think the Flat Files we get ( my company) can sometimes be a bit different, but they should be like the schema because it is used as a standard.

    <?xml version="1.0" encoding="utf-16" ?>

    – <xs:schema xmlns="http://Schemas.HA34_3&quot;“>http://Schemas.HA34_3&quot; xmlns:b="http://schemas.microsoft.com/BizTalk/2003&quot; targetNamespace="http://Schemas.HA34_3&quot;“>http://Schemas.HA34_3&quot; xmlns:xs="http://www.w3.org/2001/XMLSchema"&gt;

    – <xs:annotation>

    – <xs:appinfo>

    <schemaEditorExtension:schemaInfo namespaceAlias="b" extensionClass="Microsoft.BizTalk.FlatFileExtension.FlatFileExtension" standardName="Flat File" xmlns:schemaEditorExtension="http://schemas.microsoft.com/BizTalk/2003/SchemaEditorExtensions&quot; />

    <b:schemaInfo root_reference="DECL_HA34_3" count_positions_by_byte="false" standard="Flat File" allow_early_termination="true" />

    </xs:appinfo>

    </xs:annotation>

    – <xs:element name="DECL_HA34_3">

    – <xs:annotation>

    – <xs:appinfo>

    <b:recordInfo structure="delimited" preserve_delimiter_for_empty_data="true" suppress_trailing_delimiters="false" sequence_number="1" child_delimiter_type="hex" child_delimiter="0x0D 0x0A" child_order="postfix" />

    </xs:appinfo>

    </xs:annotation>

    – <xs:complexType>

    – <xs:sequence>

    – <xs:annotation>

    – <xs:appinfo>

    <b:groupInfo sequence_number="0" />

    </xs:appinfo>

    </xs:annotation>

    – <xs:element minOccurs="1" maxOccurs="unbounded" name="HA34_3">

    – <xs:annotation>

    – <xs:appinfo>

    <b:recordInfo sequence_number="1" structure="positional" preserve_delimiter_for_empty_data="true" suppress_trailing_delimiters="false" />

    </xs:appinfo>

    </xs:annotation>

    – <xs:complexType>

    – <xs:sequence>

    – <xs:annotation>

    – <xs:appinfo>

    <b:groupInfo sequence_number="0" />

    </xs:appinfo>

    </xs:annotation>

    – <xs:element name="BERICHTBEGIN_INFORMATIE">

    – <xs:annotation>

    – <xs:appinfo>

    <b:recordInfo sequence_number="1" structure="positional" preserve_delimiter_for_empty_data="true" suppress_trailing_delimiters="false" />

    </xs:appinfo>

    </xs:annotation>

    – <xs:complexType>

    – <xs:sequence>

    – <xs:annotation>

    – <xs:appinfo>

    <b:groupInfo sequence_number="0" />

    </xs:appinfo>

    </xs:annotation>

    – <xs:element name="COD001_VEKT" type="xs:string">

    – <xs:annotation>

    – <xs:appinfo>

    <b:fieldInfo sequence_number="1" justification="left" pos_length="1" />

    </xs:appinfo>

    </xs:annotation>

    </xs:element>

    – <xs:element minOccurs="0" maxOccurs="1" name="TEC025_OO1" type="xs:string">

    – <xs:annotation>

    – <xs:appinfo>

    <b:fieldInfo sequence_number="2" justification="left" pos_length="1" />

    </xs:appinfo>

    </xs:annotation>

    </xs:element>

    – <xs:element name="COD002_VEKT" type="xs:string">

    – <xs:annotation>

    – <xs:appinfo>

    <b:fieldInfo sequence_number="3" justification="left" pos_length="3" />

    </xs:appinfo>

    </xs:annotation>

    </xs:element>

    – <xs:element name="NUM001_VEKT" type="xs:string">

    – <xs:annotation>

    – <xs:appinfo>

    <b:fieldInfo sequence_number="4" justification="left" pos_length="1" />

    </xs:appinfo>

    </xs:annotation>

    </xs:element>

    – <xs:element minOccurs="0" maxOccurs="1" name="COD061_VEKT" type="xs:string">

    – <xs:annotation>

    – <xs:appinfo>

    <b:fieldInfo sequence_number="5" justification="left" pos_length="4" />

    </xs:appinfo>

    </xs:annotation>

    </xs:element>

    – <xs:element minOccurs="0" maxOccurs="1" name="COD377_VEKT" type="xs:string">

    – <xs:annotation>

    – <xs:appinfo>

    <b:fieldInfo sequence_number="6" justification="left" pos_length="8" />

    </xs:appinfo>

    </xs:annotation>

    </xs:element>

    – <xs:element name="COD085_VEKT" type="xs:string">

    – <xs:annotation>

    – <xs:appinfo>

    <b:fieldInfo sequence_number="7" justification="left" pos_length="8" />

    </xs:appinfo>

    </xs:annotation>

    </xs:element>

    – <xs:element minOccurs="0" maxOccurs="1" name="COD181_VEKT" type="xs:string">

    – <xs:annotation>

    – <xs:appinfo>

    <b:fieldInfo sequence_number="8" justification="left" pos_length="8" />

    </xs:appinfo>

    </xs:annotation>

    </xs:element>

    – <xs:element minOccurs="0" maxOccurs="1" name="DAT105_NEN" type="xs:string">

    – <xs:annotation>

    – <xs:appinfo>

    <b:fieldInfo sequence_number="9" justification="left" pos_length="8" />

    </xs:appinfo>

    </xs:annotation>

    </xs:element>

    – <xs:element name="DAT106_NEN" type="xs:string">

    – <xs:annotation>

    – <xs:appinfo>

    <b:fieldInfo sequence_number="10" justification="left" pos_length="8" />

    </xs:appinfo>

    </xs:annotation>

    </xs:element>

    – <xs:element name="NUM029_VEKT" type="xs:string">

    – <xs:annotation>

    – <xs:appinfo>

    <b:fieldInfo sequence_number="11" justification="left" pos_length="12" />

    </xs:appinfo>

    </xs:annotation>

    </xs:element>

    – <xs:element name="COD363_ISO" type="xs:string">

    – <xs:annotation>

    – <xs:appinfo>

    <b:fieldInfo sequence_number="12" justification="left" pos_length="3" />

    </xs:appinfo>

    </xs:annotation>

    </xs:element>

    – <xs:element name="DAT031_NEN" type="xs:string">

    – <xs:annotation>

    – <xs:appinfo>

    <b:fieldInfo sequence_number="13" justification="left" pos_length="8" />

    </xs:appinfo>

    </xs:annotation>

    </xs:element>

    – <xs:element name="NUM002_VEKT" type="xs:string">

    – <xs:annotation>

    – <xs:appinfo>

    <b:fieldInfo sequence_number="14" justification="left" pos_length="2" />

    </xs:appinfo>

    </xs:annotation>

    </xs:element>

    – <xs:element minOccurs="0" maxOccurs="1" name="TEC007_085" type="xs:string">

    – <xs:annotation>

    – <xs:appinfo>

    <b:fieldInfo sequence_number="15" justification="left" pos_length="85" />

    </xs:appinfo>

    </xs:annotation>

    </xs:element>

    </xs:sequence>

    </xs:complexType>

    </xs:element>

    – <xs:element name="VERZEKERDEN">

    – <xs:annotation>

    – <xs:appinfo>

    <b:recordInfo sequence_number="2" structure="positional" preserve_delimiter_for_empty_data="true" suppress_trailing_delimiters="false" />

    </xs:appinfo>

    </xs:annotation>

    – <xs:complexType>

    – <xs:sequence>

    – <xs:annotation>

    – <xs:appinfo>

    <b:groupInfo sequence_number="0" />

    </xs:appinfo>

    </xs:annotation>

    – <xs:element minOccurs="1" maxOccurs="unbounded" name="VERZEKERDE">

    – <xs:annotation>

    – <xs:appinfo>

    <b:recordInfo sequence_number="1" structure="positional" preserve_delimiter_for_empty_data="true" suppress_trailing_delimiters="false" />

    </xs:appinfo>

    </xs:annotation>

    – <xs:complexType>

    – <xs:sequence>

    – <xs:annotation>

    – <xs:appinfo>

    <b:groupInfo sequence_number="0" />

    </xs:appinfo>

    </xs:annotation>

    – <xs:element name="COD001_VEKT" type="xs:string">

    – <xs:annotation>

    – <xs:appinfo>

    <b:fieldInfo sequence_number="1" justification="left" pos_length="1" />

    </xs:appinfo>

    </xs:annotation>

    </xs:element>

    – <xs:element name="NUM040_VEKT" type="xs:string">

    – <xs:annotation>

    – <xs:appinfo>

    <b:fieldInfo sequence_number="2" justification="left" pos_length="12" />

    </xs:appinfo>

    </xs:annotation>

    </xs:element>

    – <xs:element minOccurs="0" maxOccurs="1" name="COD061_VEKT" type="xs:string">

    – <xs:annotation>

    – <xs:appinfo>

    <b:fieldInfo sequence_number="3" justification="left" pos_length="4" />

    </xs:appinfo>

    </xs:annotation>

    </xs:element>

    – <xs:element name="NUM003_ZNET" type="xs:string">

    – <xs:annotation>

    – <xs:appinfo>

    <b:fieldInfo sequence_number="4" justification="left" pos_length="15" />

    </xs:appinfo>

    </xs:annotation>

    </xs:element>

    – <xs:element minOccurs="0" maxOccurs="1" name="COD038_VEKT" type="xs:string">

    – <xs:annotation>

    – <xs:appinfo>

    <b:fieldInfo sequence_number="5" justification="left" pos_length="11" />

    </xs:appinfo>

    </xs:annotation>

    </xs:element>

    – <xs:element minOccurs="0" maxOccurs="1" name="NUM050_BELA" type="xs:string">

    – <xs:annotation>

    – <xs:appinfo>

    <b:fieldInfo sequence_number="6" justification="left" pos_length="9" />

    </xs:appinfo>

    </xs:annotation>

    </xs:element>

    – <xs:element minOccurs="0" maxOccurs="1" name="NUM087_ECP" type="xs:string">

    – <xs:annotation>

    – <xs:appinfo>

    <b:fieldInfo sequence_number="7" justification="left" pos_length="12" />

    </xs:appinfo>

    </xs:annotation>

    </xs:element>

    – <xs:element name="DAT006_NEN" type="xs:string">

    – <xs:annotation>

    – <xs:appinfo>

    <b:fieldInfo sequence_number="8" justification="left" pos_length="8" />

    </xs:appinfo>

    </xs:annotation>

    </xs:element>

    – <xs:element name="COD134_NEN" type="xs:string">

    – <xs:annotation>

    – <xs:appinfo>

    <b:fieldInfo sequence_number="9" justification="left" pos_length="1" />

    </xs:appinfo>

    </xs:annotation>

    </xs:element>

    – <xs:element minOccurs="0" maxOccurs="1" name="COD241_NEN" type="xs:string">

    – <xs:annotation>

    – <xs:appinfo>

    <b:fieldInfo sequence_number="10" justification="left" pos_length="1" />

    </xs:appinfo>

    </xs:annotation>

    </xs:element>

    – <xs:element minOccurs="0" maxOccurs="1" name="NAM067_NEN" type="xs:string">

    – <xs:annotation>

    – <xs:appinfo>

    <b:fieldInfo sequence_number="11" justification="left" pos_length="25" />

    </xs:appinfo>

    </xs:annotation>

    </xs:element>

    – <xs:element minOccurs="0" maxOccurs="1" name="NAM025_NEN" type="xs:string">

    – <xs:annotation>

    – <xs:appinfo>

    <b:fieldInfo sequence_number="12" justification="left" pos_length="6" />

    </xs:appinfo>

    </xs:annotation>

    </xs:element>

    – <xs:element minOccurs="0" maxOccurs="1" name="NAM068_NEN" type="xs:string">

    – <xs:annotation>

    – <xs:appinfo>

    <b:fieldInfo sequence_number="13" justification="left" pos_length="10" />

    </xs:appinfo>

    </xs:annotation>

    </xs:element>

    – <xs:element name="COD083_KPN" type="xs:string">

    – <xs:annotation>

    – <xs:appinfo>

    <b:fieldInfo sequence_number="14" justification="left" pos_length="6" />

    </xs:appinfo>

    </xs:annotation>

    </xs:element>

    – <xs:element name="NUM030_NEN" type="xs:string">

    – <xs:annotation>

    – <xs:appinfo>

    <b:fieldInfo sequence_number="15" justification="left" pos_length="5" />

    </xs:appinfo>

    </xs:annotation>

    </xs:element>

    – <xs:element minOccurs="0" maxOccurs="1" name="NUM079_NEN" type="xs:string">

    – <xs:annotation>

    – <xs:appinfo>

    <b:fieldInfo sequence_number="16" justification="left" pos_length="4" />

    </xs:appinfo>

    </xs:annotation>

    </xs:element>

    – <xs:element minOccurs="0" maxOccurs="1" name="COD032_NEN" type="xs:string">

    – <xs:annotation>

    – <xs:appinfo>

    <b:fieldInfo sequence_number="17" justification="left" pos_length="2" />

    </xs:appinfo>

    </xs:annotation>

    </xs:element>

    – <xs:element minOccurs="0" maxOccurs="1" name="COD151_VEK1" type="xs:string">

    – <xs:annotation>

    – <xs:appinfo>

    <b:fieldInfo sequence_number="18" justification="left" pos_length="2" />

    </xs:appinfo>

    </xs:annotation>

    </xs:element>

    – <xs:element minOccurs="0" maxOccurs="1" name="COD180_VEK1" type="xs:string">

    – <xs:annotation>

    – <xs:appinfo>

    <b:fieldInfo sequence_number="19" justification="left" pos_length="5" />

    </xs:appinfo>

    </xs:annotation>

    </xs:element>

    – <xs:element minOccurs="0" maxOccurs="1" name="COD316_VEKT" type="xs:string">

    – <xs:annotation>

    – <xs:appinfo>

    <b:fieldInfo sequence_number="20" justification="left" pos_length="11" />

    </xs:appinfo>

    </xs:annotation>

    </xs:element>

    – <xs:element minOccurs="0" maxOccurs="1" name="TEC007_010" type="xs:string">

    – <xs:annotation>

    – <xs:appinfo>

    <b:fieldInfo sequence_number="21" justification="left" pos_length="10" />

    </xs:appinfo>

    </xs:annotation>

    </xs:element>

    – <xs:element minOccurs="0" maxOccurs="1" name="COMMENTAREN">

    – <xs:annotation>

    – <xs:appinfo>

    <b:recordInfo structure="positional" preserve_delimiter_for_empty_data="true" suppress_trailing_delimiters="false" sequence_number="22" />

    </xs:appinfo>

    </xs:annotation>

    – <xs:complexType>

    – <xs:sequence>

    – <xs:annotation>

    – <xs:appinfo>

    <b:groupInfo sequence_number="0" />

    </xs:appinfo>

    </xs:annotation>

    – <xs:element minOccurs="1" maxOccurs="unbounded" name="COMMENTAAR">

    – <xs:annotation>

    – <xs:appinfo>

    <b:recordInfo structure="positional" preserve_delimiter_for_empty_data="true" suppress_trailing_delimiters="false" sequence_number="1" />

    </xs:appinfo>

    </xs:annotation>

    – <xs:complexType>

    – <xs:sequence>

    – <xs:annotation>

    – <xs:appinfo>

    <b:groupInfo sequence_number="0" />

    </xs:appinfo>

    </xs:annotation>

    – <xs:element name="COD001_VEKT" type="xs:string">

    – <xs:annotation>

    – <xs:appinfo>

    <b:fieldInfo sequence_number="1" justification="left" pos_length="1" />

    </xs:appinfo>

    </xs:annotation>

    </xs:element>

    – <xs:element name="NUM040_VEK4" type="xs:string">

    – <xs:annotation>

    – <xs:appinfo>

    <b:fieldInfo sequence_number="2" justification="left" pos_length="12" />

    </xs:appinfo>

    </xs:annotation>

    </xs:element>

    – <xs:element name="NUM109_VEKT" type="xs:string">

    – <xs:annotation>

    – <xs:appinfo>

    <b:fieldInfo sequence_number="3" justification="left" pos_length="4" />

    </xs:appinfo>

    </xs:annotation>

    </xs:element>

    – <xs:element name="OMS016_143" type="xs:string">

    – <xs:annotation>

    – <xs:appinfo>

    <b:fieldInfo sequence_number="4" justification="left" pos_length="143" />

    </xs:appinfo>

    </xs:annotation>

    </xs:element>

    </xs:sequence>

    </xs:complexType>

    </xs:element>

    </xs:sequence>

    </xs:complexType>

    </xs:element>

    – <xs:element minOccurs="0" maxOccurs="1" name="PRESTATIES">

    – <xs:annotation>

    – <xs:appinfo>

    <b:recordInfo structure="positional" preserve_delimiter_for_empty_data="true" suppress_trailing_delimiters="false" sequence_number="23" />

    </xs:appinfo>

    </xs:annotation>

    – <xs:complexType>

    – <xs:sequence>

    – <xs:annotation>

    – <xs:appinfo>

    <b:groupInfo sequence_number="0" />

    </xs:appinfo>

    </xs:annotation>

    – <xs:element minOccurs="1" maxOccurs="unbounded" name="PRESTATIE">

    – <xs:annotation>

    – <xs:appinfo>

    <b:recordInfo sequence_number="1" structure="positional" preserve_delimiter_for_empty_data="true" suppress_trailing_delimiters="false" />

    </xs:appinfo>

    </xs:annotation>

    – <xs:complexType>

    – <xs:sequence>

    – <xs:annotation>

    – <xs:appinfo>

    <b:groupInfo sequence_number="0" />

    </xs:appinfo>

    </xs:annotation>

    – <xs:element name="COD001_VEKT" type="xs:string">

    – <xs:annotation>

    – <xs:appinfo>

    <b:fieldInfo sequence_number="1" justification="left" pos_length="1" />

    </xs:appinfo>

    </xs:annotation>

    </xs:element>

    – <xs:element name="NUM040_VEKT" type="xs:string">

    – <xs:annotation>

    – <xs:appinfo>

    <b:fieldInfo sequence_number="2" justification="left" pos_length="12" />

    </xs:appinfo>

    </xs:annotation>

    </xs:element>

    – <xs:element name="NUM003_ZNET" type="xs:string">

    – <xs:annotation>

    – <xs:appinfo>

    <b:fieldInfo sequence_number="3" justification="left" pos_length="15" />

    </xs:appinfo>

    </xs:annotation>

    </xs:element>

    – <xs:element minOccurs="0" maxOccurs="1" name="COD038_VEKT" type="xs:string">

    – <xs:annotation>

    – <xs:appinfo>

    <b:fieldInfo sequence_number="4" justification="left" pos_length="11" />

    </xs:appinfo>

    </xs:annotation>

    </xs:element>

    – <xs:element minOccurs="0" maxOccurs="1" name="COD383_VEKT" type="xs:string">

    – <xs:annotation>

    – <xs:appinfo>

    <b:fieldInfo sequence_number="5" justification="left" pos_length="12" />

    </xs:appinfo>

    </xs:annotation>

    </xs:element>

    – <xs:element minOccurs="0" maxOccurs="1" name="COD089_VEKT" type="xs:string">

    – <xs:annotation>

    – <xs:appinfo>

    <b:fieldInfo sequence_number="6" justification="left" pos_length="8" />

    </xs:appinfo>

    </xs:annotation>

    </xs:element>

    – <xs:element minOccurs="0" maxOccurs="1" name="NUM032_VEKT" type="xs:string">

    – <xs:annotation>

    – <xs:appinfo>

    <b:fieldInfo sequence_number="7" justification="left" pos_length="12" />

    </xs:appinfo>

    </xs:annotation>

    </xs:element>

    – <xs:element name="DAT021_NEN" type="xs:string">

    – <xs:annotation>

    – <xs:appinfo>

    <b:fieldInfo sequence_number="8" justification="left" pos_length="8" />

    </xs:appinfo>

    </xs:annotation>

    </xs:element>

    – <xs:element name="COD367_VEKT" type="xs:string">

    – <xs:annotation>

    – <xs:appinfo>

    <b:fieldInfo sequence_number="9" justification="left" pos_length="3" />

    </xs:appinfo>

    </xs:annotation>

    </xs:element>

    – <xs:element name="COD322_CTG" type="xs:string">

    – <xs:annotation>

    – <xs:appinfo>

    <b:fieldInfo sequence_number="10" justification="left" pos_length="5" />

    </xs:appinfo>

    </xs:annotation>

    </xs:element>

    – <xs:element name="ANT054_VEKT" type="xs:string">

    – <xs:annotation>

    – <xs:appinfo>

    <b:fieldInfo sequence_number="11" justification="left" pos_length="4" />

    </xs:appinfo>

    </xs:annotation>

    </xs:element>

    – <xs:element minOccurs="0" maxOccurs="1" name="NUM015_VEK1" type="xs:string">

    – <xs:annotation>

    – <xs:appinfo>

    <b:fieldInfo sequence_number="12" justification="left" pos_length="12" />

    </xs:appinfo>

    </xs:annotation>

    </xs:element>

    – <xs:element name="COD078_VEKT" type="xs:string">

    – <xs:annotation>

    – <xs:appinfo>

    <b:fieldInfo sequence_number="13" justification="left" pos_length="1" />

    </xs:appinfo>

    </xs:annotation>

    </xs:element>

    – <xs:element minOccurs="0" maxOccurs="1" name="BED023_VEKT" type="xs:string">

    – <xs:annotation>

    – <xs:appinfo>

    <b:fieldInfo sequence_number="14" justification="left" pos_length="8" />

    </xs:appinfo>

    </xs:annotation>

    </xs:element>

    – <xs:element name="BED020_VEKT" type="xs:string">

    – <xs:annotation>

    – <xs:appinfo>

    <b:fieldInfo sequence_number="15" justification="left" pos_length="8" />

    </xs:appinfo>

    </xs:annotation>

    </xs:element>

    – <xs:element name="COD043_VEKT" type="xs:string">

    – <xs:annotation>

    – <xs:appinfo>

    <b:fieldInfo sequence_number="16" justification="left" pos_length="1" />

    </xs:appinfo>

    </xs:annotation>

    </xs:element>

    – <xs:element minOccurs="0" maxOccurs="1" name="TEC007_039" type="xs:string">

    – <xs:annotation>

    – <xs:appinfo>

    <b:fieldInfo sequence_number="17" justification="left" pos_length="39" />

    </xs:appinfo>

    </xs:annotation>

    </xs:element>

    – <xs:element minOccurs="0" maxOccurs="1" name="COMMENTAREN">

    – <xs:annotation>

    – <xs:appinfo>

    <b:recordInfo structure="positional" preserve_delimiter_for_empty_data="true" suppress_trailing_delimiters="false" sequence_number="18" />

    </xs:appinfo>

    </xs:annotation>

    – <xs:complexType>

    – <xs:sequence>

    – <xs:annotation>

    – <xs:appinfo>

    <b:groupInfo sequence_number="0" />

    </xs:appinfo>

    </xs:annotation>

    – <xs:element minOccurs="1" maxOccurs="unbounded" name="COMMENTAAR">

    – <xs:annotation>

    – <xs:appinfo>

    <b:recordInfo sequence_number="1" structure="positional" preserve_delimiter_for_empty_data="true" suppress_trailing_delimiters="false" />

    </xs:appinfo>

    </xs:annotation>

    – <xs:complexType>

    – <xs:sequence>

    – <xs:annotation>

    – <xs:appinfo>

    <b:groupInfo sequence_number="0" />

    </xs:appinfo>

    </xs:annotation>

    – <xs:element name="COD001_VEKT" type="xs:string">

    – <xs:annotation>

    – <xs:appinfo>

    <b:fieldInfo sequence_number="1" justification="left" pos_length="1" />

    </xs:appinfo>

    </xs:annotation>

    </xs:element>

    – <xs:element name="NUM040_VEK4" type="xs:string">

    – <xs:annotation>

    – <xs:appinfo>

    <b:fieldInfo sequence_number="2" justification="left" pos_length="12" />

    </xs:appinfo>

    </xs:annotation>

    </xs:element>

    – <xs:element name="NUM109_VEKT" type="xs:string">

    – <xs:annotation>

    – <xs:appinfo>

    <b:fieldInfo sequence_number="3" justification="left" pos_length="4" />

    </xs:appinfo>

    </xs:annotation>

    </xs:element>

    – <xs:element name="OMS016_143" type="xs:string">

    – <xs:annotation>

    – <xs:appinfo>

    <b:fieldInfo sequence_number="4" justification="left" pos_length="143" />

    </xs:appinfo>

    </xs:annotation>

    </xs:element>

    </xs:sequence>

    </xs:complexType>

    </xs:element>

    </xs:sequence>

    </xs:complexType>

    </xs:element>

    </xs:sequence>

    </xs:complexType>

    </xs:element>

    </xs:sequence>

    </xs:complexType>

    </xs:element>

    </xs:sequence>

    </xs:complexType>

    </xs:element>

    </xs:sequence>

    </xs:complexType>

    </xs:element>

    – <xs:element name="BERICHTEINDE_INFORMATIE">

    – <xs:annotation>

    – <xs:appinfo>

    <b:recordInfo sequence_number="3" structure="positional" preserve_delimiter_for_empty_data="true" suppress_trailing_delimiters="false" />

    </xs:appinfo>

    </xs:annotation>

    – <xs:complexType>

    – <xs:sequence>

    – <xs:annotation>

    – <xs:appinfo>

    <b:groupInfo sequence_number="0" />

    </xs:appinfo>

    </xs:annotation>

    – <xs:element name="COD001_VEKT" type="xs:string">

    – <xs:annotation>

    – <xs:appinfo>

    <b:fieldInfo sequence_number="1" justification="left" pos_length="1" />

    </xs:appinfo>

    </xs:annotation>

    </xs:element>

    – <xs:element name="ANT074_VEKT" type="xs:string">

    – <xs:annotation>

    – <xs:appinfo>

    <b:fieldInfo sequence_number="2" justification="left" pos_length="5" />

    </xs:appinfo>

    </xs:annotation>

    </xs:element>

    – <xs:element name="BED025_VEKT" type="xs:string">

    – <xs:annotation>

    – <xs:appinfo>

    <b:fieldInfo sequence_number="3" justification="left" pos_length="10" />

    </xs:appi

  4. Gilles says:

    Norbert: Wjat do you mean by "can sometimes be a a bit different"? As you already know, you flat file is currently parsed with a sequence of 2 record A and two records of type B. If that strcuture changes,then you will fail on thsoe messages.

    As far as your error is concerned (error looking for positional data lenght = 30) it usually means that the parser ran out of data while parsing the line. Either way, you specified a record with a larger with that it is or you did not get your line separator fine.

    I understand that once parsed, the XML should validate against the above schema. As I am sure you understand, there are two ways to deal with this: 1) you parse the file and produce the right XML using the flat file disassembler. 2) you parse the file to some XML that is close to the schema you need but not exactly it and you then use a map to adjust the XML to validate against your schema.

  5. Martins says:

    Hi Gilles,

    where can I find these tricks and not published annotation’s like allow_early_termination="true" ?

    Is there any help not shipped with bts2004 ?

    thanks,

    Martins

  6. Gilles says:

    Martins: You might not have updated your documentation to the latest refresh. Downloads are available on http://www.microsoft.com/biztalk.

    More specifically, the attribute you mention (and others) is documented here:

    http://msdn.microsoft.com/library/en-us/sdk/htm/ebiz_prog_edit_ytch.asp?frame=true

    Thanks.

  7. John says:

    Hi Gilles,

    How do you handle file that has has spaces. Simple example:

    ———————————————–

    Definition –

    Header 10 characters

    Detail 10 characters

    Footer 10 characters

    CRLF as the end of line.

    Data –

    Header XXXXXXX**

    Detail XXXXX****

    Detail XXXXXX***

    Detail XX***XX**

    Footer XXXX***XX

    * = space

    X = data

    ———————————————–

    I can read the data in but all spaces are striped & when we send the data out via a send pipeline assembler this is reflected. Is there a way to get the XML to preserve the spaces? Thats the trick I am trying to work out – spaces both between data and at the end of the data line. I am also trying to avoid defining all fields, just want to specify the record as one field with CRLF the delimiter between records.

    Thanks in advance.

    John.

  8. Gilles says:

    John: By default, as I explained, padding is set to "whitespace" and "ignore padding".

    If you do not wish this to happen, set "pad character type" to "none". Everything will be considered as valuable data. This setting is per field, so you will have to do this on every single field you want to keep spaces.

  9. John says:

    Thank Gilles,

    I was able to read in any flat file with CRLF at the end of each row and maintain the spaces by specifying: {pad_char_type="hex" pad_char="0x0A"}

    Note all rows are read into 1 field called "Field" – Schema below.

    Thanks for pointing me in the right direction!!

    Cheers.

    ——————————————————————————-

    <?xml version="1.0" encoding="utf-16"?>

    <xs:schema xmlns="http://BizTalk_Server_Project1.Schema1&quot;“>http://BizTalk_Server_Project1.Schema1&quot; xmlns:b="http://schemas.microsoft.com/BizTalk/2003&quot; targetNamespace="http://BizTalk_Server_Project1.Schema1&quot;“>http://BizTalk_Server_Project1.Schema1&quot; xmlns:xs="http://www.w3.org/2001/XMLSchema"&gt;

    <xs:annotation>

    <xs:appinfo>

    <b:schemaInfo count_positions_by_byte="false" standard="Flat File" root_reference="Root" />

    <schemaEditorExtension:schemaInfo namespaceAlias="b" extensionClass="Microsoft.BizTalk.FlatFileExtension.FlatFileExtension" standardName="Flat File" xmlns:schemaEditorExtension="http://schemas.microsoft.com/BizTalk/2003/SchemaEditorExtensions&quot; />

    </xs:appinfo>

    </xs:annotation>

    <xs:element name="Root">

    <xs:annotation>

    <xs:appinfo>

    <b:recordInfo structure="delimited" preserve_delimiter_for_empty_data="true" suppress_trailing_delimiters="false" sequence_number="1" child_delimiter_type="hex" child_delimiter="0x0D 0x0A" child_order="postfix" />

    </xs:appinfo>

    </xs:annotation>

    <xs:complexType>

    <xs:sequence>

    <xs:annotation>

    <xs:appinfo>

    <b:groupInfo sequence_number="0" />

    </xs:appinfo>

    </xs:annotation>

    <xs:element maxOccurs="unbounded" name="Record">

    <xs:annotation>

    <xs:appinfo>

    <b:recordInfo structure="delimited" preserve_delimiter_for_empty_data="true" suppress_trailing_delimiters="false" child_delimiter_type="hex" child_delimiter="0x0D 0x0A" sequence_number="1" />

    </xs:appinfo>

    </xs:annotation>

    <xs:complexType>

    <xs:sequence>

    <xs:annotation>

    <xs:appinfo>

    <b:groupInfo sequence_number="0" />

    </xs:appinfo>

    </xs:annotation>

    <xs:element maxOccurs="unbounded" name="Field" type="xs:string">

    <xs:annotation>

    <xs:appinfo>

    <b:fieldInfo justification="left" sequence_number="1" pad_char_type="hex" pad_char="0x0A" />

    </xs:appinfo>

    </xs:annotation>

    </xs:element>

    </xs:sequence>

    </xs:complexType>

    </xs:element>

    </xs:sequence>

    </xs:complexType>

    </xs:element>

    </xs:schema>

  10. Mark says:

    Please explain the use of <Sequence> nodes in your schema. I’ve never had to use them, but maybe I should be??

    Mark

    replies to mark_at_markscott_dot_net would be great.