Transferring binary data to/from WebServices (and to/from COM (Automation) objects)


A number of people have asked for guidance on how to transfer data to/from COM and WebServices in NAV 2009.

In the following I will go through how to get and set a picture on an item in NAV through a Web Service Connection.

During this scenario we will run into a number of obstacles – and I will describe how to get around these.

First of all – we want to create a Codeunit, which needs to be exposed to WebServices. Our Codeunit will contain 2 functions: GetItemPicture and SetItemPicture – but what is the data type of the Picture and how do we return that value from a WebService function?

The only data type (supported by Web Services) that can hold a picture is the BigText data type.

We need to create the following two functions:

GetItemPicture(No : Code[20];VAR Picture : BigText)
SetItemPicture(No : Code[20]; Picture : BigText);

BigText is capable if holding binary data (including null terminals) up to any size. On the WSDL side these functions will have the following signature:

image

As you can see BigText becomes string – and strings in .net are capable of any size and any content.

The next problem we will face is, that pictures typically contains all kinds of characters, and unfortunately when transferring strings to/from WebServices there are a number of special characters that are not handled in the WebServices protocol.

(Now you wonder whether you can have <> in your text – but that isn’t the problem:-)

The problem is LF, CR, NULL and other characters like that.

So we need to base64 (or something like that) encode our picture when returning it from Web Services. Unfortunately I couldn’t find any out-of-the-box COM objects that was able to do base64 encoding and decoding – but we can of course make one ourselves.

Lets assume for a second that we have a base64 COM object – then this would be our functions in AL:

GetItemPicture(No : Code[20];VAR Picture : BigText)
CLEAR(Picture);
Item.SETRANGE(Item."No.", No, No);
IF (Item.FINDFIRST()) THEN
BEGIN
  Item.CALCFIELDS(Item.Picture);
  // Get Temp FileName
  TempFile.CREATETEMPFILE;
  FileName := TempFile.NAME;
  TempFile.CLOSE;

  // Export picture to Temp File
  Item.Picture.EXPORT(FileName);

  // Get a base64 encoded picture into a string
  CREATE(base64);
  Picture.ADDTEXT(base64.encodeFromFile(FileName));

  // Erase Temp File
  FILE.ERASE(FileName);
END;

SetItemPicture(No : Code[20];Picture : BigText)
Item.SETRANGE(Item."No.", No, No);
IF (Item.FINDFIRST()) THEN
BEGIN
  // Get Temp FileName
  TempFile.CREATETEMPFILE;
  FileName := TempFile.NAME;
  TempFile.CLOSE;

  // Decode the bas64 encoded image into the Temp File
  CREATE(base64);
  base64.decodeToFile(Picture, FileName);

  // Import picture from Temp File
  Item.Picture.IMPORT(FileName);
  Item.Modify();
  // Erase Temp File
  FILE.ERASE(FileName);
END;

A couple of comments to the source:

  • The way we get a temporary filename in NAV2009 is by creating a temporary file, reading its name and closing it. CREATETEMPFILE will always create new GUID based temporary file names – and the Service Tier will not have access to write files in e.g. the C:\ root folder and a lot of other places.
  • The base64 automation object is loaded on the Service Tier (else it should be CREATE(base64, TRUE, TRUE);) and this is the right location, since the exported file we just stored is located on the Service Tier.
  • The base64.encodeFromFile reads the file and returns a very large string which is the picture base64 encoded.
  • The ADDTEXT method is capable of adding these very large strings and add them to a BigText (BTW – that will NOT work in the classic client).
  • We do the cleanup afterwards – environmental protection:-)

So, why does the ADDTEXT support large strings?

As you probably know, the ADDTEXT takes a TEXT and a position as parameter – and a TEXT doesn’t allow large strings, but what happens here is, that TEXT in C# becomes string – and the length-checking of TEXT variables are done when assigning variables or transferring parameters to functions and the ADDTEXT doesn’t check for any specific length (which comes in handy in our case).

The two lines in question in C# looks like:

base64.Create(DataError.ThrowError);
picture.Value = NavBigText.ALAddText(picture.Value, base64.InvokeMethod<String>(@"encodeFromFile", fileName));

Note also that the base64.decodeToFile function gets a BigText directly as parameter. As you will see, that function just takes an object as a parameter – and you can transfer whatever to that function (BigText, Text, Code etc.). You actually also could give the function a decimal variable in which case the function would throw an exception (str as string would return NULL).

So now you also know how to transfer large strings to and from COM objects:

  1. To the COM object, you just transfer a BigText variable directly to an object parameter and cast it to a string.
  2. From the COM object to add the string return value to a BigText using ADDTEXT.
  3. You cannot use BigText as parameter to a by-ref (VAR) parameter in COM.

In my WebService consumer project I use the following code to test my WebService:

// Initialize Service
CodeUnitPicture service = new CodeUnitPicture();
service.UseDefaultCredentials = true;

// Set the Image for Item 1100
service.SetItemPicture("1100", encodeFromFile(@"c:\MandalayBay.jpg"));

// Get and show the Image for Item 1001
string p = "";
service.GetItemPicture("1001", ref p);
decodeToFile(p, @"c:\pic.jpg");
System.Diagnostics.Process.Start(@"c:\pic.jpg");

and BTW – the source code for the two functions in the base64 COM object are here:

public string encodeFromFile(string filename)
{
    FileStream fs = File.OpenRead(filename);
    BinaryReader br = new BinaryReader(fs);
    int len = (int)fs.Length;
    byte[] buffer = new byte[len];
    br.Read(buffer, 0, len);
    br.Close();
    fs.Close();
    return System.Convert.ToBase64String(buffer);
}

public void decodeToFile(object str, string filename)
{
    FileStream fs = File.Create(filename);
    BinaryWriter bw = new BinaryWriter(fs);
    bw.Write(Convert.FromBase64String(str as string));
    bw.Close();
    fs.Close();
}

If you whish to download and try it out for yourself – you can download the sources here:

The two Visual Studio solutions can be downloaded from http://www.freddy.dk/VSDemo.zip (the base64 COM object and the VSDemo test project)

The NAV codeunit with the two functions above can be downloaded from http://www.freddy.dk/VSDemoObjects.fob.

Remember that after importing the CodeUnit you would have to expose it as a WebService in the WebService table:

image

And…. – remember to start the Web Service listener (if you are running with an unchanged Demo installation).

The code shown in this post comes with no warranty – and is only intended for showing how to do things. The code can be reused, changed and incorporated in any project without any further notice.

Comments or questions are welcome.

Enjoy

Freddy Kristiansen
PM Architect
Microsoft Dynamics NAV

Comments (22)

  1. GaspodeTheWonderDog says:

    Thanks for this post Freddy – it’s awesome. What would be your recomended way of _calling_ a web service from within C/AL? I have used WINHTTP with a hand-crafted SOAP envelope but I realise I could have created an Interop COM component to handle the service calls and simply called that from within C/AL. Is there a recomended approach and are we likely to see the ability to add services or even call the Framework from within C/AL in the future?

  2. FreddyDK says:

    It really depends on a number of things.

    If you can/want to take dependency on COM objects – I would prefer to create a proxy/Interop with a higher level of functionality than the low level WebService layer – and keep glue code away from AL code.

    (like if you want to get exchange rates – have a high level function, which underneath uses Web Services).

    If you do the WebService directly from AL code (XMLHTTP) it will be done from the ServiceTier – if you go for the proxy – you can decide whether you want it to run from the Service Tier or the Client Tier (could be authentication issues).

    Reg. framework / C/AL – as you know, 2009 runs the business logic in managed code (converted to c#) – indicating that we are getting closer to that world. I don’t think we would want to use Visual Studio as the primary editor for our Business Application – but I definitely see the two worlds coming closer together – where it makes sense.

    Today you can do a lot in C# – In the near future I will post how the Edit In Excel was done (which have been shown a number of times) as one example of what you can accomplish with C#, VSTO and Web Services.

  3. sanoj72 says:

    In NAV5 I used the xmldom to base64 code streams but that doesn’t seem to work in NAV2009 RTM ServiceTier ( it works in classic client).

    I tried this this code:

       PROCEDURE GetLargeData@1000000000(VAR LargeData@1000000000 : BigText);

       VAR

         xmldom@1000000001 : Automation "{F5078F18-C551-11D3-89B9-0000F81FE221} 6.0:{88D96A05-F192-11D4-A65F-0040963251E5}:’Microsoft XML, v6.0′.DOMDocument60";

         objStream@1000000002 : Automation "{2A75196C-D9EB-4129-B803-931327F72D5C} 2.8:{00000566-0000-0010-8000-00AA006D2EA4}:’Microsoft ActiveX Data Objects 2.8 Library’.Stream";

         xmlnode@1000000003 : Automation "{F5078F18-C551-11D3-89B9-0000F81FE221} 6.0:{2933BF80-7B36-11D2-B20E-00C04F983E60}:’Microsoft XML, v6.0′.IXMLDOMNode";

       BEGIN

         CLEAR(LargeData);

         CREATE(xmldom);

         REPORT.SAVEASPDF(115, ‘c:115.pdf’);

         //Read the file into the stream

         CREATE(objStream);

         objStream.Type := 1;

         objStream.Open;

         objStream.LoadFromFile(‘c:115.pdf’);

         xmldom.loadXML(‘<data/>’);

         xmlnode := xmldom.selectSingleNode(‘//data’);

         xmlnode.dataType := ‘bin.base64’;

         xmlnode.nodeTypedValue := objStream.Read;

         LargeData.ADDTEXT(xmlnode.text);

       END;

    In NAV2009 classic it works fine until the line LargeData.ADDTEXT(xmlnode.text). And thats fine because of the 1024 limit ( in NAV5 i returned the xmldom ).

    But in ServiceTier ( called via a web service ) a get an error on line       xmlnode.nodeTypedValue := objStream.Read;.

    The error says "The type ClrByteArray is unknown".

    Do you have any idea why this error occures in ServiceTier?

  4. FreddyDK says:

    The error says it all – NAV 2009 doesn’t support the byte[] type from COM objects.

    It looks like you could use the encodefromfile function from this post – which would give you a string with the encoded file contents.

  5. Jut says:

    Great post again!

    Due to my missing experience with COM objects I do not know how to install/deploy that base 64 com component – I tried to build that project in visual studio and tried to look for dll files but had no success. Do I understand correctly that this COM Component needs to be installed on the development-machine (at least in case if I want to change the NAV Codeunit) and on each machine that is supposed to call that Webservice/Codeunit? Could you give a hint how that is possible?

    Thanks!

    Jut

  6. FreddyDK says:

    If you downloaded the VSDemo.zip – then the base64 project has a post build command which invokes RegAsm with the newly compiled assembly.

    This DLL needs to be on the ServiceTier and you need to use RegAsm to register the COM object.

    If you use C# on the clients you wouldn’t have to register the DLL – you could either make a reference to the project or you could copy the class.

  7. Jut says:

    Thanks your your quick response. The build process unfortunately errors out telling me that neither Input assembly base 62.dll nor dependable assembly could be found (translated to English). Furthermore it is telling me that regasm was terminated with code 100.

    Do you have any idea what I am doing wrong?

  8. FreddyDK says:

    You should be able to find out what reference isn’t available by looking in Visual Studio under references or under the error list.

    I am assuming that you are running Visual Studio 2008 with SP1

  9. Jut says:

    Actually I am using Visual Studio without SP1 – could that be the problem?

    The Error list unfortunately does not give me any additional information but "missing base 64.dll"

    I see the following references related to base 64 project:

    – System

    – System.core

    – System.data

    – System.Data.DataSetExtension

    – System.Xml

    – System.Xml.Linq

  10. FreddyDK says:

    I don’t know whether a missing SP1 is the problem, but the list you are referring to here are not references – they are using statements. You need to consult the Solution Explorer where you will find a list of references. The problem you are having seems very basic Visual Studio / C# – and of course you cannot register the bas64 assembly before you have built it.

  11. Jut says:

    It works now after restarting my computer – actually I do not know why it did not work before but at least it is working now. Thanks a lot!

  12. FreddyDK says:

    Sounds like you had the COM object loaded in either the Classic or the RTC. You need to close down clients before being able to rebuild the library.

  13. Jut says:

    Would it be possible to change the encodeFromFile function to work with files that are not stored locally but accessible via URL?

  14. Jut says:

    Hmmm, I tried a couple of things like creating a function  encodeFromUrl that looks like:

     public string encodeFromUrl(string url)

       {

           System.Net.WebClient wc = new System.Net.WebClient();

           Stream data = wc.OpenRead(url);

           StreamReader reader = new StreamReader(data);

           string s = reader.ReadToEnd();

           int len = (int)s.Length;    

           byte[] buffer = new byte[len];

           buffer = wc.DownloadData(url);        

           data.Close();

           reader.Close();    

           return System.Convert.ToBase64String(buffer);

       }

    The system actually imports something into NAV, unfortunately  my Classic-Client is not able to show that image (what is reflected by that small "no access"-sign.) However, if I export that picture it looks like the original picture. If I import the same image using your encodeFromFile fucntion, everything works fine.

    Any idea?

    Thanks

    Jut

  15. Jut says:

    I just double checked, looking at that picture from RTC however works hmmm.

  16. FreddyDK says:

    Classic only supports few image types.

  17. Sandeep.Chhillar says:

    Thanx for the great post mate..

    I tried this and successfully import the image/large text file form website (via nav web service) to nav’s blob filed. But When i tried to send a NAV’s blob field content to the webservice it gives me compilation error in "GetItemPicture" function.

    Error

     Picture.ADDTEXT(base64.decodeToFile(FileName));

    Comipation error on the above line saying  

    —————————

    Microsoft Dynamics NAV Classic

    —————————

    When the function is called, the minimum number of parameters should be used. For example:

    MyFunc( .. , .. , .. )

    ROUND(MyVar)

    ROUND(MyVar,0.05)

    —————————

    OK  

    —————————

    Can you pls update me where i am missing the parameter.

    Regards,

    Sandeep

  18. Stephen Abela says:

    Thanks for the info.  But what would happen if rather than a pic or a single field, I would like to pass over a whole record?  Is it possible without having to specify each field one by one?

  19. Kenneth Fuglsang Christensen says:

    Thanks for the inspiration on this Freddy. During my implementation I thought of another way to do this, without having to create or install the custom Base64-COM automation.

    I wrote a blog post about it on http://kfuglsang.com

  20. I rewrote your code using .NET interop instead. This is what I came up with:

    GetItemPicture(No : Code[20];VAR Picture : BigText)

    ———————————————————————–

    CLEAR(Picture);

    Item.SETRANGE(Item."No.", No, No);

    IF (Item.FINDFIRST()) THEN

    BEGIN

     Item.CALCFIELDS(Item.Picture);

     // Get Temp FileName

     TempFile.CREATETEMPFILE;

     FileName := TempFile.NAME;

     TempFile.CLOSE;

     // Export picture to Temp File

     Item.Picture.EXPORT(FileName);

     Picture.ADDTEXT(DNConvert.ToBase64String(DNFile.ReadAllBytes(FileName)));

     // Erase Temp File

     FILE.ERASE(FileName);

    END

    Where:

    DNFile = System.IO.File

    DNConvert = System.Convert

  21. Mayank` says:

    you have declared an Automation variable as "base64" . can u specify what is subtype of this automation variable. Because it is showing "Unknown Automation Server.Unknown Class"…..

    Regards:

    Mayank

  22. Peter says:

    Thank you very much Kenneth Fuglsang Christensen !

    Your solution did the trick for me in NAV2009 without any custom component.

    PS: Why have you made your solution so hard to find? I had to find it here: web.archive.org/…/kfuglsang.com

    /Peter – Aalborg 🙂