Merging Application Objects using Windows PowerShell in Microsoft Dynamics NAV 2015


Upgrading a Microsoft Dynamics NAV solution is time consuming. You have to identify which changes you have to make, you have to upgrade the application objects and the application code, and you might have to move the existing data around so that it fits the new database schema. In Microsoft Dynamics NAV 2013 R2 Cumulative Update 9, we introduced a new set of Windows PowerShell cmdlets that can help you through the code upgrade. In Microsoft Dynamics NAV 2015, the functionality and sample scripts have been further improved.

You can use the new cmdlets to modify application object source files in the Microsoft Dynamics NAV 2015 Development Shell, or by importing the Microsoft.Dynamics.NAV.Model.Tools.psd1 module into the Windows PowerShell Integrated Scripting Environment (ISE). The new application merge utilities install when you choose the Developer option in Microsoft Dynamics NAV 2015 Setup, or if you add the development environment to another installation option.

The application merge utilities include the following Windows PowerShell cmdlets:

Name

Description

Merge-NAVApplicationObject

Compares the changes that have been made between two sets of Microsoft Dynamics NAV application objects, and applies the difference to a third set of application objects. The result of the merge is a number of text files with the merged application objects. Any conflicts that the cmdlet cannot merge are identified in conflict files.

Compare-NAVApplicationObject

Compares text files that contain Microsoft Dynamics NAV application objects, and then calculates the delta between the two versions. The result of the comparison is a number of text files with the calculated delta.

Update-NAVApplicationObject

Applies a set of deltas to the specified application objects. The files that describe the delta are generated by the Compare-NAVApplicationObject cmdlet.

Join-NAVApplicationObjectFile

Combines multiple application object files into one text file

Split-NAVApplicationObjectFile

Splits a text file that contains two or more application objects into separate text files for each application object.

Get-NAVApplicationObjectProperty

Gets Microsoft Dynamics NAV application object properties from the specified application object text files.

Set-NAVApplicationObjectProperty

Sets Microsoft Dynamics NAV application object properties in the specified application object text files.

Export-NAVApplicationObjectLanguage

Exports captions from the specified text files with Microsoft Dynamics NAV application objects. The captions are exported to text files.

Import-NAVApplicationObjectLanguage

Imports strings in the specified language into text files that contain Microsoft Dynamics NAV application objects.

Join-NAVApplicationObjectLanguageFile

Combines multiple text files with captions for Microsoft Dynamics NAV application objects into one text file.

Remove-NAVApplicationObjectLanguage

Deletes captions in the specified language from Microsoft Dynamics NAV application objects.

Split-NAVApplicationObjectLanguageFile

Splits a text file that contains multilanguage captions for two or more application objects into separate text files for each application object.

Test-NAVApplicationObjectLanguageFile

Tests captions in Microsoft Dynamics NAV application objects to test to validate if they have translated strings for the specified languages.

 Getting started

You can read more about the cmdlets and how to use them in the MSDN Library, but you can also type Get-Help “NAV” in the Windows PowerShell ISE or the Microsoft Dynamics NAV Development Shell.

If you don’t want to use the Microsoft Dynamics NAV Development Shell, use the Windows PowerShell ISE. But before you can access the cmdlets, you must import the Microsoft.Dynamics.Nav.Model.Tools.psd1 module. Here is an example of the command you can type:

Import-Module “${env:ProgramFiles(x86)}\Microsoft Dynamics NAV\80\RoleTailored Client\Microsoft.Dynamics.Nav.Model.Tools.psd1” -force

Get-Help “NAV”

Now you can see the Help for the cmdlets and take a closer look at the examples for how to use them.

Now you can see the Help for the cmdlets and take a closer look at the examples for how to use them. You can also see detailed Help for each cmdlet by typing the following command:

Get-Help cmdletname -detailed

And you can concentrate on the examples by typing the following command:

Get-Help cmdletname -examples

 For all of the new cmdlets, the starting point is 3 versions of application objects that you want to merge. The following table describes the three versions of the Microsoft Dynamics NAV application that you want to compare and merge.

Version

Description

Original

The baseline of the application merge. For example, the Microsoft release of Microsoft Dynamics NAV 2013 R2.

Modified

The updated version of the original. For example, this can be your add-on.

In many cases, the modified application is the version that contains fewer changes to the original than the version that is the target of the merge. This is because you want to apply fewer changes to a large application rather than applying a large change to a small application.

Target

The version of the application that you want to apply the difference between the original and the modified application to. For example, this can be the new major release from Microsoft of Microsoft Dynamics NAV 2015 that you want to apply your modified solution to.

Each of these versions can be any version that you want to do a three-way merge between. ORIGINAL can be your add-on, MODIFIED can be a customization of your add-on, and TARGET can be a new release of Microsoft Dynamics NAV from Microsoft. But for the purposes of this blog post, we’ll keep the definitions as described in the table above.

As input to the cmdlets, you can provide a text file, a list of text files, or a folder with text files. So you need to export the relevant application objects as text files. Optionally, you can use the development environment command ExportObjects. You can export each application object to a separate text file, or you can export all objects to a single text file. Optionally, you can use the Join-NAVApplicationObjectFile and Split-NAVApplicationObjectFile cmdlets to structure the text files in the way that works better for you. Also, depending on your scenario, you can work with a subset of your application, such as all codeunits, objects within an ID range, or a specific group of objects. Use the tools to get the text files that you need, and take a look at the sample scripts for inspiration.

The Windows PowerShell sample scripts are available on the product media. Start by opening the HowTo-Start-Import-NAV-Module.ps1 script in the Windows PowerShell IDE, navigate the command prompt to the folder where you placed the samples, and then run the script. Then open one of the other scripts, such as HowTo-Merge-1-General.ps1, and follow the guidance in the script.

We suggest that you open each of the sample scripts in the Windows PowerShell IDE and read through them to get acquainted with the new cmdlets. Then, set up a small test environment of your own where you can safely use the cmdlets on your own application objects to upgrade your solution to Microsoft Dynamics NAV 2015.

 

Comments (30)

  1. js@agidon.dk says:

    These cmdlets rocks.

    But, what is new in 2015?

    Does the merge noew handle the Version tag which was left out in 2013 R2 and had to be handled by separate scripts?

  2. The performance of the split cmdlet hasn't improved, still slows down drastically at the end for the last few objects in a bunch of 4340, the same behaviour which I already commented here:  blogs.msdn.com/…/10539409.aspx

  3. PowerShell ISE gives a warning about "unapproved verbs" when importing the new Microsoft.Dynamics.Nav.Model.Tools.psd1 version each time it is started, due to the new COMPILE,CREATE and DELETE cmdlets.

    The warning can be ignored; to suppress it you can add "-DisableNameChecking" as an additional parameter at the end of the Import-Module line.

  4. Pallea says:

    Also remember if the Import-Module does not work just do a

    Set-ExecutionPolicy RemoteSigned

    before the Import-Module command.

  5. The known issue the split cmdlet has with square brackets still exists.

    If you split an object file with Report 117 you get this result in all array fields, a surplus bracket on the left and two on the right.

    Before:

       { 46  ;2   ;Column  ;CustAddr8           ;

                  SourceExpr=CustAddr[8] }

    After:

       { 46  ;2   ;Column  ;CustAddr8           ;

                  SourceExpr=[CustAddr[8]]] }

    Luckily, when the report is imported into NAV 2015, the surplus brackets are removed again, the data source field in the report shows CustAddr[8] again. But if need to compare the files manually to confirm the merge result that bug is a nuisance.

    The are also some new cmdlets missing in the list, like the new cmdlet Export-NAVApplicationObject. That is much easier to use than switching to the command prompt to engage the ExportObjects command, and more versatile too, because you can omit unlicensed objects with the switch "-ExportTxtSkipUnlicensed".

  6. Claus says:

    When I used the Merge-NAVApplicationObject the result folder contains all NAV Objects. In my case 87 objects were merged, 4 objects had conflicts, 74 objects were inserted and 4215 objects were unchanged. I am only interested in the 87 objects that were merged and the 4 objects that had conflicts. Is the a way to identify these objects from the Result folder? Thanks

  7. Francois Jaccard says:

    Hi,

    I am trying to upgrade our NAV2013R2 to NAV2015 and I am following the MSDN articles but on "Upgrading the Application Code" (msdn.microsoft.com/…/dn271652%28v=nav.80%29.aspx) at the end I must "import into an existing, empty database". What do you mean by existing, empty database? I tried to create a new one but I can't import due to permission error to create objects. I tried to import into the "Demo Database (8-0)" but I am getting several errors on variables having the same ID. I removed the objects but I get "You do not have permission to create the 'BOM Ledger Entry' Table". I do have a dev license installed. I would like to automate the upgrade process as much as possible instead of manually modifying the objects to add the custom modifications. I am using a Swiss version if it matters.

  8. Pallea says:

    @Francois Jaccard

    What I did myself, was:

    Go into the Dvelopment Client, and do a File, Database, New, then make sure that you afterwards says Tools, License, Upload (and upload a License file).

    Then open the database with your Windows Client, and verify that your license file is correctly uploaded.. (Help, About, View your Licensefile information)

    If you have a partner licensefile the you should NOT have any issues with importing objects (actually you should never have any issues if you are importing fob files).

    /Palle

  9. Francois Jaccard says:

    @Pallea

    I did exactly that: create a new db, upload the dev license. I rechecked that I have a dev license. I import the all-merged.txt (not fob) created via the cmdlets and I am still getting the permission error 🙁

    Is it possible to create a fob instead of all the text files?

  10. Pallea says:

    @Francois Jaccard

    The error means that you do not have "access" a object in "sourcecode". Did you use the same License-file to export to *.Txt?

    Verify if the table 238 exist in a standard 2015 database if not then just skip the table in your code-upgrade. I Can see the table in Dynamics NAV 2009 but not in NAV 2013…

  11. Francois Jaccard says:

    @Pallea

    Yes I exported with the same license. I am getting the error on creating the 1st object in the all-merge.txt which is Report 1 "Chart of Accounts" and it is in NAV2015.

  12. Pallea says:

    Hello again @Francois Jaccard

    Ahh, your "problem" is more than just one table..Your problem is clearly a Licensefile issue.

    You are trying to import standard objects (IDs below 50000) as text which is not possible with our licensefile. You kinda got two options

    1) Import all standard objects to your new database (as fobs) then import the text objects and compile objects again (The problem is that your license-file does not allow you to CREATE objects below 50000 – which is quite normal). Objects that does not exist as a compiled object which results in a CREATE must be skipped.

    OR

    2) Remove all text-objects below 50000 and only imports above this ID.

  13. Morten Jensen (MSFT) says:

    Thanks for all the valued feedback on the Application Merge Utilities. Let’s me address the comments one at a time.

    First, what’s new in the 2015 release. Well, basically two parts:

    FINSQL.EXE WRAPPERS

    ——————-

    These makes finsql.exe act better in a PowerShell script, and shields some peculiarities of the command line parameters present:

       Compile-NAVApplicationObject

       Delete-NAVApplicationObject

       Export-NAVApplicationObject

       Import-NAVApplicationObject

    Use Get-Help on any of these functions.

    As noted by @Kai, these trigger a PowerShell warning when loaded. While odd at first, this is a deliberate choice, since these are wrapper functions as opposed the fully tested cmdlets. You can find and change the source code yourself. We decided to stick to the established terms like “compile” as opposed to the approved verbs (do Get-Verb for approved list).

    HANDLING CAPTIONS

    —————–

    Functionality for exporting, removing, importing and testing captions/languages in TXT files, plus splitting and joining translation files for completeness:

       Export-NAVApplicationObjectLanguage

       Import-NAVApplicationObjectLanguage

       Remove-NAVApplicationObjectLanguage

       Test-NAVApplicationObjectLanguage

       Split-NAVApplicationObjectLanguageFile

       Join-NAVApplicationObjectLanguageFile

    Use Get-Help on any of these cmdlets.

    // Morten

  14. Morten Jensen (MSFT) says:

    Filtering the output of the Merge Cmdlet – and using the Sample Scripts provided.

    @Claus asks how to only look at files merged? To answer that, I’d like to encourage you to visit the samples scripts provided with the product. They are not installed as such, but are on the product media in the folder

           WindowsPowerShellScriptsApplicationMergeUtilities

    If you change to that folder in either the NAV Development Shell, or the PowerShell Integrated Script Environment (ISE), see description in blog text above, you can run any of the scripts showcasing the utilities using the provided demo data.

    From the script

            HowTo-Merge-1-General.ps1

    a slightly modified script launches the files of the MERGED TXT files in the GridView:

    Merge-NAVApplicationObject -OriginalPath .demodataORIGINAL*.txt `

                              -TargetPath .demodataTARGET*.txt `

                              -ModifiedPath .demodataMODIFIED*.txt `

                              -ResultPath .demodataRESULT -Force -PassThru |

       Where-Object MergeResult -eq 'Merged' |

       Select Original, Target  |

       Out-GridView

    // Morten

  15. Pallea says:

    Just to clarify one very important thing. The Powershell Cmdlets are AWESOME!! BUT, there is one very important but here. If you don't know how to do a regular manual upgrade then these cmdlets are not for you!!

    Please either read the training material or attend a training class. If upgrades are done the wrong way, you are in risk of lawsuits, projects prolonging unneccesarily etc. etc.

    Take a closer look at:

    Course 80549A: Data Upgrade and Code Upgrade to Microsoft Dynamics NAV 2013

    And at:

    msdn.microsoft.com/…/dn271649(v=nav.71).aspx for upgrading to NAV 2013R2 –

    the upgrade to 2015 are quite simpel and can be followed at:

    msdn.microsoft.com/…/dn466744(v=nav.80).aspx

  16. Morten Jensen (MSFT) says:

    @Kai

    The issue with excessive square brackets is recognized and will be fixed.

    // Morten

  17. Morten Jensen (MSFT) says:

    With regards to the performance hick-ups mentioned, I saw the apparent issue with some of the cmdlets, e.g. Split-NAVApplicationObject, in a workshop at Directions recently. I have seen similar behavior once, but cannot reproduce :-(

    If *anyone* can help narrow it down somehow, I’d be very happy. What I tried was below script (merging the NA version of NAV 2013 R2 into NAV 2015) as well as joining and splitting files. Approx. 4000 files in each version folder. I monitored the Task Manager / Resource Monitor for CPU, Memory, and Disk I/O. But no true hint for bottlenecks.

    How is you monitoring?

    // Morten

    —sample-code-w/all-folders-created-in-advance—————————————————————————–

    $original = "C:testnav71rtmW1"

    $modified = "C:testnav71rtmNA"

    $target = "C:testnav80rtm"

    $result = "C:testresult"

    $misc = "C:testmisc"

    Merge-NAVApplicationObject -OriginalPath $original -ModifiedPath $modified -TargetPath $target -ResultPath $result -Force

    Join-NAVApplicationObjectFile -Source $target -Destination "$resultjoined-file.txt" -Force

    Split-NAVApplicationObjectFile -Source "$resultjoined-file.txt" -Destination $misc -Force

  18. Morten Jensen (MSFT) says:

    @Johannes, thanks for the nice words 

    Please find below an example on how to implement custom VersionList-handling. This was presented at Directions recently and will be included in the sample script library moving forward. Please copy and paste the script into the PowerShell ISE for true formatting and syntax-coloring.

    Basically it calls a custom function for each of the objects processed by Merge-NAVApplicationObject, a function you can build or modify and in this example named “New-NAVVersionList”. Refer to other sample scripts for filtering before the foreach-loop, if needed.

       Merge-NAVApplicationObject -OriginalPath .ORIGINAL -ModifiedPath .MODIFIED -TargetPath .TARGET -ResultPath .RESULT –Force |

           foreach { Set-NAVApplicationObjectProperty -Target $_.Result `

                          -VersionListProperty (New-NAVVersionList $_ ('NAVW1',’YOURTAG’)); `

                     Get-NAVApplicationObjectProperty -Source $_.Result `

           }

    // Morten

    —sample-code-for-implementing-versionlist-handling ——–

    <#

       .SYNOPSIS

       Creates version list string based on the three inputs of a merge operation (original, modified, target).

       The returned version includes the "highest" version for each prefix and any other non-prefixed versions.

       This script does a 3-way merge and then updates the version list in the resulting objects.

       .EXAMPLE

          $mergeInfo = Merge-NAVApplicationObject …

          $mergeInfo | Foreach-Object { Set-NAVApplicationObjectProperty $_.Result -VersionListProperty (New-NAVVersionList $_ ('MyTag', 'NAVDEU')) }

    #>

    function New-NAVVersionList($MergeInfo, [string[]]$VersionPrefix = 'NAVW1')

    {

       $allVersions = @()

       if ( $mergeInfo.original.versionlist -ne $null )

           { $allVersions += $mergeInfo.original.versionlist.Split(',') }

       if ( $mergeInfo.modified.versionlist -ne $null )

           { $allVersions += $mergeInfo.modified.versionlist.Split(',') }

       if ( $mergeInfo.target.versionlist -ne $null )

           { $allVersions += $mergeInfo.target.versionlist.Split(',') }

       $mergedVersions = @()

       foreach ($prefix in $VersionPrefix)

       {

           # add the "highest" version that starts with the prefix

           $mergedVersions += $allVersions | where { $_.StartsWith($prefix) } | sort | select -last 1

           # remove all prefixed versions

           $allVersions = $allVersions.Where({ !$_.StartsWith($prefix) })

       }

       # return a ,-delimited string consisting of the "highest" prefixed versions and any other non-prefixed versions

       $mergedVersions += $allVersions

       $mergedVersions -join ','

    }

    $result = Merge-NAVApplicationObject -OriginalPath .ORIGINAL -ModifiedPath .MODIFIED -TargetPath .TARGET -ResultPath .RESULT -Force

    $result |

       foreach { Set-NAVApplicationObjectProperty -Target $_.Result `

                          -VersionListProperty (New-NAVVersionList $_ ('NAVW1',’YOURTAG’)); `

                 Get-NAVApplicationObjectProperty -Source $_.Result `

               }

  19. Bartel de Leeuw says:

    Hi Morten,

    chiming in on the performance issue. As noted by Kai previously, it seems that the first part of the split on larger files is fast, and it slows down the closer it gets to the end. In order to eliminate as much as possible, I ran the split on two objects only, codeunits 1 and 2 from the 2015 NA database. I am using the following code:

    ————————————————————————————————-

    set-executionpolicy unrestricted -force

    Import-Module "${env:ProgramFiles(x86)}Microsoft Dynamics NAV80RoleTailored ClientMicrosoft.Dynamics.Nav.Model.Tools.psd1" -force

    Split-NAVApplicationObjectFile -Source c:upgradePerformancetarget.txt -Destination c:upgradePerformanceTarget -force

    ————————————————————————————————-

    The split on these two codeunits takes anywhere between 60 and 90 seconds. I am running 64bits Powershell 4.0 through ISE on a Windows Server 2008 R2 with latest updates installed. The server has 4 cpus and 6.5 GB of RAM. Eliminating ISE does not change the outcome. There is no noticable strain on CPU, memory or drive during the time the system is attempting the split.

    Let me know if you need any more information, we appreciate you looking into this. We are excited about the tools, and we hope you can help us move past this issue.

    Bartel

  20. Morten Jensen (MSF) says:

    @Bartel, can I ask you to try using parameter -PreserveFormatting to see if it makes a difference? It's a mystery…

    Have you seen this slow-pattern using any of the other cmdlets?

  21. Bartel de Leeuw says:

    Hi Morten,

    no difference on the first run, time was still around 90 seconds. The second run took less than a second, the third, fourth and fifth runs were back to 90 seconds. Not sure what to make of the second run.

    The effect can be seen on the Split and the Join, Compare and Merge, and is especially noticable on the Set-NAVApplicationObjectProperty, probably because that process typically gets run for each object seperately. When I do a typical version list update after a merge, it will run for hours on a large set of objects.

    I have also tested Powershell 2.0 and 3.0, but that does not seems to make a difference.

    Anything else you can think of?

    Thanks for your help,

    Bartel

  22. @Morten

    Thanks for tackling the square bracket issue.

    Another issue I have found in the split-cmdlet is that names of variables or caption lables in reports which include special characters like äüö etc. are encapsulated in quotation marks. These are also removed by the import routine, so you will not find them in the NAV objects afterwards, but in the object text files these differences do show up.

  23. Morten Jensen (MSFT) says:

    Hi @Bartel, if you don't mind helping digging into this issue, could you contact us by email using the [Email Blog Author] link in the upper left corner of this page? We surely would like to nail this issue once and for all.

    // Morten

  24. @Morten

    I'm trying out the New-NAVVersionList function at the moment and, watching that, it may well be that the bottleneck actually is something in the "Writing File" function. Though it does not always happen there, some "Writing File" processes in the course of this function do show this performance slump shortly before the finish line.  

  25. Morten Jensen (MSFT) says:

    Provided no unforeseen issues hit us, three fixes are planned for the NAV 2015 cumulative update to be made available in early December. They cannot make it for this month's cumulative update:

    *   The OCX property does not preserve variable subtype.

    *   Brackets [] are added for pages and reports where the SourceExpr includes Arrays.

    *   Severe performance degradation running Application Merge Utilities cmdlets on some machines, factor 15 or worse.

    Thanks for sharing and helping getting to this,

    Morten

  26. Bartel de Leeu says:

    @Morten,

    thanks for getting to the bottom of this. Looking forward to the new and improved tools.

    Bartel

  27. John O'Donovan says:

    I am using the Remove-NAVApplicationObjectLanguage cmdlet to remove language captions from my text file (leaving ENU captions intact) and I have found that all of the captions are removed except for the PromotedActionCategoriesML value.  I have removed these manually for the moment but was wondering if this is a known issue and if it can be updated in future versions

    John?

  28. Morten Jensen (MSFT) says:

    Hi @John,

    The issue with PromotedActionCategoriesML is known and reported and is part of the cumulative update for Dynamics NAV 2015 being released in the beginning of December.

    // Morten

  29. @Morten

    All the performance issues which I experienced on my system are gone with new versions supplied with NAV 2015 Cumulative Update 2. This also applies to other cmdlets like Get-NAVApplicationObjectProperty which could take anything from 3 to 30 seconds in the buggy old version and now only some milliseconds. Thanks for fixing these issues.

    For the moment, as far as I can tell, only one minor bug remains: The double quotes which are added to variables or report labels with special characters after the Split-Cmdlet is applied to an object file.

    Example: MyVarPräfix -> "MyVarPräfix"

  30. Have these cmdlets been tested with PowerShell 5 (Windows 10)? I just used the Split cmdlet on an object pack and all Pages are ruined, in every page the code stops abruptly in Line 11 after CaptionML=[.  

Skip to main content