Migrate Sharepoint document with version to Sharepoint online using CSOM


It has been observed quite  a few time that METALOGIX failed to migrate large document library content with version enabled or library using managed metadata. Very often it gets timed out exception during the operation.

To overcome this issue in one of our migration project we used CSOM to migrate the documents with versions.

Using the following code piece we can migrate the documents with their version from one platform to the other one.

 

1. Prepare the source and target client context here I am assuming target is sharepoint online

 ServicePointManager.ServerCertificateValidationCallback += customXertificateValidation;
ClientContext sourceContext = new ClientContext(sourceSiteUrl);
sourceContext.Credentials = new NetworkCredential(sourceUserName, sourcePassWord, sourceDomain);

ClientContext targetContext = new ClientContext(targetSiteUrl);
var passWord = new SecureString();
foreach (char c in targetPassWord.ToCharArray()) passWord.AppendChar(c);
targetContext.Credentials = new SharePointOnlineCredentials(targetUserName, passWord);
2. Get the source folder
 Folder sourceFolder = sourceContext.Web.GetFolderByServerRelativeUrl(sourceFolderUrl); 
sourceContext.Load(sourceFolder, fldr => fldr.Name);
sourceContext.Load(sourceFolder.Files, files => files.Include(
file => file.ListItemAllFields
));

try
{
sourceContext.ExecuteQuery();
}
catch (Exception e)
{

}
3. Get the target folder
 Folder targetSiteFolder = targetContext.Web.GetFolderByServerRelativeUrl(targetFolderUrl);
targetContext.Load(targetSiteFolder.Files, files => files.Include(
file => file.ListItemAllFields
));
targetContext.Load(targetSiteFolder);

try
{
targetContext.ExecuteQuery();
}
catch (Exception ex)
{

}
4. Do the migration operation
 int fileCount = sourceFolder.Files.Count();
for (int i = 0; i < fileCount; i++)
{
Microsoft.SharePoint.Client.File sourceFile = sourceFolder.Files[i];
FileVersionCollection sourceFileVersions = sFile.Versions; sContext.Load(sourceFileVersions, fileVersions => fileVersions.Include( version => version.Url, version => version.VersionLabel, version => version.CheckInComment )); try { sContext.ExecuteQuery(); } catch (Exception e) { //handle exception   
 }
foreach (FileVersion fileVer in sourceFileVersions)
{ UploadFiles(sContext.Url + "/" + fileVer.Url, targetFolder, sourceItem["FileLeafRef"].ToString());
}
}
5.
private static void UploadFiles(string sourceFileUrl, Folder targetFolder, string fileName)
 {
 FileCreationInformation targetFileVersionCreationInfo = new FileCreationInformation(); targetFileVersionCreationInfo.Overwrite = true; try { WebRequest request = HttpWebRequest.Create(sourceFileUrl); request.Credentials = sContext.Credentials; using (WebResponse response = request.GetResponse()) { using (Stream stream = response.GetResponseStream()) { byte[] verBuffer = new byte[32768]; using (MemoryStream versionMS = new MemoryStream()) { int read; while ((read = stream.Read(verBuffer, 0, verBuffer.Length)) > 0) { versionMS.Write(verBuffer, 0, read); } versionMS.Seek(0, SeekOrigin.Begin); targetFileVersionCreationInfo.ContentStream = versionMS; tContext.RequestTimeout = System.Threading.Timeout.Infinite; targetFileVersionCreationInfo.Url = targetFolder.ServerRelativeUrl + "/" + fileName; Microsoft.SharePoint.Client.File targetVersionFile = targetFolder.Files.Add(targetFileVersionCreationInfo); try { tContext.ExecuteQuery(); } catch (Exception ex) { //handle exception   
 } } } } } catch (Exception ex) { //handle exception   
 }
 }
 
The above piece of code is able to migrate the document with version but it has few issues 
Issue 1. The created date,modified date,credated by and modified by field will be changed to keep these field intact we need to do the following
 List list = targetContext.Web.Lists.GetByTitle(fldr.Name);
ListItem item = file.ListItemAllFields;
targetContext.Load(list, listObject => listObject.EnableVersioning, listObject => listObject.EnableMinorVersions);
targetContext.Load(item, itemObject => itemObject.File);
targetContext.ExecuteQuery();
[PLACEHOLDER 1]
targetContext.ExecuteQuery();
FieldUserValue targetCreatedUser = new FieldUserValue();
targetCreatedUser.LookupId = createdUser.Id;
FieldUserValue targetModifiedUser = new FieldUserValue();
targetModifiedUser.LookupId = modifieddUser.Id;
item["Editor"] = targetModifiedUser;
item["Author"] = targetCreatedUser;
item["Created"] = created;
item["Modified"] = modified;


item.Update();
[PLACEHOLDER 2]
 targetContext.ExecuteQuery();
Issue 2: In attempt to keep the created and modified information intact the code internally create additional version as the library is enabled with version changes.
To resolvethat we need to put the following code in PLACEHOLDER1 and PLACEHOLDER 2 subsequently
PLACEHOLDER1
 var isVersionEnable = list.EnableVersioning;
list.EnableVersioning = false;
 list.Update(); 
PLACEHOLDER 2 
 list.EnableVersioning = true; 
list.Update();
 
Issue 3: With the above fixes we are done with migrating documents with versions for the libraries which are version enabled and Forcecheckout checkin option set as false.
 If the library is configured for Forcechecckout then with the above piece of code execution we will be end up with documents with latest version all the previous version info will not be available. To overcome this issue we need to do a small hack before staring actual migration code.
We need to set the forcecheckout option for the target library as false.
  targetList.ForceCheckout = false;
and at the end of migrating all the doc just set it back with actual settings.
Comments (3)

  1. Joe Zhou says:

    Great article! Thanks for sharing!

    One issue I encountered is

    WebRequest request = HttpWebRequest.Create(sourceFileUrl);

    request.Credentials = sContext.Credentials

    It throws 403 error. I have to use something like

                   HttpWebRequest request = (HttpWebRequest)WebRequest.Create(sourceFileUrl);

                   request.Credentials = sContext.Credentials;

                   request.Headers.Add("X-FORMS_BASED_AUTH_ACCEPTED", "f");

                   request.Method = "GET";

  2. Joe Zhou says:

    Another issue is if there is more than 5000 subfolders, the code will fail silently (0 subfolder returned). CAML query will need to be used

  3. Michele says:

    How do I run these commands/scripts from powershell?

Skip to main content