SmartMapper - Step-by-Step Guide 

Previous Top Next


Every large document migration—whether it is a SharePoint tenant-to-tenant migration, file share to SharePoint Online, SharePoint on-premises to Microsoft 365, or cloud-to-cloud migration—follows the same general workflow.

Most migration tools are responsible for moving documents from the source to the target location. However, repairing broken links inside those documents is usually a manual, time-consuming, and error-prone process.


SmartMapper automates this entire process.

All you need to provide are your migration mapping rules (source -> target locations). SmartMapper analyzes millions of links, generates optimized Search & Replace keys, validates them, and prepares everything for ReplaceMagic to repair your documents automatically.


The complete workflow consists of the following steps.



Step 1 – Configure the SmartMapper Database


Note: If you have already used the option to save runs to the database, we strongly recommend creating a new database (Configuration -> DB Wizard) due to recent database schema changes. 
ReplaceMagic will attempt to upgrade the existing databse automatically. However, to avoid any potential compatibility issues, we recommend creating a new database and using it to store your SmartMapper data.


SmartMapper stores scan results, generated Search & Replace keys, and analysis information in a database.

Supported databases are:

  • SQL Server
  • MySQL
  • SQLite


Configure the database via:

ReplaceMagic -> Configuration -> Database Configuration



This is normally a one-time configuration.

If you enable Use SmartMapper before configuring a database, ReplaceMagic will automatically guide you through the required setup.

Whenever a newer ReplaceMagic version requires a database upgrade, the application performs the upgrade automatically while preserving all existing SmartMapper data.

Step 2 – Scan Your Documents


Scan the documents you want to repair.

The scan can be performed:

  • before migration (source environment)
  • after migration (target environment)


After scanning, save the scan results into the SmartMapper database.

There are two ways to do this.

Tip - linked sites in one run. If documents link to documents on other SharePoint sites, add all these sites in the SharePoint Selection form and scan them in one run. All documents are then in the same run, so SmartMapper can resolve ID-based links between the sites without additional runs.

Option 1 – Export the Scan Manually

Click Export Run after the scan completes.



Option 2 – Enable SmartMapper (Recommended)

Simply enable Use SmartMapper before starting the scan.


ReplaceMagic will automatically save the scan results after scanning finishes.
This is the recommended approach because no additional action is required.

Trial Mode

If you are using the ReplaceMagic trial version, or your license does not include SmartMapper, enabling Use SmartMapper automatically starts the SmartMapper trial.

During the trial, up to 100 documents can be processed.



Optional - scan the migrated location as well. If the content has already been migrated, also scan the new location (for example the target SharePoint library) and save it to the same database. The Ultra fast scan option is sufficient, because only the list of files is needed. SmartMapper then uses this scan to verify your mapping rules and to find folders the migration tool renamed (see Step 4.5).

Step 3 – Prepare Your Mapping Rules


SmartMapper needs to know how documents were moved.
Provide a list of mapping rules that describe the relationship between the original locations and the new locations.
In most migration projects, these mappings already exist because they are required by the migration tool itself. Many migration solutions also generate migration reports that contain exactly this information.

The required format is:

Source|Target


One mapping per line.

Example:

\\ServerABC\Finance|https://contoso.sharepoint.com/sites/Finance
       \\ServerABC\HR|https://contoso.sharepoint.com/sites/HR


The separator character is | by default.
Although it can be changed in ReplaceMagic Configuration, using the default separator is strongly recommended.

Important

Make sure:

  • there are no leading or trailing spaces
  • both source and target use consistent paths
  • if one path ends with "/" the corresponding path should also end with "/" (same applies also for "\"). Depending on source/target type one might end-up with "\" and other with ""/". Key is that ending has to be of the same type (ending with any type of slash or ending without slashes).

Small formatting differences can prevent automatic matching.

SmartMapper checks your rules. When the keys are created, SmartMapper checks the rules against the scan and writes a Mapping rules check section to the log. A typical mistake it catches: the rule Z: -> https://contoso.sharepoint.com/sites/Finance/Shared Documents with a scan of Z:\Finance sends every document to .../Shared Documents/Finance/.... If only the contents of Z:\Finance were migrated into the library, the correct rule is Z:\Finance -> .../Shared Documents - otherwise ReplaceMagic will not find any key for the migrated documents.

Step 4 – Generate SmartMapper Search & Replace Keys

Open the main Search & Replace page and click:

Search & Replace



Then click:

SmartMapper



The SmartMapper wizard will open.

4.1 Select the Scan

Choose the previously saved scan that should be analyzed.



4.2 Load the Mapping Rules

Paste the mapping rules directly or import them from a text file.



Option "Align source and target path endings" will ensure that path endings between source and target are always of the same type:



If you are entering source and targets having same endings (with or without slashes) this option is not required.

IMPORTANT:  a mapping rule must say where the CONTENTS of a folder were copied

The Source of a rule is the folder whose contents were copied, and the Target is the place they were copied to. SmartMapper builds each new address by replacing the Source part of the old path with the Target and keeping the rest of the path unchanged.

Example: you scanned Z:\FolderForMigration, and its contents (the subfolders and files inside it) were copied to https://company.sharepoint.com/sites/finance/Shared Documents.



A rule for a whole drive (Z: -> …) is correct only if the drive was copied together with its folder structure, so that FolderForMigration exists as a folder in the target.

Add a rule with the same Target for every other drive or share that points to the same folder, for example G:\FolderForMigration or \\server\share\FolderForMigration. Do not add rules for folders that were not migrated: links to them should stay unchanged.

Tip: SmartMapper checks your rules before it creates the keys. If the scanned folder’s name would be kept in the target, it shows a [CHECK] with the rule to add. If the database also contains a scan of the migrated location (Ultra fast scan is enough), it corrects such a rule automatically.

4.3 Configure Optional Filters

You can exclude links that should never be processed.
Examples:

  • backup
  • archive
  • obsolete locations


 
Filters can be applied to links that:

  • begin with
  • contain
  • end with

the specified text.

This helps eliminate obsolete or intentionally ignored links from the analysis.

4.4 Generate Search & Replace Keys

Special characters in folder names. Before you start, answer the question at the bottom of the page: Did the migration replace special characters in folder names (~ # % & ...)? Many migration tools replace characters that SharePoint does not accept or handles badly, for example ~Rec Logs becomes _Rec Logs. The answer is saved and written at the top of the log; the line under the question shows what it means with your current settings in Configuration -> SharePoint 2.

  • I don't know - SmartMapper warns about keys that go through folders with such characters. A scan of the target location (Ultra fast scan is enough) in the same database shows how the migration named them.
  • Yes - ReplaceMagic replaces these characters itself during the replacement, as set in Configuration -> SharePoint 2 (Replace SharePoint illegal characters? for " * < > | ?, and Use optional illegal characters? for ~ # % & { }). If a setting is missing, the line under the question says so in red.
  • No - no warning. If Configuration -> SharePoint 2 would still replace the characters, the line under the question tells you to turn that off, otherwise names the migration kept would be changed.

Click Create keys. SmartMapper will automatically:

  • analyze every discovered link
  • check your mapping rules against the scans (Mapping rules check in the log) and add rules that are backed by evidence
  • compare every link with the mapping rules
  • detect matching migration patterns
  • generate optimized Search & Replace keys
  • identify conflicts
  • prepare everything required for ReplaceMagic

The progress and the complete report are shown in the log; Export processing data saves it. When the keys are created, the line next to the buttons gives the result, for example Keys created! 6 suggested rule(s) added, 6 rule(s) to check - see next page. If keys have already been generated previously, SmartMapper asks before it regenerates them.

SmartMapper - Start search &amp; replace keys creation

Message about special characters. When keys go through folder names with special characters, a message appears after the keys are created. It names the folders and the number of keys and files:

  • Answer Yes: these characters will be replaced by ReplaceMagic during the replacement. Folders the migration did NOT rename will then be wrong - make sure the scan of the target location covers these folders and create the keys again to verify.
  • Answer I don't know: ReplaceMagic cannot know the new names of these folders. Recommended: scan the target location (Ultra fast scan is enough) in the same database and create the keys again - renamed folders are then detected automatically.
  • If the scan of the target location shows such folders UNCHANGED while ReplaceMagic would replace their characters, the message warns that these links will be wrong: turn the conversion off in Configuration -> SharePoint 2, or replace these documents separately.

SmartMapper - Special characters in folder names

Performance. Even for very large environments, SmartMapper is fast. For our test case (1 million documents, 12 million links) the processing time on the ReplaceMagic test computer (Intel i9 with 32GB RAM) was approximately 25-30 minutes. Performance depends on your hardware and on the number and complexity of your documents, links, migration rules and filter patterns.

4.5 Review Suggested Mapping Rules and Rules to Check

Before generating the keys, SmartMapper checks your mapping rules against the scans (see the Mapping rules check section in the log). Right after the keys are created, the wizard shows a page with two lists. The page is skipped when both lists are empty.

SmartMapper - Suggested mapping rules and rules to check

Suggested mapping rules (upper list). Rules that are backed by evidence are added automatically - for example the correct rule when only the contents of a folder were migrated (realign), a rule for each folder the migration tool renamed (renamed, ~Rec Logs -> _Rec Logs), or a rule for another drive or share that points to the same folders as the scanned one (alias). Your own rules are never changed. The list shows the kind of each rule and the evidence for it; the keys were already created with these rules.

  • Accept selected / Accept all - keep the rules. Their origin becomes Accepted.
  • Delete selected - remove rules you do not want.

Rules to check (lower list). Your existing rules that map no scanned document and are probably wrong or not needed:

  • local paths - most of the rule's links are local user or system paths (C:\Users, C:\Documents and Settings, C:\Program Files ...). Mapping them to the cloud breaks links that work on the users' computers.
  • outside scan - all of the rule's links point outside the scanned folders, so their target cannot be verified. Keep the rule only if these folders really were migrated to its target. (Links that the rule maps to a document of the scan count as verified - for example when you scanned the migrated documents on SharePoint and their links still point to the old file server.)
  • superseded - the rule mapped documents before, more specific rules map them now; it may still map a few links.
  • unused - the rule matches no document and no link of this run (not shown when the run was limited, for example in the trial).

Links is the number of links the rule maps, Files only the number of files that are changed only because of this rule. Nothing is selected or deleted automatically: select the wrong rules and click Delete selected rule(s).

Both lists show the full text of every column; scroll horizontally to read long evidence. Export both lists saves them in one file: in Excel one worksheet per list (Suggested rules, Rules to check), in CSV one block per list - useful to discuss the rules with the people who ran the migration.

Apply and recreate keys. The current keys still contain the deleted rules. Click Apply and recreate keys (or answer Yes when you continue to the next page): the keys are recreated without the deleted rules and without adding suggestions again. Deleting only unused rules does not require new keys.

In the mapping rules list (4.2) the column Origin shows for every rule whether you added it (User), SmartMapper added it (Suggested) or you accepted a suggestion (Accepted).


4.6 Review the Generated Keys

Review generated Search & Replace keys. For performance reasons (and hardware resources consumption) up to 250k keys 
are shown here and to see all keys export them in Excel or CSV file.



You can:

  • review them directly inside SmartMapper
  • export them to Excel or CSV
  • Drag and drop column headers in "Drag a column here to group by this column"
  • import the updated Excel file again

When importing an edited Excel file:



Select whether you want to replace the existing Search/Replace keys or append additional keys from an Excel file.

  • Insert mode deletes all Search/Replace keys previously generated for the selected RunID and imports the keys from the Excel file.
  • Append mode keeps the existing Search/Replace keys and adds the imported keys to the selected RunID.

If you have not modified first 4 rows of the exported Excel file (which is strongly recommended), you can keep the default Skip Rows and Skip Columns settings. The first four rows contain header information and should not be imported. Likewise, the first column (RowID) is skipped by default, allowing you to import Search/Replace keys from one RunID into another without preserving the original database row identifiers. ReplaceMagic automatically assigns new RowIDs for the selected RunID during the import.

4.7 Review Successfully Resolved Links

SmartMapper shows every link that will be repaired. For performance reasons (and hardware resources consumption) up to 250k
resolved links are shown here and to see all keys export them in Excel or CSV file.

You can:

  • review them directly inside SmartMapper
  • export them to Excel or CSV
  • Drag and drop column headers in "Drag a column here to group by this column"



4.8 Review Unresolved Links

Not every link can always be repaired automatically.

The Unresolved Links page lists every remaining unresolved link. For performance reasons (and hardware resources consumption) 
up to 250k unresolved links are shown here and to see all keys export them in Excel or CSV file.




This allows you to:

  • investigate missing mapping rules
  • identify invalid links
  • improve your mapping before running ReplaceMagic
  • export them to Excel or CSV
  • Drag and drop column headers in "Drag a column here to group by this column"


4.9 Final step

Documents to process. The last page shows the result for the run at the top - the number of Search & Replace keys and of documents with keys (and the coverage, if you created the keys in this session). Below it, choose which documents the replacement processes:

SmartMapper - Documents to process

  • All documents (default) - every document in the current list is opened and checked.
  • Only documents with links - skips the documents in which the scan found no links (needs a full scan).
  • Only documents with SmartMapper keys (fastest) - only the documents that have a key of this run, in the source location or at the target location given by the mapping rules. It works on a scan of the source and on a scan of the target (Ultra fast scan is enough).

The list can be a scan of the source, an imported run, or a scan of the target location after the migration. Click Finish to start the replacement with these keys.

After reviewing the generated Search & Replace keys you will have following options:

  • Continue with replacements - ReplaceMagic will use SmartMapper prepared search & replace keys for immediate replacement 
  • Go back & review - you will have possibility to readjust search & replace keys
  • Close without replacements - search & replace keys are generated but will not be used during this session



    Step 5 – Repair Your Documents

Return to the main Search & Replace page.

Select the document areas that should be processed and start the replacement.

If you chose Only documents with links or Only documents with SmartMapper keys on the last page of the wizard, ReplaceMagic first shows how many documents will be processed:

SmartMapper - Documents to process question

Yes processes only these documents, No all documents. If no document matches (for example a scan of another location), all documents are processed and you are told so. A list that was already reduced when the run was imported is processed without this question.

SmartMapper automatically prepares Search & Replace keys for link-related content, including:

  • Hyperlinks
  • OLE Objects
  • Link Sources
  • Pivot Tables
  • Power Queries

Other replacement areas are processed normally by ReplaceMagic but are not specifically optimized by SmartMapper.

For document migration projects, we recommend selecting only the link-related replacement areas.

After confirmation, SmartMapper confirmation will appear in the standard ReplaceMagic Search & Replace list.


Note: do not select Formulas (Excel) for SmartMapper keys - ReplaceMagic de-selects it automatically before the replacement starts and tells you so (in a command-line run it is written to the log). Links to other workbooks are changed by Link Sources and the formulas follow them; with Formulas (Excel) a key could be applied twice to the same external link and the link would then contain its path twice.


From this point onward, ReplaceMagic performs the document repair exactly like any other Search & Replace project.

Note: do not select Formulas (Excel) for SmartMapper keys - ReplaceMagic de-selects it automatically before the replacement starts and tells you so (in a command-line run it is written to the log). Links to other workbooks are changed by Link Sources and the formulas follow them; with Formulas (Excel) a key could be applied twice to the same external link and the link would then contain its path twice.

Key Takeaway

The only information you need to provide is your migration mapping rules.

Everything else is performed automatically by SmartMapper:

  • Scan analysis
  • Link discovery
  • Pattern detection
  • Search & Replace key generation
  • Validation
  • Conflict detection
  • Review
  • Link repair preparation

No manual creation of Search & Replace keys. No guesswork. No scripting.

Simply provide your migration mappings and let SmartMapper do the rest.

Command-Line Support

SmartMapper-generated Search & Replace keys can also be used in unattended or automated executions via the ReplaceMagic XML configuration file.

See Command Line Mode (Silent Run Parameters) for details.

Tip: if you work with a saved run, import it with Only documents with SmartMapper keys (Saved RUN(s) → Import) - then only the documents that will change are opened.