Data Migration: Key Steps to a Seamless Transition to SOLIDWORKS PDM

What is Data Migration?

The process of transferring (often large) quantities of data from an existing system a new system is known as data migration.

This process takes ‘legacy’ data and moves it to a new PDM environment to unify critical product information from outdated or disparate systems.

By unifying data on a single platform, we can improve data accuracy, accessibility, and efficiency for future product development and lifecycle management.

Businesses can then leverage historical product data while benefiting from the advanced features of a modern PDM system.

How long does Data Migration take?

With careful planning and thorough testing, the migration process can proceed smoothly and even enhance data integrity, with the actual process of transferring the prepared data potentially happening in a few hours.

This guide will explain the options for data migration into SOLIDWORKS PDM, outlining the pros and cons of various manual approaches along with the additional benefits of an automated approach, and the risks of not migrating at all.

6 Reasons Why Businesses Should Migrate Legacy Data

Migrating legacy data into your new Product Data Management (PDM) environment is important for several reasons, which also reflect the reasons why you are implementing PDM.

#1 - Increased speed of adoption for faster ROI

When introducing a new PDM system, employees may be resistant to adopting those systems and processes, especially if the option to continue ‘outside PDM’ remains.

As well as effective organisational change management and communication plans being essential to address any resistance, it is key to ensure that legacy data is part of the new PDM environment at the point of go-live.

This helps to avoid the challenges of maintaining separate streams of data and the risks that users continue to follow previous practices.

#2 - Improved Data Management & Security

The Intellectual Property locked in your legacy CAD data could be as valuable as your plant, premises or stock.

At the extreme, it could even be your single most valuable asset.

SOLIDWORKS PDM provides features that enhance data organisation, accessibility, and security. You’ll be able to manage access permissions and include legacy data within audit trails. This makes it easier to manage and retrieve migrated legacy data.

#3 - Enhanced Collaboration and Operational Efficiency

SOLIDWORKS PDM facilitates better collaboration among team members and enhances operational efficiency by providing tools that allow you to find data faster, reducing data duplication and enhancing re-use.

#4 - Data Integrity and Consistency

Migrating to a new SOLIDWORKS PDM environment can help ensure that all data is consistent and free from errors.

Identifying and dealing with issues like duplicate file names and missing references that can be widespread when working without PDM.

#5 - Compliance and Standards

SOLIDWORKS PDM helps businesses to comply with industry standards and regulations.

Migrating legacy data ensures that your data management practices are consistent across all data and that old data is still compliant with current requirements.

#6 – Protect Valuable Data

So, whilst there may be an initial investment in both time and money to migrate your data into SOLIDWORKS PDM, the long-term benefits associated with leveraging that data will soon pay back on that investment.

Those longer-term benefits include reduced maintenance costs, fewer data-related issues, and improved productivity.

In summary, while it may not be in active use, legacy data holds significant value for organisations.

Alongside providing historical insights and support for regulatory compliance, legacy data is a valuable resource for future strategic decision-making.

By migrating your legacy data into your new PDM environment, you can take advantage of these benefits and ensure that your data management practices are aligned with current and future needs.

Pre-Migration Considerations

Before embarking on the data migration journey, several critical considerations must be addressed and the source(s) of your data will, alongside your existing file and PDM/PLM systems, determine the actual processes involved.

Initial Data Assessment

The first step in any migration project is a review of the data to be loaded. This will establish the scope of the migration effort and set reasonable expectations.

Regardless of the project size, an initial review gives a feel for what is involved and what challenges there may be. The following are key aspects to consider during the initial data assessment.

Source(s) of Legacy Data

Here we determine where the legacy data is coming from, its location, and its nature by asking exploratory questions:

• Is data solely on the company’s network and primarily folder/file-based?
• Is it coming from a legacy PDM or PLM system?
• How is data stored and accessed within the existing system?
• What types of data are being migrated? Consider SOLIDWORKS files and non-CAD files, as well as legacy CAD tools.

Sometimes the migration may also involve merging external metadata contained in Excel spreadsheets or output from an ERP system.

If migrating from a different PDM/PLM system, we will need to determine compatibility with the destination system; we need to consider both CAD and non-CAD data including BOMs, projects, and processes (record data) that need to be migrated.

Data Accessibility

For data coming from a legacy or third-party PDM/PLM system, this is likely to require further detailed analysis.

We must consider if there is a database available in a readable format, whether the files reside in the database or in an external ‘archive’, and determine the encryption status of said archive.

Quality of Data

Whilst data being migrated from another PDM/PLM system is likely to be complete, consistent and error free, that can certainly not be assumed for data being migrated from a Windows File System.

Duplicates, broken references (deleted or moved), missing metadata. All these issues introduce challenges but also opportunities to improve that data.

Data Structure/Process and Metadata Mapping

An initial consideration of how the data will be migrated should be completed at the beginning of the project, as it may involve additional set-up within SOLIDWORKS PDM to accommodate or, in some case, create additional API work to integrate external data.

Typical mapping considerations include:

• Folder Structures: How is the data currently structured, and what structure will be applied to migrated data inside SOLIDWORKS PDM?
• Metadata Mapping: Based on availability, how will existing or inconsistent metadata values be mapped to the new system? Consider ‘mandatory’ metadata fields and include requirements to combine, concatenate, or manipulate existing custom properties and/or inject new values from external sources as part of the migration timeline.

• Lifecycle State Mapping: Both SOLIDWORKS PDM Standard and PDM Professional use Workflow states to control security and revision labels on vaulted data.

  During the migration, we will need to consider what states and workflows the data will be migrated into, and whether any pre/post processing is required for standardisation.

Key Assessment Strategies

In our experience, most datasets are too big to check by hand.

Whilst it may be possible, it is impractical to use different strategies and small data samples to test imports for all file types, so it's important to understand the issues affecting the whole dataset.

To address this, Solid Solutions use an automated process, and a bespoke piece of software called the Data Doctor.

The Data Doctor from TriMech is a tool that allows us to capture and asses the health of a company’s data.

We’ll be able to examine appropriate data stored within Windows Explorer or SOLIDWORKS Workgroup PDM, essentially acting as a diagnostic tool for your data, to identify any potential issues with data integrity and organisation.

Using a combination of Windows, AutoCAD and SOLIDWORKS Document Manager API’s, the tool can list and validate data against specific risks such as duplicates, missing files, incorrect file paths, or inconsistent metadata.

After analysing the data, we provide a high-level summary report with suggestions on a practical and appropriate path forward, be that a manual migration, a fully-managed migration service or something in between.

6 Key Stages of the Data Migration Process

Once you have identified the need for a Migration, the process can be broken down into several key steps.

Stage 1 – Identification of Scope, Risks, and Approach

In this stage, we develop a detailed migration plan outlining the scope, timeline, resources, and responsibilities.

Potential risks are identified, and mitigation strategies are developed to address them.

When considering the scope of any migration, requirements will likely be dependent on the type of business model a company falls under.

For product-based industries, the inclusion of live product lines and associated standard part libraries in PDM could be the minimum consideration, whilst for consulting or contracting businesses, the minimum requirement may simply be current or active projects.

Once the scope of a migration has been identified, we can then consider the approach that would be best.

There are several possible methods for bulk import of legacy data. So, when choosing an import strategy, we must also consider the end-user requirements for consuming that data alongside the associated effort, risks, and both the direct and indirect costs of each approach.

There are 5 key approaches to consider:

1. Import all files into a Single Folder: all files are collated and imported into a single folder for the initial check-in and then moved into subfolders.

   This can be a simple process that sets all references properly and is best suited to small datasets where duplicate filenames are not a concern.

   However, restructuring the data post-migration can be cumbersome, time consuming, and ultimately can impact performance due to how PDM maintains those changed references in the database. Fortunately, it is unlikely for this approach to break any references.

2. Import files utilising Reference Search Paths: This can be an effective solution when bringing data in over a longer time period, and where the reference paths (folder structure in PDM) are both manageable and predictable.

   However, if the destination vault structure means there is a requirement for many search paths, it can be time consuming to enter all of the paths individually.

   This is compounded by the fact that ‘Reference Search Paths’ are not searched recursively so each sub folder needs to be explicitly added.

3. Import files by Using Relative Paths: If the existing dataset is stored in a folder structure that will be reproduced in the new PDM vault, then it is possible to ‘lift and shift’ that data to import files and maintain the file references on check-in.

   

   This method is ideal when the legacy data is contained within a single source; when no ‘broken’ references exist at source, or when no other data transformation is required. This is often the simplest solution for a single ‘manual’ migration where only minor post processing may be needed.

4. Import files by Using Pack and Go: This approach can be appropriate when bringing in data progressively, since Pack and Go can help you bring structured data into PDM in discrete packets.

   A key challenge is that extensive manipulation and repairs of references may be required after importing this data in order to avoid duplication in the new environment.

   Hence, this approach is best applied when a highly-selective migration is required (e.g. a handful of top-level assemblies). It is also worth noting that Pack and Go can only locate drawings that share the same filename and location as the model.

5. Import files by Using Automation Tools including pre-processing Repair Tools: In our experience, nearly all migrations experience the pains of existing datasets having issues with incorrect references, duplicated files (names and/or names and content) and it is recommended to fix up the data prior to, or as part of, a Migration service.

   Within the Data Migration Service (DMS), the existing dataset can be manipulated to perform corrective actions such as restructuring data into a replacement folder structure and to read or make corrections to file references.

   Duplicate (content) files can be rationalised to a single ‘master’, referenced by all parents, while any duplicates (file name) can be renamed to unique values based on defined rulesets.

Stage 2 - Data Mapping

Prior to importing any data, it is likely that some work will be required to map the existing data structure to the SOLIDWORKS PDM schema.

This involves defining how files, folders, metadata, and relationships will be translated into the new system with consistency and accuracy in data mapping crucial to ensuring data integrity.

Experience tells us that, when working outside of PDM, inconsistency in metadata names and values are easily introduced, often requiring the need to rationalise multiple custom properties into a single property as part of the migration.

The Data Doctor report can help us to identify the scale of any potential issues so that the DMS process can be refined to resolve these types of issue during the migration process.

Stage 3 - Data Cleansing

The results of the analysis and reviews completed in stages 1 and 2 will determine the scale and processes required to cleanse the data to handle duplicates, errors, inconsistent and irrelevant information.

This cleansing step is paramount to ensure that only high-quality and relevant data is migrated, reducing clutter, enhancing performance, and improving end-user adoption of the new PDM environment.

Within the DMS process there can be complex file conditioning requirements including folder restructuring, renaming of files, and the manipulation of custom properties that can be defined and executed at the point of delivery.

Stage 4 – Importing the Data

Using your preferred manual approach or other migration utilities (such as the DTS or DMS Service), you will then be able to import the data into the new system.

This step should be performed in a controlled environment to monitor and address any issues that arise during the import process.

Within a DMS this is typically into a sandbox (non-production) environment for the validation (pilot migrations), with the live migration only completed after full validation and go-live planning.

Stage 5 – Validation

A migration is in most cases, something you only want to do once since as soon as that destination environment is in use, reversing and redoing any migration will be problematic and likely result in data loss.

It is therefore incredibly important that you validate the migrated data, ensuring it is accurate, complete, and functional before go-live.

This may involve running process tests, verifying structures and metadata, and checking for any discrepancies or errors reported within the process.

User acceptance testing (UAT) is also crucial to ensure the system meets the users’ needs, so ensuring your users are aware in advance of the ‘new’ environment is also critical to success.

Within a DMS we allow for multiple migration pilots/validation stages.

The initial Pilot Migration will be metadata only and will allow you to validate structures and property mappings quickly.

Later, ‘seeded’ migrations (including both partial and the full file system) will allow for more extensive validation of expected behaviours prior to the final ‘live’ migration run.

Stage 6 – Go Live!

Once the data has been validated and approved, the system can go live.

You should monitor the system closely during this phase to address any process issues promptly and ensure a smooth transition.

Conclusion

Migrating data into SOLIDWORKS PDM can be a complex, but ultimately rewarding, process that can significantly enhance an organisation's data management capabilities.

Using the Data Transition or Data Migration Services offered by Solid Solutions and the wider Trimech Group partners, we can ensure a pain free transition that allows you to maximise the benefits of your investment in SOLIDWORKS PDM in the shortest time possible.

The Data Transition Service (DTS) provides a streamlined approach for rapidly bringing files into a new SOLIDWORKS PDM vault from Windows Explorer or SOLIDWORKS Workgroup PDM. Please do note that this streamlined service is limited to certain conditions.

The Data Migration Service (DMS) provides a comprehensive, fully customisable solution for migrating data into new or existing vaults, from various data sources with custom data de-duplication and conditioning requirements.