Common Data Migration Issues

Common Data Migration Issues

by: Brandon Amick

There are many reasons your organization might be undertaking a data migration: a change in the platform your data is stored on, a major change in software, or even a change in versions of certain software or hardware might require a data migration. But moving your data is never as easy as it seems. There are a few things you should keep in mind before starting the data migration process.

 

 

  1. Make sure all your stakeholders are on board with making a change. Spend time to determine if it is really the right decision for your company. On many occasions, we have migrated a company from one software to another or one database to another – only to go back and reverse all the migration we had previously done because they hadn’t considered the functionality of the new software or database in their existing environment. Mistakes can happen, and sometimes other circumstances may cause these data reversals, such as an acquisition or a merger, but such errors can be costly.
  2. Making a detailed plan is essential very early on. A thorough study of the new software, the skill level of the users, and potentially a gap analysis between the platforms can help you plan better. It can assist in addressing the upcoming changes in software or platform, create a plan for the management of the upcoming changes, and provide a process to discover any potential pitfalls. Plan, plan again, and then plan some more.
  3. Data almost never behaves the same on two different platforms. No matter how alike two pieces of software are, they will never behave exactly the same. As much as I wish that Oracle and SQL Server would just behave the same way and make my life easier, they simply do not. Anytime you migrate your data, it will almost inevitably have to be transformed. At a database level, you will have syntaxes that are no longer valid and must be rebuilt. On a software level, you may have to translate data because “Middle” worked just fine for your labels before, but now they need to say “Center” to have the correct alignment.
  4. After planning the process of the migration, create a separate plan for when and how the migration will take place. Some of our best data migrations have occurred with customers who know their data and have a plan for how it will be migrated. They have practice migration runs scheduled to predict how long the databases will be down, what personnel need to be on call, and how the actual migration will be run. Some have a “situation room” with staff on hand 24 hours a day so that someone is always available in case of an emergency. This kind of planning can have the best results, but unfortunately is not always an option, depending on budget. The process and time required to migrate data is almost always more involved than most companies realize.
  5. Most importantly, you must have a verification process in place. This is where the practice runs can be extremely useful. It allows you to find issues in the data migration before the real migration occurs. If you have verified multiple times that the data looks good after following the procedures of your migration process to the letter, then you can go into the real migration feeling confident. Your post-migration analysis will show that all your data will look just as it is supposed to.

These are just a few of the more common issues that companies tend to overlook – or are not willing to invest as many resources into – when migrating their data. It really cannot be stressed enough that the more planning, analysis, and verification you put into your migration, the better the results will be in the long run – potentially saving your company money by investing up front rather than repairing costly mistakes later.


Reality Capture: Getting Started

Reality Capture: Getting Started

by: Rick Johnston

Once you have learned about the option of Reality Capture for capturing the best version of your data, you may be asking, “How do I even get started?” In this article, we provide basic background information around starting the Reality Capture process and the equipment involved.

 

To decide on the right equipment, we first need to consider some basic determining factors on environments to be captured:

 

 

If the answers to these determining factors are unclear, it is worth spending some time internally reviewing the requirements or getting advice from an expert. Many devices and applications are available and using the right one for the job at hand is essential.

One of the key determining factors as listed above would be what the captured data will be used for. A design workflow, such as a renovation or as-built? Or a simple visualization of the area and included assets? This determines the data density required for the job; whether you choose high or low point cloud data will determine if a high-end device is necessary, or a mobile device or camera is acceptable.

 

Various types of hardware can be considered for a variety of different projects:

 

There is no “one size fits all” solution, and many factors must be considered. We have listed a few of these factors, but they are all intertwined and there is no simple hard-and-fast solution for every situation. To illustrate this, let’s consider the following scenario.

 

A company needs to capture facility assets. For the most part, these spaces are small (approximately 10′ wide x 20′ long x 9′ high) and congested with wiring and potentially dangerous spaces to be in and to access. The company desires point cloud data to be used in a future design workflow.

 

In this scenario, the most appropriate equipment for the situation is driven by the determining factor of the small and congested spaces they want captured. While a camera would work, the best choice here would be a mobile scanning device, since the aim is to capture raw point cloud data.

 

 

 

 

Image courtesy of AMLP.

 

Taking the appropriate considerations into account and understanding what the determining factors are is key to capturing and utilizing a 3D model, regardless of your workflow.

 

Is your organization or department anxious to get started but wants expert advice on how to go about it? We can help! Contact us if you want to get started on reality capture and begin planning and capturing your data.


SynCumulus Field Mapping Application

SynCumulus Field Mapping Application

By: Jessica McCall

In recent months, we have been developing SynCumulus, a web and mobile viewing application. This application allows users a simple GIS view and the creation of inspection records and GPS coordinates, as well as the ability to capture imagery. The SynCumulus solution for web and mobile utility visualization is based on years of experience in consulting with customers across North America. Our approach is uniquely rooted in the simple integration of off-the-shelf components, open source frameworks and enterprise database technology.

 

Here is a quick snapshot of the viewer within a browser window.

 

 

Some of SynCumulus’s features include:

 

Google Maps Integration

Base maps can be provided by Google, and street views and directions are integrated to show visual and test descriptions. Navigation within the map is easy with the simple icons located on the menus, and with the integration of Google Maps, users can find their location and get directions to the next inspection or to a particular feature.

 

 

Data Layers

The layers of data and imagery are published directly from the available data. Currently, this is integrated with the AutoCAD Map 3D display models and produces the same symbols and layers used in the application. In the future, we are looking to make the integration available with other GIS applications.

 

 

 

Mobile Inspection

SynCumulus will feature a list of assets for the Inspector, and the Inspector can enter the inspection data into simplified inspection forms. The Inspector can identify assets in the map that need to be removed or new assets in the field that are not on the map. Updates to the GIS are made via the synchronization architecture, and the Inspection Coordinator can review and promote the assets. Photos can be taken to add observation information to the record.

 

 

Search and Features

The search function allows for a search by Asset ID, which can retrieve the location of and directions to the desired asset. The feature’s properties are available to view upon searching for the asset, and the display allows the user to see asset information, including associated documents and previous inspections.

 

 

Because SynCumulus utilizes Microsoft Sync Framework, we have created our own components that bridge these world-class technologies. Employing these features allows users to work in a connected environment to create real-time updates to the GIS, or in a disconnected environment where synchronization can occur at any point when a connection is available.

 

Several of our customers are implementing this technology right now, and we are looking for other organizations to be among our “beta testers.” If you want to mobilize your field crews to have access to up-to-date information, please contact us. We look forward to showing you how we can make your field operations more efficient.


Geocoding Inspections in the Field

Geocoding Inspections in the Field

by: Stephen Brockwell

In the field, it is essential to capture construction and inspections data as accurately and concisely as possible. With the ability to geocode inspection data, the utility has access to as-built information to assist in design and additional construction, and also has the inspection data at its fingertips in the home office, along with imagery and asset information.

 

geo-image-1

Image 1: Potential inspections viewed on the mobile app

Using an integration of BIM 360 Field and web-based GIS, we have developed an app that can be employed in the inspection and work-order process that can gather data in the field for use in the office for scheduling, reporting and analysis.

 

In Image 1, the map shows assets, sites and upcoming inspections. The map is dynamically configured and can also be displayed in AutoCAD. It is hosted from a SQL Server database on Azure or Amazon infrastructure or on local infrastructure, and can be synchronized or used in a disconnected state. The layers are composed of WFS, DWG, SHP, and other spatial services.

 

geo-image-2

Image 2: Select an inspection for an asset near you

 

Image 2 shows that the assets are directly linked to the 360 Field records so they can gather pertinent information in the field. The user can select an asset (label A), either a site or a location, and then click the inspection button (label B).

 

geo-image-3

Image 3: Record asset information such as field notes or photos

 

The asset is linked directly to the map and is opened for reporting or analysis in the BIM 360 Field site (Image 3).

 

Capturing Field Data

Recording the asset information and the results of the inspection is simple.

Step 1: Go to the site. Our example is an existing wall-mounted hydrant, but the site can be any location, site or equipment – new or existing assets.

 

geo-image-4

Image 4

 

Step 2: Select the asset in BIM 360 Field. At this point you can enter any field notes or other data. You can also take photos at other non-asset locations.

 

Step 3: Use the camera feature in the mobile device to take an image of the asset. iPad photos are automatically tagged with the GPS location.

 

Step 4: Synchronize with the home office once you’re in range of WiFi or if you have a hotspot or LTE. The sync button synchronizes both the BIM 360 Field equipment and inspection data and the spatial database.

 

geo-image-5

Image 5: Synchronize with the home office database

 

Visualizing Inspections

Once the synchronization is complete, the map is updated to reflect the inspection. The inspection location is displayed on the map shortly following the sync after a manual refresh.

You can view the inspection record as well as the notes and any images you captured.

 

geo-image-6

Image 6: Review asset information while in the field

 

geo-image-7

Image 7: Previously captured asset information

 

Capturing field data can be valuable for a variety of reasons and will vary between organizations. Some utilities will find it useful to prepare for their upcoming construction season and others will find the asset information and images important for analysis of a certain class type of equipment throughout the organization. But the ability to see your assets and a record of any additional notes and images can offer more visibility and let you review certain characteristics without the costly time and effort associated with repeated visits to the same equipment.

 

We know what our customers are saying about how this helps them – but we want to hear from you. How can a simple integration work for your organization? What do you want your data to look like – a set of numbers or rich imagery and useful data?


Reality Capture for Utilities: The Best Version of your Data

Reality Capture for Utilities: The Best Version of your Data

by: Rick Johnston

From the Paleolithic humans of the Stone Age chiseling images on rock to the invention of the world’s first commercially available camera in the late 1800’s, there have been ever-evolving efforts to visually portray our world. Today, most industries, including the utility industry, want to go beyond that and model our world in three dimensions (3D) to include various topological characteristics and as-built asset information.

Rendering showing a cross-section view of city streets and underground pipes and utilities.

Image #1: Rendering showing a cross-section view of city streets and underground pipes and utilities.

The modelling process – be it a project site, a building or a topological characteristic – is easy to achieve and can be achieved even for a complete novice. The process can be as simple as clicking some photos with a smartphone camera and using easily obtainable software.

 

Advances in so many areas, from hardware and imagery capture devices (e.g., cameras, scanners, 3-D printers) to software packages (e.g., Autodesk ReCap 360, Autodesk ReMake), offer a variety of uses from creating a simple model to working with the imagery in the context of other existing geospatial data. One of the uses of the three-dimensional model within a design package would be to design in context. For instance, you can create an existing as-built condition model of the exterior or interior of a building for a remodel. This allows the designer to view the building in a before and the after state to review the design changes for planning purposes, which essentially allows you to view the remodel as an “as-built”.

 

Over the past several years, I’ve worked with utility companies across the U.S. that wanted and needed to capture and model their assets in their as-built state. This process of reality capture allowed them to not only capture the as-is state but also allow for review and design changes reducing costly and repeated field visits. This reality capture process works to protect the safety of operations and management staff and to aid in NERC (North American Electric Reliability Corporation) and FERC (Federal Energy Regulatory Commission) regulations.

 

An example of this occurred a few years ago, when Anchorage Municipal Light & Power (AMLP) asked me how to take what they had been learning to the next level. AMLP had a problem when it came to planning and construction cycles, due to Alaska’s extremely cold and dark fall and winter days and its hot and long spring and summer days. They wanted to find a way that the utility could maximize its short spring/summer construction season. They envisioned a better planning season in by capturing existing asset conditions, whether it was a substation, a utility trench or an underground vault. AMLP chose to start with their underground vaults since they are under a thick layer of ice half of the year.

 

I spent about 45 minutes training a qualified utility operations member (because I was unqualified to enter the vault), and he headed to the first vault with a mobile tablet scanner. AMLP had initially utilized a high-end camera, and while the results were excellent, some features of the vaults were not accessible. Certain angles of the vault were needed for use in the post-modelling engine and they proved difficult to capture. Using a mobile tablet scanner, we were able to get the images needed for the complete model of the vault to be captured. After an hour working in the vault, we were able to produce the point cloud as shown in Image #2.

Image used courtesy of MLP.

Image #2: Point cloud model of an underground vault. Used courtesy of AMLP.

 

After this project, Anchorage Municipal Light & Power is well on its way to solving their initial issue of completing a thorough planning of their construction season and also now has a trusted, documented source of 3D models to refer to rather than having to rely on only interpreted data. Their employees can review the contents of the vault and viewing the image as if they were actually in the vault.

 

The reality capture process has paved the way for many utilities to utilize 3D models in design and planning meetings and for safety purposes. As one of the workers at AMLP said, “We have only begun to scratch the surface of what is possible and have not even begun to envision everything that may be possible.”

 

Are you working with the best version of your data? Could 3D images of your utility and infrastructure assets provide even better insight into the potential design capabilities? Let us know if you want to catapult your 2D data into 3D – our Reality Capture process will make you a believer!

 


News and Events

Autodesk Authorized Consulting Partner

Brockwell IT Consulting is an Autodesk authorized consulting partner with significant expertise in geospatial asset management and IT architecture for utilities. We have a variety of customizable services to fit your organization and industry. Visit our Accelerator page to learn more.

Looking for Training?

Brockwell IT Consulting can offer general application training or customized training for Autodesk and other utility-industry focused applications. Contact us for more information.

Want to join the team?

Check out our Careers @ Brockwell IT.

Contact Us

  • This field is for validation purposes and should be left unchanged.