Posts

How to Convert Legacy Text Files [Video]


It’s a common requirement to convert non-standard or legacy text files to or from structured data formats like XML, JSON, and relational databases. However, many times legacy text files are not in a format that can be readily processed by data mapping tools, especially when they have a complex and unique structure that does not consistently fit into CSV or fixed-length field patterns. Moreover, sometimes you need to extract only portions of useful data from a legacy text file.

MapForce, Altova’s any-to-any data conversion tool, includes a unique utility called FlexText that makes it easy to visually define templates for parsing text files and making them accessible to the data mapping tool.

See how FlexText works in our video tutorial.

The example files referenced in the video are available here and you can try FlexText with a free, 30-day trial of MapForce.

Tags: , ,

Data Mapping JSON Lines


The JSON data format continues to evolve as an open standard as it is creatively applied to new data interchange requirements. JSON Lines, defined at http://jsonlines.org/, is a convenient text format for storing structured data where each record is a single line and a valid JSON object. JSON Lines handles tabular data and clearly identifies data types without ambiguity. This allows records to be processed one at a time, which makes the format very useful for exporting and sending data.

Altova MapForce supports data mapping JSON Lines as either a data source or target. Let’s look at a mapping project to extract records from a database table and map to a JSON Lines file for output.

Read more…
Tags: , , , ,

Handle HTTP Errors During Automated Data Integration


Data analysts and other professionals often need to generate real-time data through automated execution of data mappings that request Web services and save the results. During automated execution it’s important to gracefully handle any unexpected HTTP error rather than terminate the integration task.

In an earlier post we discussed conditional processing of a REST Web service response to handle HTTP errors, where separate output files were generated for a normal response and an error. Now let’s look at a revised mapping solution for the airport status example to generate a single mapping result file that contains either the requested airport status or a description of the error.

Read more…
Tags: , , , , ,

Handling HTTP Errors in Web Service Data Mappings


Data integration projects that include information from external Web services may be vulnerable to HTTP errors when retrieving remote data. When data mappings run under automated control it’s especially important to detect and report errors even if errors only occur very rarely.

A MapForce data mapping can include Web service calls and output the result directly to a file or database, or combine it with other inputs for further processing. Regardless of the final output, an HTTP Web service error encountered in a REST Web service request puts the mapping at risk.

MapForce includes features for handling HTTP errors instead of simply aborting execution of a mapping. Developers can configure the body of a REST Web service call to handle and report exceptions based on the HTTP status code returned.

Let’s look at an example.

Read more…
Tags: , , , , ,

Database Mapping with Database Exception Handling


Critical business processes depend on reliable data and database administrators and other data analysts want to be confident in the integrity of information stored in database tables. During automated ETL (Extract Transform Load) operations or other database import tasks, invalid data might be encountered that jeopardizes success of the procedure. Altova MapForce includes database exception handling to roll back the affected data when an error occurs and optionally proceed with the rest of a database mapping.

For instance, an error in a single record need not prevent execution of a mapping from continuing, such as when certain database constraints prevent the mapping from inserting or updating invalid data.

Read more…
Tags: , , ,

MapForce Tutorial (Video)


Altova MapForce is an any-to-any data transformation, conversion, and ETL tool for integrating data.

A graphical data mapping tool, MapForce has an intuitive drag-and-drop interface that lets you easily convert data between any two formats, such as XML, JSON, relational databases, EDI, and more. It also features an extensive library of conversion functions that can be chained together to form custom functions that can be reused throughout your projects.

Data translated by MapForce can be pulled to or pushed from any relational database and all data management products, and it can be adapted to customize in-house data management solutions.

The MapForce tutorial video below covers all major features offered by the data integration tool and shows example mappings between several different types of files.

You can try MapForce yourself with a free, 30-day trial.

Tags: , , , , , ,

Database Tracing to Log Changes Made by a Data Mapping Project


Database administrators and other data professionals often want to maintain a record of changes in critical databases, especially when updates are made by automated scripts or other operations. Database tracing lets administrators track critical changes or anomalies, and help recover from errors. Altova MapForce supports database tracing for all popular relational databases to log the changes made by a data mapping project to the database when the mapping runs.

When tracing is enabled, events such as database insert or update actions, or errors, are logged in an XML file that you can later analyze or process further in an automated way.

Database tracing can be enabled at the database component, table, stored procedure, or database field level. You can choose to trace all messages or only errors, or you can disable tracing completely.

In addition to tracing errors that occur during the execution of a mapping to a target database, MapForce also enables database transaction handling to roll back the affected part of the database data when an error occurs, then optionally proceed with the rest of the mapping.

Read more…
Tags: , , , ,