Data Integration Archives

Handling HTTP Errors in Web Service Data Mappings

June 19, 2019/0 Comments/in Data Integration, Tools data mapping, handling http, handling http errors, http, http error, http errors, http status, http status codes, mapforce, new response body, response, rest web, rest web service, service, status, status code, status codes, web, web service, web service call /by David McGahey

Data integration projects that include information from external Web services may be vulnerable to HTTP errors when retrieving remote data. When data mappings run under automated control it’s especially important to detect and report errors even if errors only occur very rarely.

A MapForce data mapping can include Web service calls and output the result directly to a file or database, or combine it with other inputs for further processing. Regardless of the final output, an HTTP Web service error encountered in a REST Web service request puts the mapping at risk.

MapForce includes features for handling HTTP errors instead of simply aborting execution of a mapping. Developers can configure the body of a REST Web service call to handle and report exceptions based on the HTTP status code returned.

Let’s look at an example.

Database Mapping with Database Exception Handling

June 5, 2019/in Data Integration, Database, Database mapping combined with database, data, database component, database exception, database exception handling, database tables, db query window, error, error occurs, exception, exception handling, execution, handling, invalid data, mapping, mapping execution, proceed with the rest, rest of the mapping, target database, transaction and continue /by David McGahey

Critical business processes depend on reliable data and database administrators and other data analysts want to be confident in the integrity of information stored in database tables. During automated ETL (Extract Transform Load) operations or other database import tasks, invalid data might be encountered that jeopardizes success of the procedure. Altova MapForce includes database exception handling to roll back the affected data when an error occurs and optionally proceed with the rest of a database mapping.

For instance, an error in a single record need not prevent execution of a mapping from continuing, such as when certain database constraints prevent the mapping from inserting or updating invalid data.

MapForce Tutorial (Video)

May 15, 2019/in Data Integration, Database, EDI, JSON, SQL, XML conversion, data, data management, features, functions, management, mapforce, relational, tool /by Ty Shuff

Altova MapForce is an any-to-any data transformation, conversion, and ETL tool for integrating data.

A graphical data mapping tool, MapForce has an intuitive drag-and-drop interface that lets you easily convert data between any two formats, such as XML, JSON, relational databases, EDI, and more. It also features an extensive library of conversion functions that can be chained together to form custom functions that can be reused throughout your projects.

Data translated by MapForce can be pulled to or pushed from any relational database and all data management products, and it can be adapted to customize in-house data management solutions.

The MapForce tutorial video below covers all major features offered by the data integration tool and shows example mappings between several different types of files.

You can try MapForce yourself with a free, 30-day trial.

Tags: data integration, data mapping, Database Mapping, JSON mapping, MapForce, MapForce Server, XML JSON

Database Tracing to Log Changes Made by a Data Mapping Project

May 1, 2019/in Data Integration, Database, Database mapping, Database tracing data, database component, database data, database tracing, database transaction, database transaction handling, errors, flowforce server, handling to roll, log, log file, mapping, rest of the mapping, table, trace log, tracing, tracing log, tracing log file, transaction handling, xml file /by David McGahey

Database administrators and other data professionals often want to maintain a record of changes in critical databases, especially when updates are made by automated scripts or other operations. Database tracing lets administrators track critical changes or anomalies, and help recover from errors. Altova MapForce supports database tracing for all popular relational databases to log the changes made by a data mapping project to the database when the mapping runs.

When tracing is enabled, events such as database insert or update actions, or errors, are logged in an XML file that you can later analyze or process further in an automated way.

Database tracing can be enabled at the database component, table, stored procedure, or database field level. You can choose to trace all messages or only errors, or you can disable tracing completely.

In addition to tracing errors that occur during the execution of a mapping to a target database, MapForce also enables database transaction handling to roll back the affected part of the database data when an error occurs, then optionally proceed with the rest of the mapping.

Updated JSON Schema Support Highlights v2019 Release 3

April 2, 2019/in Data Integration, Development, Software data mapping, database, diffdog server, json, json and json, json and json schema, json and json schema documents, json schema, json schema documents, json schema draft-06, mapping, mapping options, mapping options in mapforce, new json schema, rest web, rest web service, schema draft-06, updated to include, web service, web services /by Erin Cavanaugh

From updated support for standards and relational databases to several new data mapping options, Altova’s Version 2019 Release 3 product line addresses developer requests and ups the ante with innovative new functionality. Let’s take a look at the highlights.

Tags: JSON, JSON Schema, MapForce, new features, XMLSpy

Mapping Structured Data with Enhanced Node Functions

December 6, 2018/in Data Integration, Tools data, data mapping, installed with mapforce, main mapping window, mapping, mapping structured, mapping structured data, mapping window, node function, node function definition, node function parameters, node functions, node metadata, opens the node, opens the node function, simplify mapping structured, simplify mapping structured data, structured data, structured data file, xml schema /by David McGahey

We’ve reported previously on support for node functions that simplify mapping structured data by eliminating need to copy-paste a function multiple times into a mapping. Repeating the same function unnecessarily clutters the mapping layout and makes the data mapping more difficult to understand or revise.

Now in MapForce 2019, additional filters are available for defining node functions. These new parameters allow developers to apply functions and default values to specific nodes based on custom-defined criteria. For example, you can apply a node function based on node metadata such as the node name, node length, precision of the node’s data type, customized node annotations, and more.

Let’s look at a mapping with enhanced node functions.

Job Distribution on FlowForce Server

November 28, 2018/in Data Integration, Servers, XBRL, XML altova server, altova server software, cluster, cluster management interface, data integration, flowforce, flowforce server, flowforce servers, load balancing, load sharing, master, master machine, master server, raptorxml server, report generation, server software, set up a cluster, worker, worker machines, worker servers /by Erin Cavanaugh

FlowForce Server is Altova’s high-performance engine for automating workflows of XML processing, data integration, report generation, and more. It integrates with other Altova server software products to automate their functions, such as executing complex data integration processes, including ETL projects, designed in MapForce; running StyleVision report generation jobs; or validating XML, XBRL, or JSON files with RaptorXML Server.

Starting with Version 2019, FlowForce Server offers new options for distributed execution and load balancing to improve availability and performance. Let’s take a look at how configuring multiple FlowForce Servers to run as a cluster can help improve data throughput and provide redundancy.

Job distribution for high availability on FlowForce Server