MapForce Server Archives

Tag Archive for: MapForce Server

Transforming and Converting Protobuf

February 10, 2022/in Data Integration, Database, JSON, protocol buffers, XML /by David McGahey

MapForce supports mapping protocol buffers (Protobuf) to and from other structured data formats as mapping sources or targets. In the constant quest for more efficient ways to transfer, manipulate, and manage large structured data sets, Google has created a language- and platform-neutral data format similar to XML, but smaller, faster, and simpler than even JSON data. Tools are available to generate and work with Protobuf using Java, Python, C++, C#, Ruby, and other programming languages.

The structure of any Protobuf message is defined in a .proto file that defines each field name and value type. Altova MapForce lets users drop these .proto files into a data mapping as a source or target along with any other data, including XML, JSON, relational databases, Excel, flat files, REST and SOAP web services, and others. .proto files versions 2 and 3 are supported.

A MapForce data mapping creates compatibility between existing XML, JSON, database or legacy data formats and new applications leveraging the efficiency of Protobuf.

Transitioning Data Mapping Projects from Development through Testing and Production

November 29, 2019/in Data Integration, Database mapping, Servers, Software /by David McGahey

Data mapping projects often mirror software development efforts with distinct phases for design, testing, and deployment. This is especially true for ETL (Extract Transform Load) projects when repeated data mapping execution is required as new data becomes available, and the stakes increase higher with large data sets. The Altova MissionKit and Server Software products provide Global Resources to define configurations for each project phase and smoothly transition between them.

Let’s take a look at an example based on a MapForce data mapping from a source file to a database.

Handle HTTP Errors During Automated Data Integration

July 29, 2019/in Data Integration, JSON, Tools /by David McGahey

Data analysts and other professionals often need to generate real-time data through automated execution of data mappings that request Web services and save the results. During automated execution it’s important to gracefully handle any unexpected HTTP error rather than terminate the integration task.

In an earlier post we discussed conditional processing of a REST Web service response to handle HTTP errors, where separate output files were generated for a normal response and an error. Now let’s look at a revised mapping solution for the airport status example to generate a single mapping result file that contains either the requested airport status or a description of the error.

Handling HTTP Errors in Web Service Data Mappings

June 19, 2019/in Data Integration, Tools /by David McGahey

Data integration projects that include information from external Web services may be vulnerable to HTTP errors when retrieving remote data. When data mappings run under automated control it’s especially important to detect and report errors even if errors only occur very rarely.

A MapForce data mapping can include Web service calls and output the result directly to a file or database, or combine it with other inputs for further processing. Regardless of the final output, an HTTP Web service error encountered in a REST Web service request puts the mapping at risk.

MapForce includes features for handling HTTP errors instead of simply aborting execution of a mapping. Developers can configure the body of a REST Web service call to handle and report exceptions based on the HTTP status code returned.

Let’s look at an example.

Job Distribution on FlowForce Server

November 28, 2018/in Data Integration, Servers, XBRL, XML /by Erin Cavanaugh

FlowForce Server is Altova’s high-performance engine for automating workflows of XML processing, data integration, report generation, and more. It integrates with other Altova server software products to automate their functions, such as executing complex data integration processes, including ETL projects, designed in MapForce; running StyleVision report generation jobs; or validating XML, XBRL, or JSON files with RaptorXML Server.

Starting with Version 2019, FlowForce Server offers new options for distributed execution and load balancing to improve availability and performance. Let’s take a look at how configuring multiple FlowForce Servers to run as a cluster can help improve data throughput and provide redundancy.

Job distribution for high availability on FlowForce Server

MapForce Server Accelerator Edition Achieves a New Level of Data Transformation Performance

May 23, 2017/in Data Integration, Software, Tools /by David McGahey

MapForce Server automates recurring execution of data mappings and transformations designed and tested using Altova MapForce. Every day, MapForce Server is employed in business communication, financial reporting, database ETL, and many other applications to transform critical data between any of XML, JSON, database, EDI, XBRL, flat file, CSV, Excel, and/or Web service formats.

Now, MapForce Server Accelerator Edition offers even faster throughput for high-performance server platforms.

Use Join to Integrate Data in Any Format

April 18, 2017/in Data Integration, Tools /by David McGahey

Join is a powerful SQL operation implemented across most database types and familiar to database users. Join is typically used to select and combine information from multiple database tables.

Altova MapForce includes a join component for data mapping that works like a SQL join for database tables and extends data integration functionality by empowering users to join data trees of any data format. Anyone familiar with join operations for database tables will find the MapForce join component especially intuitive. A join operation in MapForce can even combine two different data formats and produce output in a new format altogether.