Data Mapping Binary Objects – Part 2


Binary objects – BLOBs — can be cumbersome to manage in databases. In an earlier post we described a MapForce data mapping to insert binary objects into a database with generated metadata to identify the BLOBs later. The companion challenge in data mapping binary objects is to extract binary data and save it in a comprehensible form faithful to the original.

Let’s look at how that’s done.

Read more…
Tags: , ,

Data Mapping Binary Objects


Binary objects are difficult to manage in databases. They are large, their content is not human readable, and they can contain bytes of data easily misinterpreted as control characters. Even the data type name for binary large objects – BLOB – reflects most database managers’ dislike of them. Before relational databases, the definition of a blob was “something undefined or amorphous.”

Altova MapForce, the award-winning, graphical data mapping tool for any-to-any conversion and integration, includes features for effortlessly data mapping binary objects to or from all popular relational databases. Data such as images, PDF files, video files, or any other binary data can be mapped. Let’s look at an example.

Read more…
Tags: ,

New Data Integration Tools


Altova MissionKit tools offer numerous ways to connect to, query, and integrate data from disparate sources. With multiple product releases each year, we’re constantly working to deliver increased power and efficiency for data integration, while adding features requested by customers. This includes ongoing updates to built-in support for all major SQL databases across the product line.

Let’s take a look at some of the recently added tools and enhancements.

New data integration tools in Altova's release
Read more…
Tags: , , , , , , ,

Data Mapping JSON Lines


The JSON data format continues to evolve as an open standard as it is creatively applied to new data interchange requirements. JSON Lines, defined at http://jsonlines.org/, is a convenient text format for storing structured data where each record is a single line and a valid JSON object. JSON Lines handles tabular data and clearly identifies data types without ambiguity. This allows records to be processed one at a time, which makes the format very useful for exporting and sending data.

Altova MapForce supports data mapping JSON Lines as either a data source or target. Let’s look at a mapping project to extract records from a database table and map to a JSON Lines file for output.

Read more…
Tags: , , , ,

Transitioning Data Mapping Projects from Development through Testing and Production


Data mapping projects often mirror software development efforts with distinct phases for design, testing, and deployment. This is especially true for ETL (Extract Transform Load) projects when repeated data mapping execution is required as new data becomes available, and the stakes increase higher with large data sets. The Altova MissionKit and Server Software products provide Global Resources to define configurations for each project phase and smoothly transition between them.

Let’s take a look at an example based on a MapForce data mapping from a source file to a database.

Read more…
Tags: , , , , ,

How to Compare CSV Files or Compare a CSV File to a Database Table


CSV files are a quick and convenient way to record structured data in a generic format. Because CSV files are so easy to create, multiple similar versions of very large CSV files can quickly proliferate. Often it becomes necessary to compare CSV files to find the desired version. In an ETL (Extract Transform Load) scenario, a data analyst may want to compare a CSV file to a database table for validation or to update data.

DiffDog, the unique XML-aware diff / merge tool from Altova, supports CSV as a native file format for comparison and can compare and selectively merge data CSV to CSV, or between a CSV file and database table. Let’s look at an example.

Read more…
Tags: , , , ,

Database Mapping with Database Exception Handling


Critical business processes depend on reliable data and database administrators and other data analysts want to be confident in the integrity of information stored in database tables. During automated ETL (Extract Transform Load) operations or other database import tasks, invalid data might be encountered that jeopardizes success of the procedure. Altova MapForce includes database exception handling to roll back the affected data when an error occurs and optionally proceed with the rest of a database mapping.

For instance, an error in a single record need not prevent execution of a mapping from continuing, such as when certain database constraints prevent the mapping from inserting or updating invalid data.

Read more…
Tags: , , ,