Validating and Debugging Data Transformations


Software developers and other data professionals often need to transform data from one format to another. These transformations can be simple one-to-one conversions or may require more complex manipulation. For instance, relationships must be generated when importing flat CSV files into a database, or source data may need to be split for the target, as in full name vs. first, middle, last, and optional suffix. Validating data transformation is critical to prevent data loss or corruption.

In an earlier post on Web service data integration, we combined a string value for GMT time with a numeric offset in seconds to generate the local time for weather forecasts. We created a user function that performed all the steps required to complete this operation. MapForce includes a powerful interactive data mapping debugger that can easily trace and validate this transformation. Let’s take a look at how it works.

A software developer working on a computer
Read more…
Tags: , , , ,

Benefits of an XML Schema Manager


Developers working with XML often need to deal with multiple DTDs and XSDs that define industry-standard vocabularies. Whether it’s DITA for technical writing, HL7 for healthcare data, CbCR for financial reporting or any number of examples, it becomes a challenge to manage the various schemas—and numerous versions thereof—on a day-to-day basis.

Altova recently introduced a new XML Schema Manager integrated in its XML-enabled products. This provides a centralized utility that makes it easy to download and manage industry schemas for use across the product line. Let’s see how it works.

Developer working at a computer
Read more…
Tags: ,

ESEF Tools


The ESEF acronym has been top of mind for finance professionals across the EU and UK since mid 2019, when an upcoming reporting mandate from the European Securities and Markets Authority (ESMA) was announced.  

ESEF, which stands for European Single Electronic Format, is a digital financial reporting standard based on XBRL. As of early 2020, companies on EU regulated markets are mandated to prepare their annual reports in accordance with ESEF rules.

What exactly is ESEF compliance, and what does it take to meet reporting requirements? Let’s take a look at the basics and some tools that make it easy.

Finance pros
Read more…
Tags: , , , ,

Certified Tools for xBRL-JSON & xBRL-CSV


XBRL International has finalized the sunrise period for its important new OIM (Object Information Model), which includes the xBRL-JSON and xBRL-CSV standards. In turn, it has completed the software certification process. Any product awarded the XBRL Certified Software designation has been thoroughly tested by XBRL International for conformance with the current XBRL specifications.

OIM represents a years-long effort of the XBRL community to modernize the financial reporting standard, providing a model for easily transforming XBRL data between XML and other popular formats like CSV and JSON. This way, organizations can take advantage of the functionality of XBRL and at the same time have XBRL documents written in the format(s) most convenient for them.

Altova XMLSpy and RaptorXML Server were some of the very first tools on the market to support xBRL-JSON and xBRL-CSV and are now officially named XBRL Certified Software for the OIM standards (in addition to being certified for a variety of other core XBRL technologies).

Learn more:

Colleagues collaborating on computers in an office
Tags: , , , , ,

Working with Avro Big Data in Your Favorite XML Editor


Big Data trends have developers working with XML alongside other data protocols such as JSON and Apache Avro, and XMLSpy supports both of these with dedicated editing views and functionality.

Let’s see how specialized Avro support in XMLSpy makes visualizing and searching Avro files, as well as editing Avro schemas, uniquely easy. We’ll also look at some of the advantages of utilizing RaptorXML Server for high-performance Avro processing.

shutterstock_88166515

Read more…

Tags: , , , , , ,

Automatically Convert EDI to XML


Electronic Data Interchange (EDI) has proven to be a durable business-to-business communication technology in use today with history dating to the 1960s and even earlier. Efficient EDI data encoding reduced transaction payload size and improved data transfer speeds at a time when messages were sent over teletype at speeds equivalent to a 300 baud modem. EDI standards bodies promote wide acceptance among enterprises, and systematic evolution of new EDI formats extends support across industries. EDI improves profitability and is a dominant format for e-commerce data exchange.

Despite the advantages, EDI files are barely human readable and need to be translated and mapped for compatibility with other business technologies. Altova MapForce is a graphical EDI mapping tool with native support for all major business data formats in use today, including XML, JSON, databases, flat files, Excel, and Web services, as well as the EDIFACT, X12, HL7, NCPDP SCRIPT, IDoc, PADIS, and SWIFT EDI transaction sets. MapForce can even automatically convert EDI to XML without the need to specify a target XML Schema or perform any manual mapping.

Read more…
Tags: , , ,

How to Compare CSV Files or Compare a CSV File to a Database Table


CSV files are a quick and convenient way to record structured data in a generic format. Because CSV files are so easy to create, multiple similar versions of very large CSV files can quickly proliferate. Often it becomes necessary to compare CSV files to find the desired version. In an ETL scenario, a data analyst may want to compare a CSV file to a database table for validation or to update data.

DiffDog, the unique XML-aware diff / merge tool from Altova, supports CSV as a native file format for comparison and can compare and selectively merge data CSV to CSV, or between a CSV file and database table. Let’s look at an example.

Read more…
Tags: , , , ,