data mapping Archives

Tag Archive for: data mapping

AI-based Database Image Classification with Altova MapForce

July 31, 2023/in Data Integration, Database mapping, Tools /by David McGahey

One of the most common examples of AI in our everyday lives is facial recognition. Facial recognition is the process of identifying or verifying a person’s identity based on their face. Facial recognition is used in many applications, such as unlocking our phones with FaceID, tagging our friends on social media platforms like Facebook, and checking in at airports or hotels with biometric scanners. Facial recognition can make our lives more convenient and secure, but it can also raise some privacy and ethical concerns. For instance, how can we ensure that our facial data is not misused or stolen by hackers or malicious actors? How can we prevent facial recognition from being used for surveillance or discrimination? How can we ensure that facial recognition is accurate and fair, and does not have any biases or errors?

The paragraph above was generated by ChatGPT in response to my request to describe the benefits and risks of artificial intelligence and include a real-life example. It’s interesting that ChatGPT chose FaceID as the example, since FaceID is simply one variation of image analysis and AI-powered image classification offers potential to automate many real-world tasks.

One common use-case is a product catalog, wherein a company manages product information provided by many different manufacturers. A product loaded into that database may have a name that does not necessarily include a precise description of the item. For instance, wellington is a boot, fedora is a hat, a mongoose is a bicycle, and a yellow watermelon shiny needlefish is a fishing lure. We can make use of AI-powered image classification using the Microsoft Azure Cognitive Services Computer Vision API to address this problem. The Computer Vision Service takes the image data or URL as its input and returns information about the content. One service generates image classification tags based on a training set of recognizable objects, living beings, scenery, and actions that the Azure AI has been trained on. These tags allow us to categorize products in the database accordingly and may even correspond to search terms a user might provide to find products in the catalog.

decorative image depicting an AI "brain"

AI-based support request sentiment analysis using MapForce and GPT-4

July 17, 2023/in Data Integration, Database, ETL /by Alexander Falk

Automated sentiment analysis of text, such as user reviews, has historically been a challenge. Because of the myriad intricacies of natural language, systems faced difficulties in analyzing context and nuances. This required an inordinate amount of manual work to overcome.

One of the many useful capabilities of modern AI systems that are based on large language models (LLMs) such as OpenAI’s GPT-4 is that they are very good at sentiment analysis of natural text inputs. We can use that capability to build a very efficient database solution in MapForce that, for example, goes through all the new incoming records in a support database and automatically determines whether a particular support request or other customer feedback is positive, negative, constitutes a bug report, or should be considered as a feature request.

Decorative lead photo depicting machine learning for sentiment analysis

How to Create a Chained Data Transformation

July 5, 2023/in Data Integration, Tools /by Erin Cavanaugh

Data mapping plays a vital role in modern data-driven organizations, enabling efficient data management and integration. Altova MapForce is a powerful, graphical data mapping tool that supports endless data transformation scenarios, including one-to-one, one-to-many, many-to-one, and chained data conversion.

While there are applications for each of these approaches, chained data mapping is especially helpful for complex data processing tasks where multiple stages of data manipulation are required. Here’s a look at the benefits of a chained data conversion approach – and a video of how MapForce makes the process easy and straightforward.

Decorative image: three business people working on laptops

How to Convert Data in MapForce [Video]

June 6, 2023/in Data Integration, Software, Tools /by Erin Cavanaugh

Altova MapForce offers a powerful, yet easy-to-use, approach to converting and transforming data. Whether you need to write XML to a database, convert JSON to EDI, or map Excel to multiple different data formats, MapForce has you covered.

From simple one-to-one conversions to complex ETL scenarios, the MapForce approach is to represent data structures as graphical components. To associate fields, drag and drop connecting lines. A comprehensive library of data filters and functions is available for transforming data before writing it to the target.

Decorative image showing a person working on a laptop with data formats supported by Altova MapForce in the background

We have recently revamped our series of Introduction to MapForce videos. Each short how-to gives a demo of a common MapForce scenario.

Start at the beginning to learn how MapForce works:

And follow along to learn:

An overview of MapForce features and functionality
How data transformation works
XML to XML transformation
Database to XML mapping (including filters and querying the database)
Mapping XML to databases (including database table actions)
One-to-many data integration (featuring Excel to JSON and XML and value-map functions)
Chained data transformation (including if-else conditions and data sorting functions)

You can follow along with the examples in these how-to videos by downloading a free, 30-day trial of MapForce. Check back for new MapForce videos, which are added often.

Tags: data conversion, data integration, data mapping, data transformation, demo videos, ETL, MapForce, videos

Validating and Debugging Data Transformations

November 30, 2022/in Data Integration, Tools /by David McGahey

Software developers and other data professionals often need to transform data from one format to another. These transformations can be simple one-to-one conversions or may require more complex manipulation. For instance, relationships must be generated when importing flat CSV files into a database, or source data may need to be split for the target, as in full name vs. first, middle, last, and optional suffix. Validating data transformation is critical to prevent data loss or corruption.

In an earlier post on Web service data integration, we combined a string value for GMT time with a numeric offset in seconds to generate the local time for weather forecasts. We created a user function that performed all the steps required to complete this operation. MapForce includes a powerful interactive data mapping debugger that can easily trace and validate this transformation. Let’s take a look at how it works.

A software developer working on a computer

Automatically Convert EDI to XML

May 18, 2022/in Data Integration, EDI, Tools /by David McGahey

Electronic Data Interchange (EDI) has proven to be a durable business-to-business communication technology in use today with history dating to the 1960s and even earlier. Efficient EDI data encoding reduced transaction payload size and improved data transfer speeds at a time when messages were sent over teletype at speeds equivalent to a 300 baud modem. EDI standards bodies promote wide acceptance among enterprises, and systematic evolution of new EDI formats extends support across industries. EDI improves profitability and is a dominant format for e-commerce data exchange.

Despite the advantages, EDI files are barely human readable and need to be translated and mapped for compatibility with other business technologies. Altova MapForce is a graphical EDI mapping tool with native support for all major business data formats in use today, including XML, JSON, databases, flat files, Excel, and Web services, as well as the EDIFACT, X12, HL7, NCPDP SCRIPT, IDoc, PADIS, and SWIFT EDI transaction sets. MapForce can even automatically convert EDI to XML without the need to specify a target XML Schema or perform any manual mapping.

Tags: data conversion, data mapping, EDI, XML

How to Debug XSLT and XQuery

April 13, 2022/in Development, Software, XML /by Erin Cavanaugh

Nothing’s more frustrating than getting unintended results from an XSLT or XQuery transformation and having to spend hours tracking down the issue – especially if you’ve inherited the project from another developer or haven’t looked at the code in a few months. Of course, XMLSpy has long included an XSLT debugger and XQuery debugger for setting break points and stepping through transformations to identify problems.

For a more interactive debugging process, XMLSpy also includes XSLT/XQuery back-mapping.

With back-mapping enabled, you can simply click on or hover over the portion of your output document you want to zero in on, and XMLSpy will immediately highlight the source XML and XSLT or XQuery instruction that is responsible. Let’s see how it works.

Debug XSLT with back-mapping

Tags: data mapping, XML Editor, XMLSpy, XQuery, XSLT