Process and Generate Files Dynamically

www.altova.com Print this Topic Previous Page Up One Level Next page

Home >  Tutorials >

Process and Generate Files Dynamically

This tutorial shows you how to read data from multiple source XML files and write it to multiple target files in the same transformation. To illustrate this technique, we will now create a mapping with the following goals:

 

1.Read data from multiple XML files in the same directory.
2.Convert each file to a new XML schema.
3.For each source XML file, generate a new XML target file under the new schema.
4.Strip the XML and namespace declaration from the generated files.

Tutorial_Process_and_Generate_Multiple_Files

Abstract model of the data transformation

We will use three source XML files as example. The files are located in the <Documents>\Altova\MapForce2019\MapForceExamples\Tutorial\ folder, and they are named bookentry1.xml, bookentry2.xml, and bookentry3.xml. Each of the three files stores a single book.

 

<?xml version="1.0" encoding="UTF-8"?>
<books xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xsi:noNamespaceSchemaLocation="books.xsd">
  <book id="1">
    <author>Mark Twain</author>
    <title>The Adventures of Tom Sawyer</title>
    <category>Fiction</category>
    <year>1876</year>
  </book>
</books>

bookentry1.xml

<?xml version="1.0" encoding="UTF-8"?>
<books xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xsi:noNamespaceSchemaLocation="books.xsd">
  <book id="2">
    <author>Franz Kafka</author>
    <title>The Metamorphosis</title>
    <category>Fiction</category>
    <year>1912</year>
  </book>
</books>

bookentry2.xml

<?xml version="1.0" encoding="UTF-8"?>
<books xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xsi:noNamespaceSchemaLocation="books.xsd">
  <book id="3">
    <author>Herman Melville</author>
    <title>Moby Dick</title>
    <category>Fiction</category>
    <year>1851</year>
  </book>
</books>

bookentry3.xml

The source XML files use the books.xsd schema available in the following folder: <Documents>\Altova\MapForce2019\MapForceExamples\Tutorial\.  To convert the source files to a new XML schema, we will use the library.xsd schema (available in the same folder). After the transformation, the mapping will generate three files according to this new schema (see the code listings below). We will also configure the mapping so that the name of the generated files will be: publication1.xml, publication2.xml, and publication3.xml. Notice that the XML declaration and the namespace declaration must be stripped.

 

<library>
  <publication>
    <id>1</id>
    <author>Mark Twain</author>
    <title>The Adventures of Tom Sawyer</title>
    <genre>Fiction</genre>
    <publish_year>1876</publish_year>
  </publication>
</library>

publication1.xml

<library>
  <publication>
    <id>2</id>
    <author>Franz Kafka</author>
    <title>The Metamorphosis</title>
    <genre>Fiction</genre>
    <publish_year>1912</publish_year>
  </publication>
</library>

publication2.xml

<library>
  <publication>
    <id>3</id>
    <author>Herman Melville</author>
    <title>Moby Dick</title>
    <genre>Fiction</genre>
    <publish_year>1851</publish_year>
  </publication>
</library>

publication3.xml

To achieve the goals, let's take the following steps.

 

Step 1: Prepare the mapping design file

This tutorial uses as starting point the BooksToLibrary.mfd mapping from the <Documents>\Altova\MapForce2019\MapForceExamples\Tutorial\ folder. You have already designed this mapping in the Convert XML to New Schema tutorial. To begin, open the BooksToLibrary.mfd file in MapForce, and save it with a new name, in the same folder.

 

Make sure to save the new mapping in the <Documents>\Altova\MapForce2019\MapForceExamples\Tutorial\ folder, because it references several files from it.

mfdBooksToLibrary-BAS

BooksToLibrary.mfd (MapForce Basic Edition)

 

Step 2: Configure the input

To instruct MapForce to process multiple XML instance files, double-click the header of the source component. In the Component Settings dialog box, enter bookentry*.xml as input file.

tut-02-01-BAS

Component Settings dialog box

The asterisk ( * ) wildcard character in the file name instructs MapForce to use as mapping input all the files that have the bookentry- prefix. Because the path is a relative one, MapForce will look for all bookentry- files in the same directory as the mapping file. Note that you could also enter an absolute path if necessary, while still using the * wildcard character.

 

Step 3: Configure the output

To create the file name of each output file, we will use the concat function. This function concatenates (joins) all the values supplied to it as argument.

 

To build the file name using the concat function:

 

1.Search for the concat function in the Libraries window and drag it to the mapping area. By default, this function is added to the mapping with two parameters; however, you can add new parameters if necessary. Click the Add parameter ( gui_add_function_parameter ) symbol inside the function component and add a third parameter to it. Note that clicking the Delete parameter ( gui_delete_function_parameter ) symbol deletes a parameter.

tut-02-02-BAS

2.Insert a constant (on the Insert menu, click Constant). When prompted to supply a value, enter "publication" and leave the String option unchanged.

tut-02-03-BAS

3.Connect the constant with value1 of the concat function.

tut-02-04-BAS

4.Connect the id attribute of the source component with value2 of the concat function.

tut-02-05-BAS

5.Search for the get-fileext function in the Libraries window and drag it to the mapping area. Create a connection from the top node of the source component (File: books.xml) to the filepath parameter of this function. Then create a connection from the result of the get-fileext function to value3 of the concat function. By doing this, you are extracting only the extension part (in this case, .xml) from the source file name.

tut-02-06-BAS

So far, you have provided as parameters to the concat function the three values which, when joined together, will create the generated file name (for example, publication1.xml):

 

Part

Example

The constant "publication" supplies the constant string value "publication".

publication

The attribute id of the source XML file supplies a unique identifier value for each file. This is to prevent all files from being generated with the same name.

1

The get-fileext function returns the extension of the file name to be generated.

.xml

 

You can now instruct MapForce to actually build the file name when the mapping runs. To do this, click the File ( btn_file ) or File/String ( btn_file_string ) button of the target component and select Use Dynamic File Names Supplied by Mapping.

tut-02-07-ENT

You have now instructed MapForce to generate the instance files dynamically, with whatever name will be provided by the mapping. In this particular example, the name is created by the concat function; therefore, we will connect the result of the concat function with the File: <dynamic> node of the target component.

tut-02-08-BAS

If you double-click the target component header at this time, you will notice that the Input XML File and Output XML File text boxes are disabled, and their value shows <File names supplied by the mapping>.

tut-02-09-BAS

This serves as an indication that you have supplied the instance file names dynamically from a mapping, so it is no longer relevant to define them in the component settings.

 

Finally, you need to strip the XML namespace and schema declaration from the target. To achieve this, clear the selection from the Add schema/DTD reference... and Write XML Declaration check boxes on the Component Settings dialog box.

tut-02-10-BAS

You can now run the mapping and see the result, as well as the name of generated files. This mapping generates multiple output files. You can navigate through the output files using the left and right buttons in the upper left corner of the output pane, or by picking a file from the adjacent drop-down list.

tut-02-11-BAS


© 2019 Altova GmbH