Altova MapForce Server 2024 Advanced Edition

Preparing Mappings for Server Execution

Home Prev Top Next

A mapping designed and previewed with MapForce may refer to resources which are outside of the current machine and operating system (such as databases). In addition to this, in MapForce, all mapping paths follow Windows-style conventions by default. Thirdly, the machine where MapForce Server runs might not support the same database connections as the machine where the mapping was designed. For this reason, running mappings in a server environment typically requires some preparation, especially if the target machine is not the same as the source machine.

 

Note:The term "source machine" refers to the computer where the MapForce is installed and the term "target machine" refers to the computer where MapForce Server or FlowForce Server is installed. In the most simple scenario, this is the same computer. In a more advanced scenario, MapForce runs on a Windows machine whereas MapForce Server or FlowForce Server runs on a Linux or macOS machine.

 

As best practice, always make sure that the mapping validates successfully in MapForce before deploying it to FlowForce Server or compiling it to a MapForce Server execution file.

 

If MapForce Server runs standalone (without FlowForce Server), the required licenses are as follows:

 

On the source machine, MapForce Enterprise or Professional edition is required to design the mapping and compile it to a server execution file (.mfx).

On the target machine, MapForce Server or MapForce Server Advanced Edition is required to run the mapping.

 

If MapForce Server runs under FlowForce Server management, the following requirements apply:

 

On the source machine, MapForce Enterprise or Professional edition is required to design the mapping and deploy it to a target machine.

Both MapForce Server and FlowForce Server must be licensed on the target machine. The role of MapForce Server is to run the mapping; the role of FlowForce is to make the mapping available as a job which benefits from features such as scheduled or on demand execution, execution as a Web service, error handling, conditional processing, email notifications, and others.

FlowForce Server must be up and running at the configured network address and port. Namely, the "FlowForce Web Server" service must be started and configured to accept connections from HTTP clients (or HTTPS if configured) and must not be blocked by the firewall. The "FlowForce Server" service must also be started and running at the designated address and port.

You have a FlowForce Server user account with permissions to one of the containers (by default, the /public container is accessible to any authenticated user).

 

General considerations

 

If you intend to run the mapping on a target machine with standalone MapForce Server, all input files referenced by the mapping must be copied to the target machine as well. If MapForce Server runs under FlowForce Server management, there is no need to copy files manually. In this case, the instance and schema files are included in the package deployed to the target machine.

If the mapping includes database components which require specific database drivers, such drivers must be installed on the target machine as well. For example, if your mapping reads data from a Microsoft Access database, then Microsoft Access or Microsoft Access Runtime (https://www.microsoft.com/en-us/download/details.aspx?id=50040) must be installed on the target machine as well.

When you deploy a mapping to non-Windows platforms, ADO, ADO.NET and ODBC database connections are automatically changed to JDBC. Native SQLite and native PostgreSQL connections are preserved as such and require no additional configuration. See also "Database connections" below.

If the mapping contains custom function calls (for example, to .dll or .class files), such dependencies are not deployed together with the mapping, since they are not known before runtime. In this case, copy them manually to the target machine. The path of the .dll or .class file on the server must be the same as in the "Manage Libraries" window in MapForce, for example:

inc-mapforce-dll-library

Some mappings read multiple input files using a wildcard path. In this case, the input file names are not known before runtime and so they are not deployed. For the mapping to execute successfully, the input files must exist on the target machine.

If the mapping output path includes directories, those directories must exist on the target machine. Otherwise, an error will be generated when you execute the mapping. This behavior is unlike MapForce, where non-existing directories are generated automatically if the option Generate output to temporary files is enabled.

If the mapping calls a Web service that requires HTTPS authentication with a client certificate, the certificate must be transferred to the target machine as well, see Digital Certificate Management.

If the mapping connects to file-based databases such as Microsoft Access and SQLite, the database file must be manually transferred to the target machine or saved to a shared directory which is accessible to both the source and the target machine and referenced from there, see "File-based databases" below.

 

Making paths portable

If you intend to run the mapping on a server, ensure that the mapping follows the applicable path conventions and uses a supported database connection.

 

To make paths portable to non-Windows operating systems, use relative instead of absolute paths when designing the mapping in MapForce:

 

1.Open the desired mapping design file (.mfd) with MapForce on Windows.

2.On the File menu, select Mapping Settings, and clear the Make paths absolute in generated code check box if it is selected.

3.For each mapping component, open the Properties dialog box (by double-clicking the component's title bar, for example), and change all file paths from absolute to relative. Also, select the Save all file paths relative to MFD file check box. For convenience, you can copy all input files and schemas into the same folder as the mapping itself, and reference them just by the file name.

 

For more information about dealing with relative and absolute paths while designing mappings, refer to MapForce documentation.

 

Importantly, both MapForce Server and FlowForce Server support a so-called "working directory" against which all relative paths will be resolved. The working directory is specified at mapping runtime, as follows:

 

In FlowForce Server, by editing the "Working-directory" parameter of any job.

In MapForce Server API, through the WorkingDirectory property of the COM and .NET API, or through the setWorkingDirectory method of the Java API.

In MapForce Server command line, the working directory is the current directory of the command shell.

 

Database connections

Be aware that ADO, ADO.NET, and ODBC connections are not supported on Linux and macOS machines. Therefore, if the target machine is Linux or macOS, such connections are converted to JDBC when you deploy the mapping to FlowForce or when you compile the mapping to a MapForce Server execution file. In this case, you have the following options before deploying the mapping or compiling it to a server execution file:

 

In MapForce, create a JDBC connection to the database

In MapForce, fill the JDBC database connection details in the "JDBC-specific Settings" section of the database component.

 

If the mapping uses a native connection to a PostgreSQL or SQLite database, the native connection is preserved and no JDBC conversion takes place. If the mapping connects to a file-based database, such as Microsoft Access and SQLite, additional configuration is required, see "File-based databases" below.

 

Running mappings with JDBC connections requires that the Java Runtime Environment or Java Development Kit be installed on the server machine. This may be either Oracle JDK or an open source build such as Oracle OpenJDK.

 

The JAVA_HOME environment variable must point to the JDK installation directory.

On Windows, a Java Virtual Machine path found in the Windows registry will take priority over the JAVA_HOME variable.

The JDK platform (64-bit, 32-bit) must be the same as that of MapForce Server. Otherwise, you may get an error with the reason: "JVM is inaccessible".

To set up a JDBC connection on Linux or macOS:

1.Download the JDBC driver supplied by the database vendor and install it on the operating system. Make sure to select the 32-bit version if your operating system runs on 32-bit, and the 64-bit version if your operating system runs on 64-bit.

2.Set the environment variables to the location where the JDBC driver is installed. Typically, you will need to set the CLASSPATH variable, and possibly a few others. To find out which specific environment variables must be configured, check the documentation supplied with the JDBC driver.

 

Note:On macOS, the system expects any installed JDBC libraries to be in the /Library/Java/Extensions directory. Therefore, it is recommended that you unpack the JDBC driver to this location; otherwise, you will need to configure the system to look for the JDBC library at the path where you installed the JDBC driver.

 

Oracle Instant Client connections on macOS

These instructions are applicable if you connect to an Oracle database through the Oracle Database Instant Client, on macOS. Prerequisites:

 

Java 8.0 or later must be installed. If the Mac machine runs a Java version prior to Java 8, you can also connect through the JDBC Thin for All Platforms library, and disregard the instructions below.

Oracle Instant Client must be installed. You can download the Oracle Instant Client from the Oracle official download page. Note that there are several Instant Client packages available on the Oracle download page. Make sure to select a package with Oracle Call Interface (OCI) support, (for example, Instant Client Basic). Also, make sure to select the 32-bit version if your operating system runs on 32-bit, and the 64-bit version if your operating system runs on 64-bit.

 

Once you have downloaded and unpacked the Oracle Instant Client, edit the property list (.plist) file shipped with the installer so that the following environment variables point to the location of the corresponding driver paths, for example:

Variable

Sample Value

CLASSPATH

/opt/oracle/instantclient_11_2/ojdbc6.jar:/opt/oracle/instantclient_11_2/ojdbc5.jar

TNS_ADMIN

/opt/oracle/NETWORK_ADMIN

ORACLE_HOME

/opt/oracle/instantclient_11_2

DYLD_LIBRARY_PATH

/opt/oracle/instantclient_11_2

PATH

\$PATH:/opt/oracle/instantclient_11_2

Note:Edit the sample values above to fit the paths where Oracle Instant Client files are installed on your operating system.

 

File-based databases

File-based databases such as Microsoft Access and SQLite are not included in the package deployed to FlowForce Server or in the compiled MapForce Server execution file. Therefore, if the source and target machine are not the same, take the following steps:

 

1.In MapForce, right-click the mapping and clear the check box Make paths absolute in generated code.

2.Right-click the database component on the mapping and add a connection to the database file using a relative path. A simple way to avoid path-related issues is to save the mapping design (.mfd file) in the same directory as the database file and to refer to the latter from the mapping just by file name (thus using a relative path).

3.Copy the database file to a directory on the target machine (let's call it "working directory"). Keep this directory in mind since it will be required to run the mapping on the server, as shown below.

 

To run such mappings on the server, do one of the following:

 

If the mapping will be run by MapForce Server under FlowForce Server control, configure the FlowForce Server job to point to the working directory created previously. The database file must reside in the working directory. For an example, see "Exposing a Job as a Web Service" (https://www.altova.com/manual/en/flowforceserveradvanced/2024.0/index.html?fs_example_web_service.htm).

If the mapping will be run by standalone MapForce Server at the command line, change the current directory to the working directory (for example, cd path\to\working\directory) before calling the run command of MapForce Server.

If the mapping will be run by the MapForce Server API, set the working directory programmatically before running the mapping. To facilitate this, the property WorkingDirectory is available for the MapForce Server object in the COM and .NET API. In the Java API, the method setWorkingDirectory is available.

 

If both the source and the target machines are Windows machines running on the local network, an alternative approach is to configure the mapping to read the database file from a common shared directory, as follows:

 

1.Store the database file in a common shared directory which is accessible by both the source and the target machine.

2.Right-click the database component on the mapping and add a connection to the database file using an absolute path.

 

Global Resources

If a mapping includes references to Global Resources instead of direct paths or database connections, you will be able to use Global Resources on the server side as well. When you compile a mapping to a MapForce Server execution file (.mfx), the references to Global Resources will be kept intact, so that you can provide these on the server side, at mapping runtime. When deploying a mapping to FlowForce Server, you can optionally choose whether it should use resources on the server.

 

For mappings (or mapping functions, in case of FlowForce Server) to run successfully, the actual file, folder, or database connection details that you supply as Global Resources must be compatible with the server environment. For example, files and folders paths must use the Linux convention for paths if the mapping will run on a Linux server. Likewise, Global Resources defined as database connections must be possible on the server machine.

 

For further information, see Global Resources.

 

XBRL Taxonomy Packages

When you deploy a mapping that references XBRL Taxonomy Packages to FlowForce Server, MapForce collects all external references from the mapping and then resolves them using the current configuration and currently installed taxonomy packages. If there are resolved external references that point to a taxonomy package, then the taxonomy package is deployed together with the mapping. FlowForce Server will use that package—as it was during deployment—to execute the mapping. To refresh the taxonomy package used by FlowForce Server, you will need to change it in MapForce and redeploy the mapping.

 

Note that the root catalog of MapForce Server influences the way taxonomies are resolved on the target machine. The root catalog is found at the following path relative to the MapForce Server installation directory: etc/RootCatalog.xml.

 

Taxonomy packages that were deployed with a mapping will be used if the root catalog of MapForce Server does not already contain such a package or does not contain a package that is defined for the same URL prefix. The root catalog of MapForce Server has priority over the deployed taxonomy.

 

If MapForce Server runs standalone (without FlowForce Server), it is possible to specify the root catalog that should be used by the mapping as follows:

 

At the command line, this is possible by adding the option -catalog to the run command.

In the MapForce Server API, call the method SetOption, and supply the string "catalog" as first argument, and the path to the root catalog as second argument.

 

If a mapping uses XBRL components with table linkbases, the taxonomy package or the taxonomy package configuration file must be supplied to the mapping at runtime, as follows:

 

At the MapForce Server command line, add the option --taxonomy-package or --taxonomy-packages-config-file to the run command.

In the MapForce Server API, call the method SetOption. The first argument must be either "taxonomy-package" or "taxonomy-packages-config-file". The second argument must be the actual path to the taxonomy package (or taxonomy package configuration) file.

© 2017-2023 Altova GmbH