Custom Python Libraries
This topic explains how to import a Python module into MapForce. A Python module is a Python file (.py) that contains Python code, such as functions, classes, and variables.
Python functions, classes, and values defined in the module can appear in the Libraries window if they meet the requirements described in this topic. Python functions can then be used in a mapping like any other MapForce function.
Note: Python 3.6 or later must be installed on your machine. The MapForce platform (32-bit, 64-bit) must be the same as that of Python.
Important
•Importing and running Python functions requires MapForce Professional and Enterprise editions.
•Python functions are executed by the Built-In transformation language. They are not available in generated Java, C#, C++, or XSLT code.
•MapForce uses the Python installation configured in Tools | Options | Python. If no path is set in the Options dialog, MapForce attempts to detect a Python installation using the PYTHONHOME and PATH environment variables.
•Only members whose parameters and return values are annotated with a supported type are imported (see supported types below). The following members are not imported:
oMembers whose names begin with an underscore
oMembers that are inherited from a class defined outside the module
•If the .py file changes, MapForce notifies you about it and allows you to reload it. When you reload the module, all functions are imported again.
Automation with MapForce Server
When you run a mapping containing custom Python functions with MapForce Server, you may need to specify a path to the Python home directory. For details, see the description of the run command in the MapForce Server documentation (https://www.altova.com/documentation#mapforceserver).
Automation with FlowForce Server
When you run a mapping containing custom Python functions with FlowForce Server, you may need to configure the Python environment in the .tool file and make the Python module available on the machine where FlowForce Server runs. For details, see the topic Tool Files in the FlowForce Server documentation (https://www.altova.com/documentation#flowforceserver).
Example files
For testing purposes, you can use an example available at MapForceExamples\Python\FormatNumber.mfd, which calls the function defined in Python\Format\Format.py.
The examples used in this topic come from MapForceExamples\Python\Sales\Sales.py.
Python module import
To add a Python module as a MapForce library, take the following steps:
1.Click the Add/Remove Libraries button at the bottom of the Libraries window. The Manage Libraries window opens.
2.To import functions as a local library (in the scope of the current document only), click Add under the current mapping name. To import functions as a global library (at program level), click Add next to Global Library Imports.
3.Search for the .py file you want to import, for example, MapForceExamples\Python\Sales\Sales.py. The imported library is now visible in the Libraries window. Alternatively, double-click a .py file in the Project window to add it as a library.
If you do not see the newly imported library in the Libraries window, make sure that the transformation language is set to Built-In. To add a function to the mapping, drag it from the Libraries window into the mapping area.
Supported Python types
The table below illustrates the correspondence between the supported XML Schema and Python types:
Schema type | Python type |
|---|---|
xs:string | str |
xs:integer | int |
xs:double | float |
xs:boolean | bool |
xs:decimal | decimal.Decimal |
xs:base64Binary | bytes |
MapForce supports passing values from XML Schema to Python and from Python back to XML Schema, using the corresponding types in the table. The Python types can be used for parameters, return values and module or class attributes. In addition, MapForce supports classes defined in the same module, provided that the classes can be used by MapForce (see Objects below).
Functions and methods must have type annotations for their parameters and return values (such as a: int or -> int). If a parameter or return value is unannotated or has an unsupported type, the corresponding function or method is not imported.
Unsupported Python types
Other Python types such as the ones listed below are not supported:
•Imported classes and types other than decimal.Decimal
•list, dict, tuple, set, and other container types
•Generic types such as list[int]
•Optional[T] and Union[...]
•PEP 604 unions such as int | str
•Iterators and generators
•Asynchronous functions and generators
•Variable-length argument lists (*args, **kwargs)
Objects
MapForce supports objects that are instances of usable classes defined in the imported module. MapForce can obtain such an instance from a constructor, a module-level value, or a factory function, and can then pass the instance to functions or use it to call instance methods and read instance properties.
The code listing below shows the CurrencyFormatter class defined in Sales.py.
class CurrencyFormatter: DEFAULT_PATTERN: str = ",.2f"
def __init__(self, prefix: str, pattern: str): self._prefix = prefix self._pattern = pattern
@property def prefix(self) -> str: return self._prefix
def format_amount(self, amount: Decimal) -> str: return self._prefix + format(amount, self._pattern)
@staticmethod def plain(amount: Decimal) -> str: return format(amount, CurrencyFormatter.DEFAULT_PATTERN) |
This class contributes five entries to the Libraries window:
•CurrencyFormatter(): The constructor that creates a new CurrencyFormatter instance.
•CurrencyFormatter.DEFAULT_PATTERN: A class attribute that can be read without an instance.
•CurrencyFormatter.format_amount: An instance method that operates on a CurrencyFormatter instance.
•CurrencyFormatter.prefix: A property that can be read from a CurrencyFormatter instance.
•CurrencyFormatter.plain: A static method that can be called without an instance. Class methods are imported in the same way.
Obtaining and passing instances
MapForce can obtain an instance of a usable class by calling its constructor or through another usable member that returns an instance. For example, an instance can be provided by a module-level value or a factory function:
EURO: CurrencyFormatter = CurrencyFormatter("EUR ", CurrencyFormatter.DEFAULT_PATTERN)
def formatter_for(prefix: str) -> CurrencyFormatter: return CurrencyFormatter(prefix, CurrencyFormatter.DEFAULT_PATTERN)
def format_with(formatter: CurrencyFormatter, amount: Decimal) -> str: return formatter.format_amount(amount) |
In the code listing above, EURO is a module-level value containing a CurrencyFormatter instance, and formatter_for() is a factory function that returns a CurrencyFormatter instance. The format_with() function accepts a CurrencyFormatter instance as a parameter.
Callable objects
A value that holds an instance of a class that defines __call__ can be called like a function (code listing below). MapForce therefore imports such a value as a function and appends () to its name in the Libraries window. In the example below, ROUND_CENTS is a value that can be called like a function. MapForce imports it as ROUND_CENTS() into the Libraries window.
class Rounder: def __init__(self, places: int): self._places = places
def __call__(self, amount: Decimal) -> Decimal: return round(amount, self._places)
ROUND_CENTS: Rounder = Rounder(2) |
Functions and values
A module-level function is imported as a function. A module-level value is imported as a function without parameters that returns the value.
The code listing below shows the definition of module-level values (VAT_RATE and REPORT_TITLE) and module-level functions (net_amount and period).
VAT_RATE: Decimal = Decimal("0.20") REPORT_TITLE: str = "Regional sales"
def net_amount(gross: Decimal) -> Decimal: """Removes VAT from a gross amount.""" return gross / (1 + VAT_RATE)
def period(start: str, end: str) -> str: return start + " - " + end |
The module also contains a function decorated with @functools.lru_cache. MapForce imports such functions as usual and uses the parameters and return type of the original function to determine its interface. The decorator does not change the parameters and return type that MapForce reports for the function.
@functools.lru_cache(maxsize=None) |
Reusing objects and caching
When a function or constructor is used in a mapping, MapForce calls it for each row that passes through it. If a constructor creates an object that is time-consuming to initialize, creating a new instance for every row can be inefficient.
To reuse the same object, create it once at module level and connect that value where the instance is needed:
EURO: CurrencyFormatter = CurrencyFormatter("EUR ", CurrencyFormatter.DEFAULT_PATTERN) |
Connecting EURO to the first parameter of CurrencyFormatter.format_amount creates the object once, when the module is loaded, and reuses this object for every row. Module-level values are read fresh each time they are used. Therefore, if Python code updates a module-level value, MapForce reads the updated value on the next use.
You can also cache the results of expensive function calls. A module-level dictionary or @functools.lru_cache stores previously returned results:
@functools.lru_cache(maxsize=None) def region_name(code: str) -> str: """Expands a region code. The lookup table is built on first use only.""" return _region_names().get(code, code) |
The @functools.lru_cache decorator causes region_name() to execute once per distinct region code. If subsequent rows use a code that has already been processed, the cached function returns the cached result instead of executing the function again.
Functions and other names that start with an underscore, such as _region_names, are considered private to the module and are not imported by MapForce.
Keep in mind the following points:
•Module-level objects and cached values remain available for the duration of the mapping. They start fresh when the library is imported again.
•MapForce evaluates a mapping on a single thread, so a module-level cache does not require locking.
•Avoid using @functools.lru_cache on an instance method when a new instance is created for each row. In this case, the cache is associated with the instances and can continue to grow. Use a module-level function for caching instead.
•MapForce can read module-level and class attributes but cannot assign values to them. Use Python code to update these values.