Protocol Buffers

www.altova.com Print this Topic Previous Page Up One Level Next page

Home >  Data Sources and Targets >

Protocol Buffers

Binary files are a large category of not human-readable files that are proprietary in many cases, and not typically intended for consumption by data mapping software like MapForce. However, starting with 2019 release, MapForce supports mapping data to or from binary files encoded in Protocol Buffers format (https://developers.google.com/protocol-buffers/). Specifically, MapForce (or MapForce Server) can read data from such binary files and convert it to any other format supported by MapForce (for example, XML, CSV, database, and so on). Likewise, you can read data from any format supported by MapForce and write it as a Protocol Buffers binary file. In addition, you can use Protocol Buffers files in Web service requests and responses, see Adding a Web Service Call (REST-Style).

 

Note the following:

 

To read or write data to binary files in Protocol Buffers format, the transformation language of the mapping must be set to "Built-In". Code generation in C#, C++, Java, XSLT, or XQuery is not supported. For more information, see Selecting a Transformation Language.
To make data exchange possible, binary files in Protocol Buffers format are accompanied by so-called .proto files. MapForce supports Protocol Buffers .proto files version 2 and 3.

 

A .proto file defines the structure of the encoded binary data, similar to how an XML schema describes the structure of an XML instance file. For example, the following code listing illustrates a .proto file that describes a person object:

 

syntax="proto3";

 

message Person {

 string name = 1;

 int32 id = 2;

 string email = 3;

}

Example .proto file

The .proto file makes it possible to interpret and process the corresponding binary file. Therefore, in order for MapForce to read data in Protocol Buffers format, you need the following:

 

1.The source binary file. This file may have an extension like .bin or .dat, or any other extension.
2.The .proto file.

 

To write data in Protocol Buffers format with MapForce, all you need is the .proto file. You can either write your .proto files manually, or get them from whoever expects you to create data in the respective format. In either case, when you run the mapping, MapForce (or MapForce Server) generates the binary file based on the .proto file.

 

Under normal circumstances, in order to create or read data from Protocol Buffers files outside MapForce, you would need to generate and write program code. With MapForce, however, you do not need to write code or generate it from .proto files—just add the .proto file to the mapping, and draw the required connections visually. Once you run the mapping, MapForce (or MapForce Server, if applicable) will read data from the source binary file(s) or generate binary files, according to the mapping design. For example, the image below illustrates a mapping that reads data from a database and writes it as multiple binary files in Protocol Buffers format.

mf_proto_04

GenerateOrders.mfd

The mapping above is discussed in more detail in Example: Write Data to Protocol Buffers.


© 2019 Altova GmbH