IMPORTANT:
this is not a Support Forum! Experienced users might answer from time to time questions posted here. If you need a professional and reliable answer, or if you want to report a bug, please contact Altova Support instead.

Profile: mmt
About
User Name: mmt
Forum Rank: Newbie
Real Name:
Location
Occupation:
Interests:
Gender: None Specified
Statistics
Joined: Wednesday, May 9, 2018
Last Visit: Thursday, January 24, 2019 9:37:16 AM
Number of Posts: 4
[0.02% of all post / 0.00 posts per day]
Avatar
Last 10 Posts
Topic: Optimizing datamapping from XSLX to n XML files
Posted: Thursday, January 24, 2019 8:44:27 AM
Dear Forum.

My source file is XLSX with two sheets, called contacts and accounts.
If a contact is related to an account, then a field in the contact sheet is referring to an unique account id in the account sheet.

For generating XML I have a XSD that describes the target component XML elements.
I have then mapped the XLSX fields to the corresponding XML elements.
As the generated file is too big (to be used in a different process), I am splitting up the data into 5 or so XML files.
I do this at the moment by counting to 5000 contacts and then presume creating the next file.

My problem though is that the current Mapforce model transfers all the accounts to all 5 XML files, so each file has 5000 unique contacts, but identical accounts in all XML files.

What I have tried is to use a "Equal" and a "Filter" component.
I use the account id from the source contact account id to "a" in the "Equal" component and the account id from the source account id to "b" in the "Equal" component.
The boolean result is then passed to "Filter" component where it should only return when boolean is true, thus inserting the account id to the respective XML.

When I try to filter the accounts to be excluded in the respective 5 XML's I run into performance issues... it runs too slowly so I know I am doing something wrong.

Could any of you help with an example that shows how I can create n XML files where all the references from the 5000 contacts to accounts only include the exact accounts needed in the file?

Notice!
Bear in mind that the accounts can be referenced to from different contacts so the accounts can be part of one or more of the XML's files, just as long as the contacts in the file are referring to them.

Let me know if I need to add more insights to the problem or details of the model/XSD etc.
THANKS!


<picture XML file split>
<picture of XLSX source: account sheet>
<picture of XLSX source: contact sheet>
<picture of XML target: reference to account id from contact element and id from account element>
<picture of filtering accounts before inserting into XML target>

Topic: Duplicate output file <> detected
Posted: Wednesday, May 9, 2018 11:01:29 AM
Update:
I found the problem.
I had a direct reference from the data source table (field "Migration Project Number") to the concat function which led to this problem.
I fixed it by using another "group by" function for this field as key input and key output to concat function.
Topic: Duplicate output file <> detected
Posted: Wednesday, May 9, 2018 9:33:49 AM
K101 wrote:
Get a list of the computed output file names by connecting your concat's output to a simple text component. Then you can see if you do, indeed, have any duplicates.

If you need more help it's probably best to contact Altova Support directly and send them your actual files.


But as you see in the second screenshot, There is no duplicate reference to the same file.
It shows three lines; one with "Germany" one with "Russia" and then an empty line which I don't know if is merely a visual thing or an actual line.

Thanks.
Topic: Duplicate output file <> detected
Posted: Wednesday, May 9, 2018 9:14:23 AM
Hi all

I'm trying to create multiple files per country I have in my dataset.
My problem is merely cosmetical but nonetheless annoying as Mapforce is throwing an error about duplicate output file detected.

Here is the error message:
Duplicate output file '2018-P04 Content List Germany.xlsx' detected

The source files are two XLSX files, and the destination files are 5 different country XLSX files.
Looking at the data in the output files I can't see any missing data, so why is it throwing an error?

Use of the Altova User Forum(s) is governed by the Altova Terms of Use.