XML Schema Archives

Tag Archive for: XML Schema

New! XSLT Technology Training

February 27, 2012/in XPath /by Erin Cavanaugh

We’re excited to introduce our new XSLT Technology training as the latest Altova Online Training offering. As with all our trainings, XSLT Technology is released as a free, self-paced course, available online, so students can fit it into their busy schedules.

XSLT transforms XML data into other formats, and this course will transform a beginner XML student into an advanced user. Intermediate and advanced students will gain valuable techniques to add to their XML toolkits.

Processing the Groupon API with MapForce – Part 2

August 16, 2011/6 Comments/in Data Integration /by David McGahey

In Part 1 of this series we described how to connect Altova MapForce to the Groupon API. We queried the API for a list of Groupon divisions, then used the list to create API queries for all the current deals from every division. In this part, we will execute the /deals queries and filter the response for the most interesting data. The list of /deals queries we built previously looks like this: To process all the queries, we can connect the list as a dynamic file input to a new mapping component. When we needed a new component last time, we dropped an API /divisions query into the mapping, and let MapForce create an XML Schema automatically. We could do the same thing here by dropping in an API /deals query as an XML input file. There’s just one small issue — although the Groupon API online documentation clearly describes the queries we can make, it is vague about the information that will be returned. Before we send dozens of queries to the API for all the current deals, we probably want to know a little more about the data that will come back.

Let’s Make a Deal Like Yogi Berra said, you can observe a lot just by looking. Let’s start by running a /deals query in XMLSpy. That will let us examine the response to a query for one division before we pull in a potentially unwieldy volume of data. The XMLSpy File / Open menu includes the same Switch to URL option we used in MapForce in the earlier post. If we enter the /deals API query for a division that covers a large metro area – say Dallas – we are likely to get enough deals instances to extrapolate the characteristics of the entire data set. XMLSpy opens the response to the /deals API query in Text view just as if we opened a local file: As expected, we got quite a bit of data when we requested all the deals for a single division! A fast way to analyze the structure of this data is to use the XMLSpy DTD / Schema menu option to generate an .xsd file from the xml. Shown below is a reduced view of the entire generated .xsd file based on the response to the /deals query for Dallas: We can dig even deeper, following Yogi’s advice like déjà vu all over again. Expanding all the elements to review the XML Schema reveals some curious anomalies. For instance, there are two elements named redemptionLocation with different definitions. The first contains a sequence of child elements: And the second is defined as a simple string: Going back into the xml data for Dallas and searching for redemptionLocation displays these examples: And: And: Now this is really interesting, because redemptionLocation = ”online” identifies deals that can be redeemed from anywhere, instead of by a visit to a bricks and mortar location in the division where they are advertised. What if we ran the /deals API queries for all divisions and extracted a list of all the online deals? That would be one extreme Groupon! Only Ask for What You Need The Groupon /deals API query supports an optional parameter called &show= that allows users to limit the data returned. Applying this parameter can save bandwidth and reduce processing time for the data transformation by removing unwanted data from the API response. We can also simplify our final result by including only the most interesting information, including the link to the Groupon web page for each deal. After we remove unwanted elements from the generated Dallas schema, our final version for the summary of online deals looks like this: When we add the &show= parameter to our MapForce mapping to request only the elements included in the simplified XML Schema, the queries look like this: Now we can drop the revised .xsd file into the mapping and connect the list of API /deals queries as dynamic input. We don’t need to delete the text file we used to collect the list of queries — that might continue to be helpful for future debugging. These changes complete the input side of the data mapping. Defining the Data Transformation Output Back in XMLSpy we can make a couple more revisions to the input XML Schema to design a new version for output: We discarded the response element since it doesn’t add any value, and eliminated the redemptionLocation element that we don’t intend to include in the output. We also added a date element for a timestamp, because our output file will be a snapshot of data that is constantly changing. After saving this version of the .xsd file in XMLSpy, we can drop it into the MapForce mapping. Shown below is the output side of the mapping with the output component partially connected. The filter at the top reads the redemptionLocation element to select only online deals and the now function inserts the date: The last revision we made in the output XML Schema was to change several element types from dateTime, Boolean, and integer to the string data type to allow more descriptive text Here is the complete definition of the mapping with the final connections to the output component: Now for the Payoff When we click the Output button MapForce processes the entire mapping from beginning to end using the MapForce Built-in execution engine. Here’s a breakdown of the steps:

Run the /divisions query to get the current list of divisions
Concatenate strings to build the list of /deals queries for all divisions
Run the /deals queries to create dynamic data for the input component
Filter for online deals to generate the output component, execute the remaining mapping functions, and add the timestamp after all the deals are processed

MapForce takes only a few seconds to complete all those steps and generate an output file with a series of deals that look like this: In part 3 of this series we’ll design a stylesheet to automatically transform the XML output of our mapping into html for attractive presentation in a web browser and on mobile devices. See ya at the ballpark, Yogi! XMLSpy and MapForce are available together in the specially priced Altova MissionKit. See for yourself how easy it is to use the MissionKit to convert data from a Web API — download a free 30-day trial!
Editor’s Note: Our original series on mapping data from the Groupon API ran in three parts you can see by clicking the links here: Part 1 of Processing the Groupon API with Altova MapForce describes how to create dynamic input by collecting data from multiple URLs. Processing the Groupon API with MapForce – Part 2 describes how we filtered data from the API and defined the output to extract only the most interesting details. Processing the Groupon API – Part 3 describes formatting the output as a single HTML document optimized for desktop and mobile devices, and reviews ways to automate repeat execution.

Tags: Altova XMLSpy, data integration, data mapping, MapForce, MissionKit, XML Editor, XML Schema, XMLSpy

Processing the Groupon API with Altova MapForce

August 2, 2011/2 Comments/in Data Integration, JSON /by David McGahey

We often think of a data integration project as a translation from one singular data input file to some other data set, but Altova MapForce lets you greatly expand the concept of an input file. For instance, the MergeMultipleFiles.mfd example installed with MapForce illustrates how you can use a filename with wildcard characters to merge multiple input files into a single output. A MapForce mapping input doesn’t even need to be a physical file – it can be a URL that returns predictable structured data, like the APIs for popular Web sites like Groupon and many others.

In this blog post we’ll describe how to use Altova tools to retrieve, filter, analyze, and present data available from a Web-based API, using Groupon as an example. If you want to follow along yourself, you will first need to visit http://www.groupon.com/pages/api to request your personal Groupon API client key. The Problem: All Deals Are Local The Groupon Web site and email subscriptions are great for finding deals in your local neighborhood, but what if you’re looking for a deal to use on an upcoming vacation, or for a gift for friends or family across the country? Sure, you could enter each location manually at the Groupon Web page, but that’s so last century. Let’s use the Altova MissionKit to automate things. The Groupon API offers two URL queries that return data in .json or .xml formats: the first returns a list of all Groupon localities (called divisions), and the second returns current deals information for one named division. If we want to see all the deals for more than one division, we need to resolve multiple URLs and aggregate the data into a single result. Yes, MapForce can do that! First We Need a Schema The Groupon API documentation describes the elements that will be returned by our requests, but doesn’t provide an XML Schema. That’s okay, we can use MapForce to generate one. All we have to do is open a new mapping design and choose Insert XML Schema/File, then click the Switch to URL button. Now we can enter the URL to retrieve the Groupon divisions list: When we click the Open button MapForce offers to generate the schema: When we click Yes, the File / Save dialog opens. I saved the schema as divisions.xsd, and the mapping with the new XML Schema inserted looks like this: And the Properties dialog for the XML Schema component automatically contains the API /divisions URL as the Input XML File: Check the Work We want to filter the Groupon divisions data to build a list of id names to use for deal queries for each locality. But before we go any further, now might be a good time to apply the text file trick from the Quick Solution for Complicated Functions blog post to look at the id values. When we insert the text file and connect the divisions and id schema elements, the mapping looks like this: We connected the division element to Rows in the text file in order to generate a new row in the text file for each unique division, so that Field1 in each row will hold the id. Clicking the Output button now generates this result: All we need to do is apply the concat string function to build the list of /deal URLs for all division IDs. The next step in the mapping looks like this: Rolling the cursor over the constant connected to value1 of the concat function displays its full definition: When we click the Output button to execute the mapping, the Output file now looks like this: As a further review, we can open the generated XML Schema in XMLSpy and display it in graphical Schema View: So far we have:

built a MapForce mapping that queries the Groupon API for all divisions
extracted the division id fields
and built a list of URLs for API queries to get the deals in each division

In the next post in this series we will process the list of deal queries as the input for a new mapping component and filter the output for some interesting information. Find out for yourself how easy it is to apply MapForce to convert data from a Web API! Download a free 30-day trial of MapForce.
Editor’s Note: Our original series on mapping data from the Groupon API ran in three parts you can see by clicking the links here: Part 1 of Processing the Groupon API with Altova MapForce describes how to create dynamic input by collecting data from multiple URLs. Processing the Groupon API with MapForce – Part 2 describes how we filtered data from the API and defined the output to extract only the most interesting details. Processing the Groupon API – Part 3 describes formatting the output as a single HTML document optimized for desktop and mobile devices, and reviews ways to automate repeat execution.

Tags: Altova XMLSpy, data integration, data mapping, MapForce, MissionKit, Web services, XML Schema

New XML Schema Editing Tools in XMLSpy

March 2, 2011/in Tools /by Erin Cavanaugh

Let’s take a look at some of the new features for working with XML Schemas in the latest release of XMLSpy.

Sorting in Schema View

When you’re working in the graphical XML Schema editor, you can now sort some or all of the schema components alphabetically with a single click. This is a huge time saver when you need to organize large schemas that have evolved over time and for understanding new schemas that come across your desk. Simply click the A-Z button in the Schema Overview window and select the sort options you prefer. Your components are instantly sorted in the graphical view… …and when you click over to Text View, the schema code itself is reordered and organized.

Schema Refactoring

Another new feature that’s invaluable when working with complex schemas or ones inherited from other developers is schema refactoring support. Taking advantage of XMLSpy’s powerful Find in Schemas functionality, you can quickly locate all the instances of a global element or type across a schema – and all imported and included schemas – and then rename that component in each place it occurs. This makes schema refactoring easy while ensuring the validity of all impacted schemas.

Intelligent Support for Changing Types

You may also find the need to change the type of an element or a base type, which previously meant reconfiguring all the facets associated with that component. This new feature gives you the option to preserve any facets or attributes that are compatible with the new type. Rename components to refactor schema

Customizable XML Schema Documentation

Another option schema developers have requested quite often is the ability to customize the comprehensive XML Schema documentation generated by XMLSpy. This is now possible via integration with StyleVision. The StyleVision stylesheet design tool provides countless advanced options for customizing the documentation templates, from adding your company’s logo and branding to changing the appearance and organization of the documentation completely. StyleVision integration also gives you the option to generate documentation in PDF in addition to the HTML, Word, and RTF output options available for the fixed documentation in XMLSpy. Here’s an an example of XML Schema documentation we customized for the fictional Nanonull corporation: Embed image in XML XMLSpy 2011r2 also ships with several useful documentation templates that can be used as is or customized further in StyleVision:

XML Schema Structure Overview: documents the structure of global elements and complex types to the number of levels you specify
All XML Schema Type Derivations: displays all simple and complex types and their derived types in a tree format
XPath List: generates all possible XPath statements that are possible for a schema based on a user-specified number of levels
Statistics: lists the numbers of elements, types, attributes, groups, etc., for the overall schema, and for each associated schema file

To use a custom schema documentation template, you must have XMLSpy and StyleVision installed. If you are a MissionKit customer, you already have both tools and can take advantage of this feature and countless other options for working with XML Schema, from stylesheet and report design, to data mapping, and more. If you aren’t currently a StyleVision customer, grab a free trial from our Web site. The XBRL and WSDL documentation generated by XMLSpy may also be customized using StyleVision. As always, we rely on your input and feature requests when planning each release – please let us know what you think of these new features and what you’d like to see in the next product version by leaving a comment here on the blog.

Tags: v2011r2, XML Editor, XML Schema, XMLSpy

Updated Industry Schema Library

May 26, 2010/in XBRL /by Niki Devgood

“The nice thing about standards is that there are so many to choose from.” – Andrew S. Tanenbaum (attr.)

Maybe we can help. Altova’s updated schema library is a collection of over 100 industry and cross-industry XML Schema and DTD specifications – all in one central location. XML standards (and technology standards in general) are used to promote information sharing and interoperability across disparate software and systems. In a perfect world, this would translate to a global network of data being exchanged seamlessly between information partners… in a perfect world. However, there are some industries that are reaping the rewards of standardization, and hopefully these efforts will pave the way for more. Very recently we have seen an increased interest and even some actionable mandates in XBRL for financial data, HL7 for exchanging healthcare messages, and NIEM for inter-agency communication within the United States government. Of course, if you do find yourself working with some conflicting technology standards, you should probably take a look at Altova MapForce. With native support for visually mapping pretty much any data formats you can think of (XML, databases, flat files, EDI, Excel 2007+, XBRL, and Web services), easily adding data processing functions, and a whole bunch of automation options, you may even find that creating data integration solutions is well… kind of fun. Check out Altova’s Industry Schema Library, or download a free 30-day trial of MapForce today!

Tags: Altova, data mapping, EDI, HL7, NIEM, XBRL, XML Schema

Altova Adds to NIEM Support in v2010r3

May 10, 2010/2 Comments/in Technology /by Niki Devgood

Clashes between government agencies are the stuff of legends and entire TV shows: the FBI and the CIA, the local sheriff’s department and the big city CSI, Homeland Security and the Pentagon, Jack Bauer and CTU. Perhaps this has to do with the territorial nature of some of these entities, but perhaps, just perhaps, technology has a part to play in these conflicts. Incompatible architectures and data formats, legacy systems, and other technology road blocks within the United States government sometimes made it difficult for agencies to efficiently share pertinent information such as arrest reports, amber alerts, immigration details, and more.
The National Information Exchange Model (NIEM) is an XML-based standard for data exchange between United States government agencies and their information partners. It aims to provide a means for these entities to improve decision making, achieve greater operating efficiency, mitigate risk, improve public safety, and increase ROI through intergovernmental information exchange.
As a leader in XML tools and technologies, the Altova MissionKit already inherently supports NIEM on a variety of different levels. In v2010r3, we added two very specific features to XMLSpy that extend the XML editor with capabilities that are invaluable for NIEM development. For more on these features, see the recent NIEM blog post by our CEO and XML Aficionado, Alexander Falk.
Below is a brief overview of the support provided for NIEM across the MissionKit, from XML Schema editing in XMLSpy, to XMI import and UML diagramming in UModel, to data integration in MapForce, and beyond.

XMLSpy

XMLSpy® 2010 provides a graphical schema design interface which simplifies the development of XML Schemas, allowing XML code to be generated in real-time behind the scenes. Users can switch back and forth between this view and text view at any time during their design process.

The graphical schema editor provides built-in support for creating the IEPD Extension Schema, Constraint Schema, and Exchange Schema.

In addition to the broad XML Schema support described above, XMLSpy provides comprehensive support for all other prevalent XML technologies, including:

Graphical and text-based editing views
Sophisticated XML Schema and XML instance validation
XML instance document creation
XSLT editor, debugger, and profiler
Web services tools (WSDL editor, SOAP client, SOAP debugger)

UML Diagrams

The first stage in any development lifecycle is a thorough business requirements review. UML is particularly well-suited to modeling software projects because it provides a standardized approach to the design process. Scenario-based planning is recommended for NIEM, and though there is no UML diagram requirement, class diagrams, use-case diagrams, and sequence diagrams all provide value to IEPD documentation. In addition, NIEM provides a free tool for mapping data requirements based on an uploaded XMI representation of a UML model
UModel® 2010 is an advanced UML tool that supports all fourteen UML 2.3 diagrams, BPMN, SysML, and more. It works seamlessly with XML technologies like XMI, letting users easily import and export XMI based on their UML diagrams. UModel can even render XML Schemas in a format similar to UML diagrams and integrates seamlessly with all other MissionKit tools including XMLSpy, MapForce, and SchemaAgent. UModel also supports code generation, round-trip engineering, and reverse engineering, as well as integration with the popular Visual Studio and Eclipse IDEs.
UModel can be used two different phases of NIEM development:

Scenario Planning – to model and document with use case diagrams, sequence diagrams, and business process modeling
Map & Model – to develop class diagrams to create the exchange content model

XML Schema Management

Because of the potential to have many different schemas in each IEPD, XML Schema management can become an arduous process. SchemaAgent® 2010 is a unique file management tool that lets users visually manage their XML Schema, XML, XSLT, and WSDL assets in a project-based environment so that changes and structure of the file configuration can be easily recognized. SchemaAgent users can even view mapping files created using MapForce.

Data Mapping & Exchange

Data mapping is a crucial step at several levels in the NIEM process. Mapping helps users generate a subset schema to determine what elements can be used from the core and domain models, it is also used to perform a transform from one NIEM vocabulary to another. Some NIEM implementations also employ Web services to provide a mechanism for the exchange.
MapForce® 2010 is an any-to-any data mapping and integration tool that supports all of the potential data formats used in the NIEM lifecycle including XML, databases, flat files, Excel 2007+, and Web services. MapForce maps data based on its underlying structure (i.e. XML Schema), enabling mapping designs to be reused for recurrent transformations. MapForce also supports mapping to and from multiple source or target components. In addition, MapForce can be used to generate documentation detailing mapping projects for non-technical stakeholders in a variety of formats such as HTML and Microsoft® Word.

To try out this functionality on your own, download a free 30-day trial of Altova MissionKit now!

Tags: MapForce, MissionKit, NIEM, SchemaAgent, UModel, XML Editor, XML Schema, XMLSpy

A Developer’s Life is Never Simple

December 22, 2009/in /by David McGahey

Earlier this month in our blog post on Comparing XML Schemas we showed a realistic – but simplified – example to illustrate a slick new feature of DiffDog 2010 to compare XML Schemas and update corresponding XML data files by generating XSL transformations.
A real-life XML developer’s project is rarely as small and straightforward as the example we used. In this post we will take a look at some typical complications developers face every day and how the Altova MissionKit cuts through complexity to enhance productivity.
Longer XML Schemas and Larger Mappings
Your XML Schemas are not likely to be as short and simple as the ones in our earlier post, and you will likely want to save your work while you are mapping your XML Schema migration. DiffDog lets you save your mapping in an XML Schema comparison file that you can reload later to continue your work, or to share with a colleague.

If you need to compare two XML Schemas on a regular basis, the XML Schema Comparison Document can be a valuable time saver.
Embedded XML Schema Assignment
The screen shot below shows the XML data file from our original example with one important difference. In this version the story element includes an embedded reference to the XML Schema on line 2.

When we transform this file with the simple XSLT we created in DiffDog, the new XML data file will not include the updated XML Schema reference. In our earlier blog post embedded XML Schema references weren’t an issue because we used the XMLSpy Project / Properties menu option to assign default XML Schemas for each folder.
If there are many XML data files to transform, and they require embedded XML Schema references, we can take advantage of an additional feature of MapForce. We can export our mapping from DiffDog to MapForce, as we did in the earlier post, and then use the Component Settings dialog for the MapForce output component to include the XML Schema reference.

Document Your Work
Developers can lose productivity trying to retrace history when a project needs additional work months or even years after an early iteration. The MapForce Generate Documentation feature can help us avoid this frustrating experience.
MapForce will document the mapping of each element in the XML Schemas in Microsoft Word, RTF, or HTML formats. Regardless which format we choose, the resulting document is an excellent stand-alone project artifact, or it can be further edited and included in a larger report.

We can even combine the MapForce mapping documentation with full descriptions of each version of the XML Schema generated with the XMLSpy XML Schema Editor documentation feature.
Source/Version Control
Source/version control systems let teams of developers work closely together on the same project without a risk of overwriting each others’ changes. Because a version of the source code is saved at each stage of the design process, it is very easy to look at or revert to an earlier version when needed.
Altova has implemented the Microsoft Source Code Control Interface (MSSCCI) v1.1 – v1.3 in XMLSpy and tested support for many popular source control systems, so we can manage the files in our XML Schema evolution project across the development enterprise. Additionally, DiffDog can be integrated with source control systems as the default comparison tool. DiffDog can even generate differences report files in a variety of formats.
See for yourself how the tools in the Altova MissionKit can cut through the complexity of your own XML, Web Services, data integration, XML publishing, XBRL, and UML modeling development projects – download a free 30-day trial!

Tags: Altova XMLSpy, diff merge tool, DiffDog, MapForce, MissionKit, v2010, XML Schema, XMLSpy, XSLT