Processing the Groupon API – Part 3


Concluding the series in this post, we will apply a stylesheet to transform the XML data created from our mapping of the Groupon API into HTML. Here is an example of the XML output from the data mapping we created last time: XML produced by MapForce from the Groupon API

Assign a Stylesheet to Transform XML The Component Settings dialog for the output component of the MapForce mapping allows us to assign a stylesheet created with Altova StyleVision.

Assigning a StyleVision Stylesheet to a MapForce component
Assigning a stylesheet to the data mapping output component integrates the operations of MapForce and StyleVision, and a new series of buttons appears at the bottom of the MapForce mapping window for HTML, RTF, PDF, and Microsoft Word formats. (You must have both MapForce and StyleVision installed on your computer.) MapForce ouput formats available through a stylesheet When you click any of these output format buttons, MapForce executes the data mapping exactly as we saw in the previous post. MapForce seamlessly passes the XML output to StyleVision, where it is transformed to the selected format. MapForce then displays the formatted document in the Output window. Here is the MapForce Output window for HTML, based on the StyleVision Power Stylesheet assigned above: HTML output produced by MapForce from the Groupon API The MapForce Output menu lets you save the XML data mapping output or the HTML document formatted according to the stylesheet. How to Make a Stylesheet We designed a stylesheet for the Groupon API data mapping using Altova StyleVision, based on the XML Schema for the MapForce output component. The intuitive StyleVision interface and powerful data access and manipulation features make it easy to create attractive documents in HTML, RTF, PDF, and Microsoft Word formats from XML files. The screenshot below shows the StyleVision Design View of the Extreme Groupon stylesheet. The blue numbered circles identify the location of each design feature listed following the image. StyleVision stylesheet for the MapForce Groupon API mapping Features of the SPS file

  1. User-defined html specifies the viewport meta tag for mobile devices. This lets us design one stylesheet to generate a single HTML file for computers and mobile devices
  2. Document title with customized font, size, and color
  3. An autocalc element uses the XPath count function to count the number of deals in the XML input document and inserts the total
  4. The date timestamp is placed at the top of the document even though the date element occurs at the end of the XML input data file
  5. Images from the Altova and Groupon Web sites are referenced by hyperlinks, not as inline image data
  6. Horizontal rules set off each individual deal. For HTML documents, the rules automatically fit the browser window width
  7. Customized fonts and sizes assigned to different elements
  8. A two-column table organizes each deal description
  9. URLs in the XML file are dynamically assigned as hyperlinks for embedded images, fixed text, and dynamic data

StyleVision Power Stylesheets can combine multiple .xsd files, existing .css stylesheets, database schemas, XBRL taxonomies, and more to produce richly formatted reports that can even include automatically generated charts in various styles. You can also use StyleVision to define e-forms with data entry fields, drop down menus, radio buttons and other advanced features. Previewing Stylesheet Transformations StyleVision lets you assign a working XML file to preview your output as you design the stylesheet, and the buttons along the bottom of the Design window make it convenient to display the formatted working file as you refine your design. We saved the XML output of the MapForce mapping and assigned it as our working document. When the stylesheet was complete, the HTML Preview in StyleVision was identical to the MapForce HTML Output window shown above. To view the document on a mobile device you can either deploy the HTML as a page on a Web site or email it as an attachment. HTML version of the MapForce mapped data on a mobile device In addition to the stylesheet itself and formatted versions of the working document, StyleVision lets you save generated XSLT files to transform other XML files using your stylesheet design outside the StyleVision application. Automation Next Time In the future when you want to re-run a data mapping and refresh the HTML document with up to date data, there are two ways to automate the process:

  • You can run MapForce from a command line with parameters to name the mapping definition file and even call StyleVision to create formatted output
  • You can generate royalty-free code for the mapping in XSLT, Java, C++, or C# to combine with the XSLT code from StyleVision to build your own end-to-end application

XMLSpy, MapForce, and StyleVision are all available together in the specially priced Altova MissionKit. See for yourself how easy it is to use the MissionKit to convert data from a Web API — download a free 30-day trial!
Editor’s Note: Our original series on mapping data from the Groupon API ran in three parts you can see by clicking the links here: Part 1 of Processing the Groupon API with Altova MapForce describes how to create dynamic input by collecting data from multiple URLs. Processing the Groupon API with MapForce – Part 2 describes how we filtered data from the API and defined the output to extract only the most interesting details. Processing the Groupon API – Part 3 describes formatting the output as a single HTML document optimized for desktop and mobile devices, and reviews ways to automate repeat execution.

Tags: , , , , , , , ,

Processing the Groupon API with MapForce – Part 2


In Part 1 of this series we described how to connect Altova MapForce to the Groupon API. We queried the API for a list of Groupon divisions, then used the list to create API queries for all the current deals from every division. In this part, we will execute the /deals queries and filter the response for the most interesting data. The list of /deals queries we built previously looks like this: List of Groupon /deals queries generated by Altova MapForce To process all the queries, we can connect the list as a dynamic file input to a new mapping component. When we needed a new component last time, we dropped an API /divisions query into the mapping, and let MapForce create an XML Schema automatically. We could do the same thing here by dropping in an API /deals query as an XML input file. There’s just one small issue — although the Groupon API online documentation clearly describes the queries we can make, it is vague about the information that will be returned. Before we send dozens of queries to the API for all the current deals, we probably want to know a little more about the data that will come back.

Let’s Make a Deal Like Yogi Berra said, you can observe a lot just by looking. Let’s start by running a /deals query in XMLSpy. That will let us examine the response to a query for one division before we pull in a potentially unwieldy volume of data. The XMLSpy File / Open menu includes the same Switch to URL option we used in MapForce in the earlier post. If we enter the /deals API query for a division that covers a large metro area – say Dallas – we are likely to get enough deals instances to extrapolate the characteristics of the entire data set. XMLSpy opens the response to the /deals API query in Text view just as if we opened a local file: Example from the response to a Groupon /deals query, shown in XMLSpy As expected, we got quite a bit of data when we requested all the deals for a single division! A fast way to analyze the structure of this data is to use the XMLSpy DTD / Schema menu option to generate an .xsd file from the xml. Shown below is a reduced view of the entire generated .xsd file based on the response to the /deals query for Dallas: An xsd file generated by XMLSpy from the Groupon query We can dig even deeper, following Yogi’s advice like déjà vu all over again. Expanding all the elements to review the XML Schema reveals some curious anomalies. For instance, there are two elements named redemptionLocation with different definitions. The first contains a sequence of child elements: First use of the remdemptionLocation element And the second is defined as a simple string: Second use of the remdemptionLocation element Going back into the xml data for Dallas and searching for redemptionLocation displays these examples: One example of redemptionLocation in the body of the response And: One example of redemptionLocation in the body of the response And: One example of redemptionLocation in the body of the response Now this is really interesting, because redemptionLocation = ”online” identifies deals that can be redeemed from anywhere, instead of by a visit to a bricks and mortar location in the division where they are advertised. What if we ran the /deals API queries for all divisions and extracted a list of all the online deals? That would be one extreme Groupon! Only Ask for What You Need The Groupon /deals API query supports an optional parameter called &show= that allows users to limit the data returned. Applying this parameter can save bandwidth and reduce processing time for the data transformation by removing unwanted data from the API response. We can also simplify our final result by including only the most interesting information, including the link to the Groupon web page for each deal. After we remove unwanted elements from the generated Dallas schema, our final version for the summary of online deals looks like this: XMLSpy Schema diagram of the simplified Groupon xsd file When we add the &show= parameter to our MapForce mapping to request only the elements included in the simplified XML Schema, the queries look like this: Modified list of queries with the &show= parameter Now we can drop the revised .xsd file into the mapping and connect the list of API /deals queries as dynamic input. We don’t need to delete the text file we used to collect the list of queries — that might continue to be helpful for future debugging. Mapforce dynamic input file mapping These changes complete the input side of the data mapping. Defining the Data Transformation Output Back in XMLSpy we can make a couple more revisions to the input XML Schema to design a new version for output: XMLSpy schema diagram of the output file xsd We discarded the response element since it doesn’t add any value, and eliminated the redemptionLocation element that we don’t intend to include in the output. We also added a date element for a timestamp, because our output file will be a snapshot of data that is constantly changing. After saving this version of the .xsd file in XMLSpy, we can drop it into the MapForce mapping. Shown below is the output side of the mapping with the output component partially connected. The filter at the top reads the redemptionLocation element to select only online deals and the now function inserts the date: Partial view of the MapForce output file mapping The last revision we made in the output XML Schema was to change several element types from dateTime, Boolean, and integer to the string data type to allow more descriptive text Here is the complete definition of the mapping with the final connections to the output component: Mapforce data mapping for the Groupon API Now for the Payoff When we click the Output button MapForce processes the entire mapping from beginning to end using the MapForce Built-in execution engine. Here’s a breakdown of the steps:

  • Run the /divisions query to get the current list of divisions
  • Concatenate strings to build the list of /deals queries for all divisions
  • Run the /deals queries to create dynamic data for the input component
  • Filter for online deals to generate the output component, execute the remaining mapping functions, and add the timestamp after all the deals are processed

MapForce takes only a few seconds to complete all those steps and generate an output file with a series of deals that look like this: Output data from the MapForce mapping for the Groupon API In part 3 of this series we’ll design a stylesheet to automatically transform the XML output of our mapping into html for attractive presentation in a web browser and on mobile devices. See ya at the ballpark, Yogi! XMLSpy and MapForce are available together in the specially priced Altova MissionKit. See for yourself how easy it is to use the MissionKit to convert data from a Web API — download a free 30-day trial!
Editor’s Note: Our original series on mapping data from the Groupon API ran in three parts you can see by clicking the links here: Part 1 of Processing the Groupon API with Altova MapForce describes how to create dynamic input by collecting data from multiple URLs. Processing the Groupon API with MapForce – Part 2 describes how we filtered data from the API and defined the output to extract only the most interesting details. Processing the Groupon API – Part 3 describes formatting the output as a single HTML document optimized for desktop and mobile devices, and reviews ways to automate repeat execution.

Tags: , , , , , , ,

The PXF File as the Software Equivalent of Plug and Play – A Database Editing Use Case


A few weeks ago we introduced you to the Portable XML Form (PXF), a file format in which all design elements supporting a StyleVision design including XML Schema and instance documents, SPS design files, XSLT, images, and other external files are embedded. End users simply open the PXF file in Authentic and can immediately start editing XML and database data. clip_image014In this post we’ll create an electronic form for business users to record donations and enter new donors for a fictitious charity called the Green Planet Fund.

Altova’s new PXF file format is a boon to developers and end users alike –the PXF file can be transported, downloaded, copied, and saved like any other office document, increasing both efficiency and error-free deployments. Developers no longer have to send or install multiple files or create a server application to support a design and business users can distribute critical business information quickly and reliably. This is especially true in distributed organizations where business data needs to be sent around via e-mail between different locations or between field employees and their counterparts in the home office. A PXF that supports electronic forms designed in StyleVision and edited by business users in Authentic makes editing business data a snap. In most cases that business data will be contained directly within the PXF file as an embedded XML document, but it is also possible to connect a PXF file to a database server and store the data directly in the database, as the following use case shows.

For this example we’ll be using StyleVision to create an electronic form on which business users will record donations and enter new donors for the fictitious Green Planet Fund. End users interact with these forms in the Authentic WYSIWYG editor, a sophisticated word processor-style interface that allows end users to capture, view, and update XML and database content. Once you click File, New from DB in StyleVision, the Connection Wizard is invoked. Simply select the database type – you will be prompted to browse for the source. clip_image001 Here we’ve identified the DonorsDatabase from the MARKETING006SQLEXPRESS server. The connection string is automatically saved with the design file and re-established each time a business user accesses the form in Authentic. The database is updated with the information that the business user enters in the Authentic form. clip_image002 Once you establish a connection, you can select tables, data views and even SQL SELECT statements in the Insert Database Objects dialog box. clip_image003 For this example we’ve added a SQL SELECT statement (below) that concatenates donor title, first name, and last name and calculates the total amount contributed by each donor. Once you click Add SELECT Statement the SQL SELECT statement becomes available to the StyleVision design, as seen above. clip_image004 Please note that we used Altova DatabaseSpy, a multi-database query, design, and database comparison tool, to build our SQL statement. Although you can export data from the SQL Editor in DatabaseSpy into a range of formats, for this example we simply copied the SELECT statement from DatabaseSpy into the window in the SQL SELECT dialog box in StyleVision. StyleVision generates a temporary XML Schema based on the structure of the database and displays it in the Schema Window. Note that both the Contributions and Donors tables as well as our SQL SELECT statement appear in the Schema Tree. clip_image005 During this transformation StyleVision creates internal XML files – a non-editable one for previews and as the source of the generated XML data file as well as an editable XML file that will write modifications back to the database. Now that we’ve established and saved the database connection string we’re ready to design the electronic form that Green Planet Fund will use to add new donors and log contributions. For this example we’ve created two simple tables in StyleVision – one for recording contributions and another for looking up and adding donors. (Please note that our example is for illustrative purposes only and we’ve shown only a small sample of the fields that we would typically include in a real-world application. The functionality too is far less sophisticated than it would be in an actual project.) We’ve done the following to our design: · Created a design fragment with a table for entering new donors and a table of registered donors that is populated dynamically · Added a checkbox that when toggled will show the registered donors table and the data entry table from the design fragment · Edited the Authentic properties of the registered donors table to retrieve all records · Edited the Authentic properties of the data entry table to retrieve only one record to make it easier to enter data · Used XPath to concatenate each donor’s title, first, and last names for the table of registered donors, using the normalize-space function to remove all white spaces · Used Value Formatting (below) to format the contribution date – this also allows the end user to edit the date via a calendar interface in Authentic clip_image006 Note the design fragment ‘DonorList’ is enclosed within the DB tags and is represented as a single element. The design fragment itself appears at the bottom of the design. clip_image008 Clicking the tabs next to the Design tab will render the report in different formats. Here we’ve presented the Authentic view. clip_image010 Notice that the design is still in standard SPS format (greenplanetdonorsII.sps) – we are now ready to save it as a PXF so that it can be emailed to the client and deployed immediately upon receipt. Once you select File – Save As from the top menu, you can choose whether to save the design as a SPS or PXF. clip_image011 When you Save as PXF File you are prompted to select the files to embed in the PXF file. All files supporting the StyleVision design are automatically made available in the dialog box. clip_image012 Here we’ve checked each of the design elements as well as the XSLT files. This will allow end users to generate output from the StyleVision-designed form in each of these formats directly from Authentic. Please note that you also have the option to include additional files. Although this design does not require other files, this feature makes transporting and deploying projects that require multiple files significantly easier. Now you can send the PXF file that you have just created to the client. All project files as well as the database connection string are embedded in the PXF so that once the end user opens it in Authentic he can immediately start viewing, editing, and entering data. To create a new record in Authentic, place the cursor in one of the fields in the contributions table on the form and click the Append row button on the tool bar. (Alternatively you can select Authentic – Append Row from the top menu.) clip_image013 Now the business user can enter contributions (new data is in bold) … clip_image014 …and new donors. clip_image015 Note that in the SQL Server database the Donor ID is an auto-generated field. Once the end user clicks File – Save from the top menu, the new information is posted to the database, the database generates an ID number, and the new information populates the other tables in the form. (As reflected above, data is populated in real time.) Edits and additions are immediately written to the database. The SQL Server database view below shows that Edward and Julie Jay, whom we have just added to the Authentic form, have been assigned a donor id of 18. clip_image017 Although we did not include it here, the form can be designed to generate an error message when the user attempts to save a new donor without at least one first and last name. This can be accomplished using XPath in the additional validation property of the Authentic properties or by creating a constraint in the SQL Server database. This use case was designed to show how easy it is to create and deploy interactive forms. Electronic forms designed in StyleVision have always been a great way to update both XML and database content – the PXF file simply makes it easier to transport and deploy them.

clip_image018

Have you used the PXF form yet? Or created a really cool database project using StyleVision or any of Altova’s other tools? Please share your story with other Altova users by commenting on this blog post. Think it would make a great case study? Email us at marketing@altova.com – if we use your story you’ll receive a $200 Amazon gift card. We’d love to hear from you!

Processing the Groupon API with Altova MapForce


We often think of a data integration project as a translation from one singular data input file to some other data set, but Altova MapForce lets you greatly expand the concept of an input file. For instance, the MergeMultipleFiles.mfd example installed with MapForce illustrates how you can use a filename with wildcard characters to merge multiple input files into a single output. MapForce MergeMultipleFiles.mfd example A MapForce mapping input doesn’t even need to be a physical file – it can be a URL that returns predictable structured data, like the APIs for popular Web sites like Groupon and many others.

In this blog post we’ll describe how to use Altova tools to retrieve, filter, analyze, and present data available from a Web-based API, using Groupon as an example. If you want to follow along yourself, you will first need to visit http://www.groupon.com/pages/api to request your personal Groupon API client key. The Problem: All Deals Are Local The Groupon Web site and email subscriptions are great for finding deals in your local neighborhood, but what if you’re looking for a deal to use on an upcoming vacation, or for a gift for friends or family across the country? Sure, you could enter each location manually at the Groupon Web page, but that’s so last century. Let’s use the Altova MissionKit to automate things. The Groupon API offers two URL queries that return data in .json or .xml formats: the first returns a list of all Groupon localities (called divisions), and the second returns current deals information for one named division. If we want to see all the deals for more than one division, we need to resolve multiple URLs and aggregate the data into a single result. Yes, MapForce can do that! First We Need a Schema The Groupon API documentation describes the elements that will be returned by our requests, but doesn’t provide an XML Schema. That’s okay, we can use MapForce to generate one. All we have to do is open a new mapping design and choose Insert XML Schema/File, then click the Switch to URL button. Now we can enter the URL to retrieve the Groupon divisions list: Inserting a new component into a MapForce mapping by URL When we click the Open button MapForce offers to generate the schema: MapForce offers to generate an XML Schema When we click Yes, the File / Save dialog opens. I saved the schema as divisions.xsd, and the mapping with the new XML Schema inserted looks like this: Generated .xsd as a new component in a MapForce design And the Properties dialog for the XML Schema component automatically contains the API /divisions URL as the Input XML File: Component properties for the generated .xsd Check the Work We want to filter the Groupon divisions data to build a list of id names to use for deal queries for each locality. But before we go any further, now might be a good time to apply the text file trick from the Quick Solution for Complicated Functions blog post to look at the id values. When we insert the text file and connect the divisions and id schema elements, the mapping looks like this: MapForce design with text file to preview output We connected the division element to Rows in the text file in order to generate a new row in the text file for each unique division, so that Field1 in each row will hold the id. Clicking the Output button now generates this result: MapForce Output window All we need to do is apply the concat string function to build the list of /deal URLs for all division IDs. The next step in the mapping looks like this: Using the MapForce concat function to build a string Rolling the cursor over the constant connected to value1 of the concat function displays its full definition: Definition of a MapForce string constant When we click the Output button to execute the mapping, the Output file now looks like this: MapForce Ouput window As a further review, we can open the generated XML Schema in XMLSpy and display it in graphical Schema View: Altova XMLSpy graphical schema view of the generated .xsd So far we have:

  • built a MapForce mapping that queries the Groupon API for all divisions
  • extracted the division id fields
  • and built a list of URLs for API queries to get the deals in each division

In the next post in this series we will process the list of deal queries as the input for a new mapping component and filter the output for some interesting information. Find out for yourself how easy it is to apply MapForce to convert data from a Web API! Download a free 30-day trial of MapForce.
Editor’s Note: Our original series on mapping data from the Groupon API ran in three parts you can see by clicking the links here: Part 1 of Processing the Groupon API with Altova MapForce describes how to create dynamic input by collecting data from multiple URLs. Processing the Groupon API with MapForce – Part 2 describes how we filtered data from the API and defined the output to extract only the most interesting details. Processing the Groupon API – Part 3 describes formatting the output as a single HTML document optimized for desktop and mobile devices, and reviews ways to automate repeat execution.

Tags: , , , , , ,