JSON Archives

Editing, Converting, and Generating JSON

March 12, 2013/in JSON /by Erin Cavanaugh

As the use of JSON as a data transport protocol increases, I thought it would be useful to take a look at JSON support in XMLSpy. There’s been much debate about advantages of JSON vs. XML, but when you boil it down, there are simply some cases for which JSON is the best choice, and others where XML makes more sense. This article on the XML Aficionado blog unpacks this topic quite well.
While you might need to choose between JSON and XML depending on the development task at hand, you don’t have to choose between code editors – XMLSpy supports both technologies and will even convert between the two. Let’s take a look at how that works.
Editing JSON
To make JSON editing as easy as possible, Altova extended its intelligent XML editing features to the JSON editor. If you choose to edit JSON in text view, XMLSpy helps you along with syntax coloring, bracket matching, source folding, entry helper windows and menus, and so on.

Here’s that same JSON file in grid view, which offers a graphical representation of the JSON structure with drag and drop editing. Both views provide JSON syntax checking and advanced error checking features.

Another common requirement is converting XML to/from JSON, which is a one-click option on the XMLSpy convert menu. The JSON data above has been converted to valid XML:

Having JSON editing and conversion functionality directly inside the XML editor is quite useful, especially if you’re using the free Eclipse or Visual Studio integration package.
Check it out and let us know what you think.

Tags: JSON, XML JSON, XMLSpy

Digging deeper with the Twitter API: iPhone 4S vs. Galaxy Nexus

November 8, 2011/in Applications, JSON /by Erin Cavanaugh

We found some interesting data when we dug below the surface of the iPhone 4S vs. Galaxy Nexus debate using the Twitter Search API.In today’s world there is a vast quantity of data available online that can be used for research, market analysis, and competitive intelligence. While “Big Data” can be a problem for those who produce it, store it, and compile it, it is highly beneficial for those of us who are looking for answers.Some of that data is fortunately available to be queried online, and, in particular, there is a vast quantity of data on social media interactions out there.In this article we will explore how to use the Twitter Search API from MapForce, Altova’s data mapping/conversion/integration tool, to aggregate data on recent user submissions (“tweets”) on two highly popular topics – the Apple “iPhone 4S” vs. the “Galaxy Nexus” as the latest hot Android phone – and extract some statistical data about the users engaged in those discussions. One of the benefits of this abundance of data available to us today is that we can query it in interesting ways and extract new meaning from it. While there are undoubtedly many existing services that already provide trends over Twitter topics (e.g., Trendistic), those services only offer very simple trends and do not allow us to query any deeper.But all of the underlying data is available for grabs if you are just willing to learn a tiny bit about web service APIs and how to use them to extract XML data for further processing. As a starting point, let’s use the Twitter Search API to query the stream of recent tweets for the last 100 postings that are about the “Galaxy Nexus”. The Usage Guidelines for Twitter Search tell us that using both words in a query will result in the use of the default operator, which is AND, so we are going to search for posts that contain “Galaxy AND Nexus”. So let’s try that and request the most recent 100 items:

http://search.twitter.com/search.atom?q=galaxy+nexus&rpp=100

If you follow this link, you will get a second window with a lot of raw XML data that is formatted according to the Atom Syndication Format specifications. Alternatively, you could request the data in JSON format, if you wanted to directly process it via JavaScript code by hand, but we will use the XML-based Atom format so that we can easily analyze the data and extract the information we want.Viewing the above search result in a browser is not very user-friendly, so we can take a quick peek at the XML data in our favorite XML Editor using the Open from URL function:As you can see, the data for each entry includes a language code, so for this example we will extract data from this Twitter feed as well as from a second search result on the “iPhone 4S” and combine them into one intermediate XML file for further analysis.Extracting XML data is really easy in MapForce: using the “Insert XML File” option to drop in an XML source, we can again specify the same URL as before. If needed, MapForce will automatically create an XML Schema for the supplied data so we can visualize it and extract information from it:In our mapping we have dropped in two sources on the left side – one using a query string to search for “Galaxy Nexus” and the other to search for “iPhone 4S” – and on the right side we have dropped in a simple XML Schema that will allow us to aggregate our data and analyze it more conveniently going forward. In this case the mapping between the two sides is straight-forward as we are only extracting basic information about the user, the date, and the language of the tweet, but in other applications the mapping could be more complicated and include functions as well as queries to other data sources, databases, or web services…Previewing the resulting XML data can be done directly inside MapForce using the output tab, and this is what we see as a result of our data transformation:Now we can easily use the reporting capabilities of StyleVision to group this data by language within each topic and count the number of posts in each language. We can then report this data in the form of pie charts, which produces the following interesting results:Obviously, this data is highly dependent on the date of execution and time of day, as well as the particular announcements happening about these products, so the numbers will fluctuate quite a bit, but it can be used as a nice monitoring for seeing different language-specific trends. And once this has been set up, the report can be refreshed easily with the click of a button to get a snapshot at that point in time. For more long-term analysis it would of course be necessary to modify the mapping a bit to query more than 100 recent tweets.In this article we have used Twitter’s Search API as one example data source and only looked at language as one unique data point, but there are many more interesting sources of data available online today, and this approach can be used on all of them in a similar fashion.If you want to experiment with other data sources and other kinds of information that you want to extract, we invite you to try for yourself. A free 30-day evaluation version of MapForce is available, and there are no limits on how you can use the other features of Altova’s data mapping and conversion tool for data processing tasks that go beyond analyzing social media trends…

Tags: API, data integration, MapForce, Twitter, XMLSpy

Processing the Groupon API with Altova MapForce

August 2, 2011/2 Comments/in Data Integration, JSON /by David McGahey

We often think of a data integration project as a translation from one singular data input file to some other data set, but Altova MapForce lets you greatly expand the concept of an input file. For instance, the MergeMultipleFiles.mfd example installed with MapForce illustrates how you can use a filename with wildcard characters to merge multiple input files into a single output. A MapForce mapping input doesn’t even need to be a physical file – it can be a URL that returns predictable structured data, like the APIs for popular Web sites like Groupon and many others.

In this blog post we’ll describe how to use Altova tools to retrieve, filter, analyze, and present data available from a Web-based API, using Groupon as an example. If you want to follow along yourself, you will first need to visit http://www.groupon.com/pages/api to request your personal Groupon API client key. The Problem: All Deals Are Local The Groupon Web site and email subscriptions are great for finding deals in your local neighborhood, but what if you’re looking for a deal to use on an upcoming vacation, or for a gift for friends or family across the country? Sure, you could enter each location manually at the Groupon Web page, but that’s so last century. Let’s use the Altova MissionKit to automate things. The Groupon API offers two URL queries that return data in .json or .xml formats: the first returns a list of all Groupon localities (called divisions), and the second returns current deals information for one named division. If we want to see all the deals for more than one division, we need to resolve multiple URLs and aggregate the data into a single result. Yes, MapForce can do that! First We Need a Schema The Groupon API documentation describes the elements that will be returned by our requests, but doesn’t provide an XML Schema. That’s okay, we can use MapForce to generate one. All we have to do is open a new mapping design and choose Insert XML Schema/File, then click the Switch to URL button. Now we can enter the URL to retrieve the Groupon divisions list: When we click the Open button MapForce offers to generate the schema: When we click Yes, the File / Save dialog opens. I saved the schema as divisions.xsd, and the mapping with the new XML Schema inserted looks like this: And the Properties dialog for the XML Schema component automatically contains the API /divisions URL as the Input XML File: Check the Work We want to filter the Groupon divisions data to build a list of id names to use for deal queries for each locality. But before we go any further, now might be a good time to apply the text file trick from the Quick Solution for Complicated Functions blog post to look at the id values. When we insert the text file and connect the divisions and id schema elements, the mapping looks like this: We connected the division element to Rows in the text file in order to generate a new row in the text file for each unique division, so that Field1 in each row will hold the id. Clicking the Output button now generates this result: All we need to do is apply the concat string function to build the list of /deal URLs for all division IDs. The next step in the mapping looks like this: Rolling the cursor over the constant connected to value1 of the concat function displays its full definition: When we click the Output button to execute the mapping, the Output file now looks like this: As a further review, we can open the generated XML Schema in XMLSpy and display it in graphical Schema View: So far we have:

built a MapForce mapping that queries the Groupon API for all divisions
extracted the division id fields
and built a list of URLs for API queries to get the deals in each division

In the next post in this series we will process the list of deal queries as the input for a new mapping component and filter the output for some interesting information. Find out for yourself how easy it is to apply MapForce to convert data from a Web API! Download a free 30-day trial of MapForce.
Editor’s Note: Our original series on mapping data from the Groupon API ran in three parts you can see by clicking the links here: Part 1 of Processing the Groupon API with Altova MapForce describes how to create dynamic input by collecting data from multiple URLs. Processing the Groupon API with MapForce – Part 2 describes how we filtered data from the API and defined the output to extract only the most interesting details. Processing the Groupon API – Part 3 describes formatting the output as a single HTML document optimized for desktop and mobile devices, and reviews ways to automate repeat execution.

Tags: Altova XMLSpy, data integration, data mapping, MapForce, MissionKit, Web services, XML Schema

Altova Named to SD Times 100

June 15, 2010/in JSON /by Niki Devgood

the award for “Tools & Frameworks” goes to Altova!

We are honored to be named to the SD Times 100 for the sixth time! The award, now in its 7th year, recognizes the top innovators and leaders in multiple software development industry areas. Each year the editors of SD Times scour over thousands of nominations in search of companies, non-commercial organizations, open source projects, and other initiatives that exemplify innovation and leadership. When choosing the SD Times 100, the editors consider each company’s offerings and reputation. They listen to the “buzz” — how much attention a company, its products and technologies are creating — as a sign of leadership within the industry. Did the company set the industry agenda? Did a company’s products and services advance the software development art? Were programmers anxiously awaiting its developments? For the judges of the SD Times 100, these are qualities that mark a leader. So what makes us a SD Times 100 winner? In Version 2009, our MissionKit suite of XML, database, and UML tools added key support for working with industry standards like Extensible Business Reporting Language (XBRL) and Health Level 7 (HL7) to help companies adopt these standards while reducing costs and development time. We also rolled out over 70 new, customer-requested features in Version 2010 as part of our “Most Wanted” release, including support for WSDL 2.0, JavaScript Object Notation (JSON), and SysML, as well as compatibility with Windows 7 and 64-bit operating systems, and much more. Download a free, 30-day trial of the MissionKit (v2010r3) and see what’s new in the latest version (v2010r3) of our awarding winning suite of XML, database, and UML tools.

Tags: Altova, Award, MissionKit, SD Times 100

Visit Altova at DevConnections

November 6, 2009/in Events, JSON /by Cindy Neely

The Altova team is gearing up for Microsoft DevConnections 2009 next week! Beginning November 10, you can find us in booth #329 at the Mandalay Bay Convention Center in Las Vegas. Want to know more about our recently released "Most Wanted" Software Version 2010? Stop by the booth for a demo of all the latest product updates. We’ll be happy to show you support for additional technologies, such as WSDL 2.0 support in XMLSpy and MapForce, JSON editing and conversion in XMLSpy, and SysML support in UModel. The new version also delivers enhanced support for technologies like XBRL, offers a completely new design paradigm that gives StyleVision users a new option for creating stylesheets and electronic forms, and much more. Viva Las Vegas! Oh, and if you don’t happen to be attending DevConnections, you can catch us the following week at PDC in LA.

Tags: Altova events, Connections, DevConnections, v2010

XMLSpy’s Most Wanted

November 4, 2009/in JSON /by Erin Cavanaugh

Altova's Most Wanted When we announced that Altova Software Version 2010 included over 70 of the features most requested by our customers, we weren’t talking about little tweaks and enhancements, but major new functionality! To show you we mean business, I’ll outline some of the most requested features added to XMLSpy 2010 here.

WSDL 2.0 Support

In response to requests from from Web services developers, the graphical WSDL editor in XMLSpy 2010 now supports the latest version of the WSDL standard, WSDL 2.0. This adds to existing support for WSDL 1.1, giving you the choice of which version of the standard to work with. The WSDL editor automatically provides the correct editing environment for the version currently being utilized, and XMLSpy even provides one-step conversion capabilities for migration between WSDL 1.1 and 2.0. Since the XMLSpy WSDL editor uses a graphical interface (you can, of course, also work in Text View if you wish), you can easily visualize the structure of your WSDL document and edit it using drag-and-drop functionality and context-sensitive entry helpers, which offer the relevant choices based on the selected WSDL version.

WSDL 1.1/2.0 editor

Enhanced XBRL Functionality

Since we added support for XBRL validation and XBRL taxonomy editing in XMLSpy 2009, we’ve received excellent feedback from customers, including some feature requests that we were able to address in v2010. The new XBRL documentation generation capabilities of XMLSpy 2010 make it easy to generate comprehensive documentation – in RTF, MS Word, or HTML – for your XBRL taxonomies. Multiple options let you choose exactly what to include in the documentation, and the resulting output (snippet shown below) includes hyperlinked components for easy navigation. XBRL taxonomy documentation

Another option for documentation is to print the graphical representation of your taxonomy as it is shown in XMLSpy’s graphical XBRL view. XMLSpy 2010 also includes the new XBRL Taxonomy Wizard to give you a head start when creating a taxonomy. Simply enter the company name, ticker, or other identifier for your XBRL taxonomy, and then select the base taxonomy to extend (if any).

XBRL Taxonomy Wizard

XMLSpy creates the required taxonomy files and prompts you to select the entry points of the base taxonomy. Once you click finish, XMLSpy 2010 displays the newly created XBRL taxonomy files in the graphical XBRL Taxonomy Editor, where you can continue editing and refining the taxonomy in a visual manner. The new Find in XBRL and XBRL Sort options in XMLSpy 2010 meet customers’ requests for quick, easy ways to find data in and

navigate through large, complex XBRL taxonomies.

XBRL Taxonomy Editor

JSON Editor

We’ve recently heard from a lot of developers working on Web 2.0 and Web services apps in XMLSpy who also use JSON – so we decided to add a JSON editor in XMLSpy 2010. You can compose JSON strings in Text View or Grid View, and even convert between XML and JSON. In Text View, the JSON editor provides syntax coloring, line numbering, source folding, bookmarking, and more, making it easy to comprehend and navigate your JSON code, and find and edit strings. Intelligent JSON editing populates the Elements entry helper window with a dynamically built list of the elements present in your JSON file, which you can insert with a double-click.

JSON editor text view

Intelligent JSON editing is also available in Grid View, which provides graphical representation that shows the structure / outline of a JSON document through a set of nested containers. These can be easily expanded and collapsed to get a clear picture of the document’s tree structure, and drag-and-drop editing is supported.

JSON Editor Grid / Outline View A final must-have feature for working with JSON is the JSON <=> XML converter in XMLSpy 2010. One click lets you, for example, convert an XML file to JSON for transport with JavaScript, or convert data received in JSON format to valid XML. No more JSON vs. XML arguments – XMLSpy gives you the best of both worlds.

Redesigned Scripting Environment & Forms Editor

XMLSpy includes an integrated scripting environment and forms editor that has been redesigned for this latest release. Scripts can be written in JScript or VBScript to access and interact with the XMLSpy API, allowing you to modify and add functionality to your installation of XMLSpy 2010. Improvements and optimizations in Version 2010 include:

Access to most of the .NET framework
New form editor controls
Testing & debugging of macros directly in the scripting editor
Execution of macros directly through XMLSpy menus
Improved entry-helpers & auto-completion in the scripting editor

Read more about the “most wanted” features in XMLSpy and the rest of the Altova MissionKit. Please be sure to let us know your most wanted features, either by commenting here on the blog or entering a feature request.

Tags: JSON, v2010, WSDL, WSDL 2.0, XBRL, XML Editor, XMLSpy

Altova’s Most Wanted Edition – Version 2010 Announced

October 28, 2009/in JSON /by Erin Cavanaugh

Altova's Most Wanted

We are excited to have launched the MOST WANTED edition of the Altova MissionKit today! The MissionKit 2010 represents the most wanted functionality as reported by you! We’ve wrangled over 70 new features into this version of Altova’s XML, database, and UML product line to deliver on your list of demands. Check out the highlights for each product included in the MissionKit 2010 tool suite below; subsequent posts will cover the new features in each product in greater detail.

WSDL 2.0

One of the features most often asked for is support for the latest version of WSDL: WSDL 2.0. In addition to existing WSDL 1.1 editing and validation capabilities, XMLSpy now supports WSDL 2.0 in the graphical WSDL editor. It also allows one-click conversion between WSDL 1.1 and 2.0 files.

WSDL 2.0 support has also been added to version 2010 of the MapForce data mapping tool, allowing you to connect to WSDL 2.0 (or 1.1) Web services and integrate their functionality into mappings of XML, databases, flat file, EDI, Excel 2007, and XBRL data. You can also use MapForce to build new Web services based on WSDL 2.0 definitions. MapForce 2010 delivers a number of other enhancements for data mapping, conversion, and integration.

XBRL Enhancements

Several new features for working with XBRL have been added in version 2010. XMLSpy includes a new XBRL Taxonomy Wizard for getting a head start creating taxonomies in the graphical XBRL Taxonomy Editor. You can also now generate comprehensive documentation for your XBRL taxonomies, or simply print the graphical XBRL representation. Find and sort capabilities let you work more easily with large, complex taxonomies.

XBRL Taxonomy Editor

If you’re not familiar with the extensive support for working with XBRL in the Altova MissionKit, check out the XBRL Solutions Center page.

JSON Support

With more and more developers working with JSON in the XML, AJAX, and Web services apps they’ve created in XMLSpy, we’ve received a lot of inquires about JSON support. Now you can edit your JSON files directly in XMLSpy, using Text View or the graphical Grid / Tree View with intelligent JSON entry helpers. XMLSpy even supports one-click JSON <=> XML conversion. Check out the rest of the features most desired by XMLSpy users.

New Design Paradigm in StyleVision

The StyleVision graphical stylesheet design / single source publishing tool now gives you a new, flexible option for designing stylesheets and electronic forms. The new design paradigm lets you create templates within layout containers, and even optionally upload a blueprint image on which to base your design. This way, you can specify your design first and add XML and/or database content after.

Electronic Forms Design

This new form-based design option is made possible by new support for absolute positioning in design layouts. Instead of having content and layout elements flow on the page automatically, like a typical web page layout, you can now specify exact X and Y coordinates for each element to absolutely position it on a page. This feature gives you more control over form design in the manner of desktop publishing applications, but also adds the powerful single source publishing capabilities for XML, XBRL, and database data that have always been present in StyleVision. Read about the numerous other enhancements in StyleVision 2010. Authentic 2010 users will directly benefit from many of the new StyleVision features for true electronic forms design described above. The sophisticated e-Forms created in StyleVision 2010 are presented in Authentic’s WYSIWYG-interface for XML and database content editing by non-technical business users. In addition, with the release of Version 2010, Authentic is now offered in Enterprise and Community Editions, both of which are available as either as a desktop application or browser plug-in. The new Authentic 2010 Enterprise Edition requires a paid license and provides advanced features for WYSIWYG XML and database content editing. Authentic 2010 Browser Plug-in Enterprise Edition is offered on a 12 month license term basis. Authentic 2010 Community Edition is available under a free license and is also offered in desktop and browser plug-in versions.

SysML Support

Altova’s affordable UML modeling tool has become even more robust with support for SysML, which is related to UML but optimized for designing software to operate and control embedded systems and other complex devices. UModel 2010 supports all SysML v1.1 diagram types and elements with all the same productivity-enhancing features available for UML and BPNM modeling, including code generation in Java, C#, and Visual Basic.

SysML modeling tool

Read about all the new features in UModel 2010.

New Diff/Merge Capabilities

Both DatabaseSpy and DiffDog have received some exciting and oft-requested new diff/merge features in version 2010. DatabaseSpy, the highly-affordable, multi-database query, design, and comparison tool now allows you to compare and merge database schemas between databases of the same type or across different database types (all major relational databases are supported). DatabaseSpy displays differences between database schemas in a graphical manner, and you can generate a change script to merge changes in either direction.

Database schema diff/merge

DatabaseSpy 2010 also adds the ability to convert database structures between different database types, for example, to migrate from MySQL® to Oracle® or SQL Server®. Version 2010 of the DiffDog diff/merge tool for files, directories, and databases includes functionality similar to DatabaseSpy for comparing and merging database schemas, and it also provides the most wanted XML Schema diff feature. Since XML Schemas are ever-evolving as requirements change, existing XML files often also need to be updated to remain valid. You can now compare two XML Schemas graphically in DiffDog 2010, and then DiffDog will generate an XSLT file to transform any related instance documents according to the new schema. Alternatively, you can choose to generate an Altova MapForce mapping file based on the DiffDog comparison to further refine the transformation

between schemas.

XML Schema Diff
clip_image012

Collect your Handsome Reward

If you’re an Altova customer with active Support and Maintenance Package, simply download and install version 2010, and your existing key code will activate the new version. New customers can download a free, fully-functional 30-day trial to start working with all these new features today.

Tags: Altova