In this episode of RedMonk Radio, Stephen O'Grady and I talk with SnapLogic's Chris Marino and John Bennett.

Really Simple Integration

We start out with an overview of what SnapLogic is and does. As Chris says, "SnapLogic is an open source data integration framework that allows people to very quickly build data integrations that replaces the hand-coded, point to point integrations they're using today." The idea that SnapLogic bills itself under is "Really Simple Integration," which drives the choice to be a highly web-influenced architecture. Put simply, SnapLogic lets you pull data from various sources, transform them, and then provide them as a simple URL with RESTful features.

We discuss SnapLogic's choice of python as the base language which I found interesting when I first heard about it. Chris comments that the language is keyed in nicely to enabling data integrations and transformations.

Using SnapLogic

I then ask Chris to walk us through a scenario of using SnapLogic to combine two data streams together - to make a mash-up, as it were. Building on the idea that SnapLogic is very web architecture centric, Chris explains how you would define "resources," for example, reading out of a database, a file, or another source of data. If needed, there are transformation resources if needed. The two are combined together into pipe-lines that eventually spit out in the RESTful interface. As I note and then we discuss each of these components are then available to be re-used and re-combined if that makes sense.

Though SnapLogic Chris and John don't cover it in depth here - this being an audio only podcast - the Flex-based SnapLogic IDE is worth checking out a screencast of SnapLogic in action.

Getting Started with SnapLogic

Stephen then asks how developers can get started with SnapLogic. As Chris notes, SnapLogic is GPLv2'ed so you're free to go download it and start using it. He gives us another scenario for pulling data from a database and then using the SnapLogic tool to assign the results of the query to a URL, something like http://theurl/cote/database_query. Again, the idea being that your end-goal is get a URL that returns the desired data in formats like XML, CSV, plain-text, or other representations.

John adds to this the idea of building up a library of these resources to re-use with different people who're interested in consuming the data. This segways into a discussion of the meta-data features that SnapLogic has. They expose meta-data about services in a web page, describing the data service, listing related services, and then allowing you to cross-launch into the web-based design tool.

Enterprise Features

I then ask about SnapLogic's enterprise-grade features. Ostensibly, you can compare what SnapLogic does to Yahoo! Pipes, but, being targeted at businesses and on-premise use, there are, no doubt, additional management and security options. Chris notes that part of the benefit of using a web-based architecture is that SnapLogic can use the same security and access management as other web-based applications.

After discussing some cross-organizational scenarios they've seen people using SnapLogic for, I ask Chris and John to compare the way SnapLogic does data integration with the traditional methods. We then get into a mini-discussion of FTP, EDI, CSV, and other exciting data formats and data extract scheduling.

SnapLogic and WaveMaker

Pulling back to more modern times, Chris talks about working with WaveMaker at Web 2.0 Expo last week - also, some brief video here. As I'm always painting out the need for RIA layer people to get a back-end - that is content and data - this topic is interesting.

Cloud Computing

Stephen then brings up the topic of cloud computing, asking how developers who're doing cloud work could use SnapLogic. This gets to an interesting discussion about using SnapLogic not only for data integration, but also as a sort of URL-accessible "stateless compute resource": that is, submitting asynchronous or synchronous jobs to a SnapLogic layer for processing.

John adds in that the web-centric nature of SnapLogic makes it a good fit for the cross-platform nature of most cloud architectures: since everything is accessible by a URL, there aren't exactly the same cross-platform problems and re-combing two different parts of a system together is, hopefully, easier to do with URLs than direct connections over traditional APIs.

SnapLogic 2.0

Rounding up the general overview of what SnapLogic does, I ask Chris and John to go over the SnapLogic 2.0 release. John starts by saying they "completely re-engineered the product" with an eye towards performance and delivering on the really simple integration idea, for example, with the meta-data directories. Also, while being in python, they've now added a Java library and are targeted to add more languages. They've also improved the designer tool. Finally, in addition to the community edition, they've introduced different commercial packaging options: developer and enterprise editions. Chris adds that they've added several different representations that data can find itself in as well as new sources of data, such as screen-scraping web pages.

When then talk about how SnapLogic fits in as a sort of data-access middleware layer for ISVs who'd want to bundle SnapLogic into their own product. The idea here being that SnapLogic can sort out the data access issues, allowing the ISV to focus on the use of that data, for example for dashboards, business intelligence, and other ways of looking through and using data.

Since Chris was at Web 2.0 Expo last week, before wrapping up, he gives us a brief overview of his thoughts. He was suprised at the amount of enterprise people there, though he heard some carping that there was perhaps too much enterprise. As I say, it sounds like there was plenty of "2.0" there, but a mix of "Web" and "Enterprise" in front of it.

Disclaimer: SnapLogic is a client and sponsored this podcast.

Download the episode directly or subscribe to the feed in iTunes or other podcatcher.

Coté and Ryan start off with some bumpy introductions and then Dion Almaer, our special guest for the week, introduces himself. Dion and Ryan were at Web 2.0 Expo and gave a combined talk about Google Gears and Adobe AIR (slides).

The conversation starts with Web 2.0 Expo. Dion says that the highlight of the event was Fake Steve Jobs. Dion notes that there haven't been any really standout applications and Ryan concurs saying he thought it was "regular Web 2.0 stuff". Ryan asks Dion what his favorite session is and Dion correctly responds that it was the combined session they did. Dion gives an overview of the session and talking about how he and Ryan covered some of the APIs and where the browser is going. Dion says his goal is to get more people involved in advancing the browser.

Coté asks about whether Google Gears is open source (it is) and then asks about external contributers and Dion says that they have started getting external people in on the project. We get into the topic of open source and Coté says he'd like to see a diagram that makes it very obvious what is open source and what isn't from a variety of companies.

One of the things that comes up is offline access and what that means even when you have an internet connection. Dion gives the example of Buxfer which uses Google Gears to store banking information on the hard drive of the user in the SQLite database and none of it is ever stored on the startups servers. Dion describes Gears as an "open source way to teach browsers to do new tricks". Dion follows up by saying that if HTML5 implemented all the new features and Gears was no longer needed, it would be a win. The group discusses the evolution of HTML 5 and where Gears will eventually fit.

Coté asks Dion "What's the deal with WebKit" because there's been a ton of interest around WebKit. Dion says the biggest thing with WebKit is that the code is cleaner than Gecko (Firefox) and that entices more people to jump into the WebKit community. The clean code also helps people iterate faster and add important features.

We then get into what's coming down the pipe at JavaOne. Dion brings up the point that Java has had a lot of really cool stuff for a long time and if they could bring their pieces together to do things like Mesh and EC2, Sun could have been doing this a while ago.

Dion and Ryan both stopped by the Curl booth and chatted with them so Ryan picks Dion's brain on Curl. Dion noted that the demos were kind of ugly and suggested that if you're competing against Adobe and Microsoft you've got to have good looking demos. We all agree that the technology is great but that the web has a certain way to develop things and it's better to fit with that model. Coté wonders out loud about getting involved in the middleware side of RIAs instead of focusing on the client plugin.

Coté brings up OpenLaszlo and we talk about what they're up to. Dion says he really likes them because they went "meta" and did a combination of Flash and Ajax. We also talk about Intuit's QuickBase and get into the development platforms as a service.

Disclaimer: see the RedMonk client list for a clients mentioned.

See detailed show notes.

Included in the feed is the audio of the video: