Easier Parallel Programming on Google App Engine

Google App Engine Java supports running parallel operations using its Task Queue API but it doesn't provide any reasonably simple way to create dependencies between tasks. For example running operations in fork-join manner (see explanation bellow if you aren't familiar with the concepts) is also very challenging. Our new framework AppSatori Pipes is trying to fill this gap.

Let's use the trivial yet illustrative example of work to be done: You are a farmer and you want to find the needle in the haystacks.

This is what you usually do if you are alone. You just start with the haystack one and continue to number two and if you don't succeed you just need to search in the third haystack to find the needle.

You application usually also looks like this. You just execute one task after another unless the work is done. 

Of course if there are more farmers each one can take care of particular number of different haystacks but since there is no easy way to let tasks depend on others so they just run independently. Everyone cares about his haystacks. If there is only one needle in multiple haystacks and one farmer will find it, there is no way how to tell it to the others. So they will be still trying to search the needle in empty haystacks.

This also describes what you can achieve on Google App Engine using the standard Task Queue API. You can run independent tasks but there is no easy way how to tell which tasks should be cancelled because they are no longer needed.

Aside of previous two ways of executing tasks AppSatori Pipes offers you two new options.

The first one is classic Fork-Join model when tasks knows that they should wait for their companions and send the results at once to the next task. In terms of our haystack-needle example this means that each farmer searches his own haystack but finally they will meet to check whether they found all the needles.

If there is only one needle to be found, you can just let the others stop as soon as you find it. We call this behaviour "Sprint" in AppSatori Pipes.

You are eager for some code examples? Here they are:

As you can see, there is no plubing, just a few pipes. You can find out more informations on the AppSatori Pipes GitHub site. If you are just looking for Google App Engine solution for you business feel free to contact us at development@appsatori.eu.

The Easiest Way to Share Your Google Apps Contacts

There are many tools that promise you to share your Google Apps contacts but they are not so user friendly. We are proud to announce our new tool ContactSatori which aims to fill this gap. The basic principle is easy. There is one contact group to add contacts to be shared and one to add contacts which you want to share no longer. Everything else is just a magic. In a few minutes, chosen contacts just appears in your colleagues’ contact books.

In case you have always wanted to be a magician (aka administrator) we could reveal you the trick. There is a simple control panel where you can set up the application (see picture bellow).

ContactSatori is currently in private beta. Please, send mail to contactsatori@appsatori.eu if you are interested in. Don’t forget to include the Google Apps domain and expected number of users and shared contacts.

The final price isn’t available yet. It will be determined as soon as the testing period is over.

Using Gradle with CloudBees' Maven Repository

CloudBees is one of the best current Platform as a Service providers. And it is one of the few (maybe only one) who provides tools such as Jenkins as a service too. Its Dev@cloud also provides 2GB free Maven repository which could be a perfect solutions for a lot of home-brewed libraries in your company. This article will show you how to use this free repository in Gradle builds.


Uploading to the repository
There are a lot of reasons why you should want to upload your library to the repository.  The most obvious one is managing the dependencies more easily. If you build your libraries with Gradle you can set up uploading to the CloudBees' Maven repository by adding a few more configuration to your build script.

Gradle to CloudBees

There is no magic about this configuration. You just have to use proper wagon library to support webdav protocol and use the right protocol in the URLs. If you write them manually you should check twice that you use "dav" not "webdav" as the protocol specification.
To upload your artifacts to the repository you just run gradle uploadArchives task.

Using the repository
Using the repository is little bit trickier because to make it work you must use the right server realm.
Using CloudBees repository in Gradle script

The only important part is setting the credentials. The resolver needs the same realm as the server which is "accountname repository".

Groovy 1.8 DSL: Google Spreadsheet Query Language

The development in AppSatori is generally all about integrating applications with some Google API. One of your current project is about exporting Google Spreadsheet contents as XML. The XML part is easy thanks to Groovy MarkupBuilder. Fetching the data from Google Spreadsheet is much more cumbersome because standard API is a little bit verbose. So the decision was made to wrap the API with SQL-like declarative domain specific language (DSL).
Creating DSL in Groovy is very easy since 1.8 release where GEP 3 - Command Expression based DSL was introduced. No more commas as method accessors are needed if you alternate method calls with method parameters. Following code snippet shows the DSL in action the difference between Groovy 1.7 and 1.8 release.
GSQLExample

First thing you must think about when creating DSL is the grammar. Our desired Google Spreadsheet Query Language wraps Google's ListQuery class and could be defined by following grammar:

select [all|'<column name>'*|<position>*]
from 'spreadsheet' [ sheet 'worksheet']
[where 'spreadsheet query']
[contains 'fulltext']
[order by column [<position>|'<column name>]]
[sort [asc|desc]]
[limit <limit>]
[offeset <offset>]

The second step is to write tests for all supported use cases. Here is one example:
Order by column

The next step is to create the entry point to start building the query. In the case of GSQL it is the select method of GSQL class which is defined statically in GSQL class and star-imported to any class or script which wants to use the DSL. Also some keywords like all or desc are defined as static properties of the GSQL class for the same reasons.

GSQL

As you could already mention from previous snippets the query is just ordinary POGO which collects the query parameters. In fact there are three of them to force the order of the commands. The only DSL magic is to add keyword(value) method to the query builder class. You should notice that these methods also constraints the expected value by type. The querying itself happens in the build method of the QueryBuilder class.
SelectBuilder

QueryBuilder

OrderBuilder

The GSQL is still work-in-progress. In future it might support also some data manipulation language for executing updated etc. There are also few things which must be done unless the business users could use the language safely. The most important one is to create secured shell which will prevent writing malicious code in the queries.

Easy switching between Groovy Eclipse 1.7 and 1.8 compilers

Groovy 1.8 compiler support for Groovy Eclipse is out for a while but many people waited for final Groovy 1.8 release to start using it. Using a new compiler is tricky because you need to restart Eclipse each time you want to switch from 1.7 to 1.8 compiler version but the switching button doesn't work as expected. This article will guide you how to set up Eclipse to work easily with Groovy 1.7 and 1.8 project in one single installation.

Switching dialog in Eclipse Preferences

The first thing which came to my mind was to create two separate workspaces - one for Groovy 1.7 projects and the other for 1.8 ones. I hoped the settings are per workspace and this will be enough but the compiler setting is set globally for whole Eclipse installation. Nevertheless, using separate workspaces gives you a good overview, which version projects use and in my opinion you should not mix 1.7 and 1.8 projects unless you want to spend a day confirming "The source cannot be build because some-lib is not compatible with Groovy 1.X version" message dialog. Spock framework libs could be especially annoying because they have two separate versions for 1.7 and 1.8 which are not compatible with the other one. The other way is to keep project in one workspace and always close the projects causing the troubles which is quite cumbersome.

As for now we should have two workspaces - one for each bunch of projects for particular Groovy version. But this doesn't fix our problem. What we need are two configuration folders. The best way to create them is to launch your Eclipse, switch to the 1.8 compiler and close it. Then go to your Eclipse installation folder and duplicate whole configuration folder to the e.g. configuration18. To switch the original configuration back to 1.7 compiler follow these steps from the switching guide

1. Go to eclipse/configuration/org.eclipse.equinox.simpleconfigurator
2. Make a backup copy of bundles.info
3. Open bundles.info in a text editor
4. Find the line for org.codehaus.groovy_1.8 and delete (thus leaving the 1.7 compiler enabled)

Do this for the configuration folder only and leave the configuration18 as it is. The last mile is to launch the Eclipse with given configuration using the command line arguments

eclipse -configuration configuration18 -data workspace18

Specifying the workspace folder is optional. You will probably want to wrap up it as a BAT file or a shell alias/script. This launcher will use the 1.8 compiler. If you want to run 1.7 one you just use the standard one.

Writing Groovy DSL descriptors (DSLD) for Eclipse

Andrew Eisenberg  recently announced new great feature in Groovy Eclipse - the new DSL descriptors which brings you better code completion based on domain specific languages into your Groovy scripts and classes. If you are library developer or just enthusiastic user who wants better Eclipse coding go ahead and create your own DSL descriptor. It's surprisingly easy. In this article I want to share my experience from creating DSLD file for the Gaelyk.

Preparation
Before you start, be sure you have all items on the following check list

• Latest Groovy Eclipse plugin, idealy inside SpringSource Tool Suite
• Latest version of DSLD files descriptor

You should also explore the sources at the end of the article. DSL Descriptors for Groovy Eclipse is actually only source describing the DSLD and DSLD Examples could give you good overview what is possible to achieve with DSLDs.

When everything in place, you can open your Eclipse, create new Groovy project and add libraries you want the DSLD create for to the classpath (for testing and documentation purposes). Then save the DSLD files descriptor into the src folder and also create your new DSLD file in the src folder too.

Creating the DSLD file
The DSL descriptor is all about finding the right places using the pointcuts and contributing right method or properties to the scope found. If you successfully download the DSLD files descriptor you should have a code completion on place and can start coding.

If you are author of the framework or library it is easy to find out which contribution you want to create. If you aren't you should find the origins of contributions you want to cover in your DSLD not to forget anything. In the case of Gaelyk there are a few places where this happens: variables for views and templates are added using GaelykBindingEnhancer, routes DSL is defined in RoutesBaseScript and finally App Engine enhancements comes from GaelykCategory.

The DSLD official documentation is little bit messy so I've created following tables of all available pointcuts and contributions.

Pointcuts and Contributions


As soon as you know what do you want contribute and how should you do it the rest is all about copy pasting and rewriting (unless your DSL comes from a category so you can use the Category Explorer for scaffolding).

Shortcuts for Groovlets
Groovlets are scripts which resides in war/WEB-INF/groovy so we could pick them using sourceFolderOfCurrentType('war/WEB-INF/groovy') pointcut in combination with enclosingScript().  Everything else is just about declaring the right methods and properties.


App Engine Shortcuts
App Engine shortcuts are all defined inside GaelykCategory class. They apply on particular App Engine types (currentType(type)) inside Groovlets which are scripts residing in war/WEB-INF/groovy as written above. We define the pointcut once and then we can use it for each supported type. Since GaelykCategory  is category the Category Explorer was used for scaffolding.


Image Manipulation Language
The definition of image manipulation language is the most advanced one. We are looking for closure passed into the method call named transform on com.google.appengine.api.images.Image type. Everything else is just the same boring copy pasting again.


Tips, tricks and pitfalls 
Here are some ideas which I learnt when I created the Gaelyk DSDL.

• The DSLD file won't compile and show in DSLD preferences screen if you import any non-standard class even it is on the project's classpath so you'd better prefer to use fully qualified strings to specify the types e.g. javax.servlet.http.HttpServletRequest.
• Some of the pointcuts could be tricky, e.g. name of the script when name() pointcut is used is always without the suffix. To get name with the suffix included use fileName() one. You can also get only the suffix by using fileSuffix() pointcut.
• I cannot find the easy way to debug the script but you can run your STS from console and watch the log. You can insert console prints in your DSLD files to explore the states of the variables.
• Divide a editor viewport on half vertically and show a 'test' file/script/class bellow your DSLD file. If something goes wrong you get visual feedback immediately.
• Try to be DRY. Combine pointcuts to the closures and reuse them.
• Find the source of contributions and try to reuse them. For example you can scaffold the DSLD from Category class using the simple Category Explorer
• Think twice when arguments are type of Map. Aren't they just named args? So you should use useNamedArgs = true and write all the desired named args in params (see e.g. add method on queues)
• DSLDs are greate but there are still some features missing. For example I couldn't find how to specify what parameters are expected to be passed into the closure or how to specify type parameters for parametrized types e.g. maps.

 Splitted viewport for better visual feedback

Summary 

DSL descriptors takes coding Groovy in Eclipse one level higher. Where used to be underlined methods and properties you now get full code completion. Even more advanced DSL could be supported, just like Gaelyk's image manipulation language. So don't wait and write DSL descriptor for your favourite library! The Groovy Eclipse users will love you for it!

Sources

• Better DSL support in Groovy Eclipse
• DSL Descriptors for Groovy Eclipse
• DSLD Examples
• Gaelyk's DSLD
• Category Explorer

Written by Vladimir Oraný (@musketyr)

Don't miss any tweet with Kiwittr

Have you ever missed an important tweet just because you were on holiday with no roaming or you were just to busy to read new tweets? This will no longer happen. Our new product Kiwittr will help you to deal with this problem. The Kiwittr adds new event to your calendar whenever new tweet you are interested in occurs. If you have SMS notification enabled the message will be also sent to your cell phone.

It is very easy to start using the Kiwittr. First of all you need to sign in using your Google Account and accept the ability to use your Google Account to sign in. Then the profile page welcomes you with its stepwise wizard. You need to authorize access to your Google Calendar so the Kiwittr will be able to add new events into it and also you need to authorize your Twitter account so the Kiwittr can run search queries using your account (the access is read only so there is guarantee we will not spam anyone using your Twitter, you can use share on Twitter button manually if you like the product).

After everything goes green you can add new Twitter query. You probably wants to add some sophisticated one such as "from:manningdotd java OR groovy" not to miss any deal of the day from Manning publisher but you can as well type a single word such as "myinterest". If you are bored of retrieving notifications you can delete particular query using the "remove" link right next to the query text.

And what happen when a new query is found? If you have SMS notification enabled in your Google Calendar you retrieve ordinal text message such as "Reminder: new tweets for query from:manningdotd java OR groovy". Now you probably hurry to the closest wifi enabled coffee house or a hotel room and read what's actually happen. You don't need to search twitter again to find what's new. The found text is recorded in the description of newly created event.

The Kiwittr is currently in beta testing phase with the limited number of users. So don't wait and try it now.