Thursday, February 16, 2012

Easier Parallel Programming on Google App Engine

Google App Engine Java supports running parallel operations using its Task Queue API but it doesn't provide any reasonably simple way to create dependencies between tasks. For example running operations in fork-join manner (see explanation bellow if you aren't familiar with the concepts) is also very challenging. Our new framework AppSatori Pipes is trying to fill this gap.

Let's use the trivial yet illustrative example of work to be done: You are a farmer and you want to find the needle in the haystacks.

This is what you usually do if you are alone. You just start with the haystack one and continue to number two and if you don't succeed you just need to search in the third haystack to find the needle.
You application usually also looks like this. You just execute one task after another unless the work is done. 

Of course if there are more farmers each one can take care of particular number of different haystacks but since there is no easy way to let tasks depend on others so they just run independently. Everyone cares about his haystacks. If there is only one needle in multiple haystacks and one farmer will find it, there is no way how to tell it to the others. So they will be still trying to search the needle in empty haystacks.
This also describes what you can achieve on Google App Engine using the standard Task Queue API. You can run independent tasks but there is no easy way how to tell which tasks should be cancelled because they are no longer needed.

Aside of previous two ways of executing tasks AppSatori Pipes offers you two new options.

The first one is classic Fork-Join model when tasks knows that they should wait for their companions and send the results at once to the next task. In terms of our haystack-needle example this means that each farmer searches his own haystack but finally they will meet to check whether they found all the needles.

If there is only one needle to be found, you can just let the others stop as soon as you find it. We call this behaviour "Sprint" in AppSatori Pipes.
You are eager for some code examples? Here they are:

As you can see, there is no plubing, just a few pipes. You can find out more informations on the AppSatori Pipes GitHub site. If you are just looking for Google App Engine solution for you business feel free to contact us at


  1. Nevertheless It can non been that event by we as anyone may acquire entirely without charge performing tcgo rules aside applying their network pokemon tcg redeem codes

  2. Advertising is your main tool to have the business ready to go. Actually, everybody is busy in their company marketing over the World Wide Web. Unfortunately, online company is scammed with scammers. On occasion the provider hires you freelance, or occasionally it is a site designer which orders the copy google At an identical time, if you decide on the most suitable company having excellent reviews and reputation, you will enjoy well composed works.

  3. Hah, it's nice) Simple and easy to understand. i liked how you used analogy to explain hard things. I love using this method too. Las time I included it to my review. If you're interested, you can read it too. I think, you'll like it)

  4. Interesting way of explaining, really, I am impressed. You should totally check out rating - creativity is their good side as well. Thanks for posting.

  5. Hello. Thanks for the simple explanation of the complex material. I'm sure that students will like it and they will be able to write for me an essay or another paper.

  6. Probably, of all the preferences, this casino is my favorite. superb slot machine tips I like everything in the casino, the roulette is excellent, I am glad that during the game you can also talk to people who also play at the same table! Muzychka great. With the conclusion there were no difficulties at all. Easily raised from $ 30 to $ 600, withdraw. Played, played, will play!

  7. Order papers for 3 years already - and am quite happy. At the beginning, I was afraid my professors will notice I didn’t write it. But over those years, Royalessays service writers proven to write so well and papers are always free from plagiarism.

  8. If you visit writign services, you can hire the professional writers who can master an impressive custom essay for your academic needs.

  9. When students get a letter or any other paper to write, they frequently come across another way out — to download a sample letter for free from the Internet. This solution cannot be called a good one either since there is no guarantee that the sample has been written by a professional writer. It is much better to buy letter writing paper and get a totally unique paper.

  10. Hi there! I found your post as really interesting. By the way, if you look for a writing service then you can find writing help and consultation. Also, there is such information argumentative essay on health care