= How to get '''your''' Shindig started =
== Preface ==
This page aims to provide a small guide on how to get started on Shindig. For my employer, Hyves, I started working on trying to get Shindig running on our development environment. The first couple of steps were surprisingly easy, although the fact that the task ''seemed'' rather daunting at first, meant that I kept postponing it. I hope this guide will allow other to overcome their "fears", and start playing with Shindig.

I think that this guide (probably with loads of changes) should eventually move to the Shindig website; at the moment there is no Shindig website, and seeing the number of requests for something like this on the [http://groups.google.com/group/opensocial OpenSocial group] and [http://mail-archives.apache.org/mod_mbox/incubator-shindig-dev/ Shindig mailinglist], I decided to use our own Hyves API trac.

It should also be noted that all my experiences are with the version of Shindig I checked out two days ago (revision 616512). Shindig is in heavy development, and things may change quickly.

=== Disclaimer ===
It should be noted that I'm not actively involved in Shindig, and I only started really digging into it two days ago. So some of my statements below may be slightly off target, half-truths, or just plain wrong. I encourage reporting any errors or issues.

=== Who is it for ===
This guide is for everyone who runs a Social Network Site (or has a product that can be used for that), who would like to get his hands dirty on playing with (not even) beta software for implementing !OpenSocial; or, more generally, for everyone with any kind of site, wanting to play with software for implementing (Google) Gadgets. This guide is not aimed at gadget developers, at people looking for information on how to build a social network or at people wanting a finished product that will implement !OpenSocial.

The way the guide is set up, it assumes you are rather confident in !OpenSocial, programming and web concepts. It will not look for the fastest way to get some proof of concepts, rather would lead you to a point where you can say you understand Shindig.

=== Discussion ===
All discussion on this document should be done in either the [http://groups.google.com/group/opensocial OpenSocial group] or [http://mail-archives.apache.org/mod_mbox/incubator-shindig-dev/ Shindig mailinglist]. I will not respond to personal emails, unless there is a very good reason for the email to be private. I will monitor both the group and the list, and respond to any questions I think I have the answer to, and I deem worth an answer.

The worthiness of the question will in general increase with the amount of time I feel people have spent on trying to figure it out for themselves. If you have questions on any of the steps, I expect you to have successfully done all previous steps (not just read over them). This meand that if a step says "play with the settings a little bit", I expect people to do just that :).

=== Questions that this guide will not address ===
The following questions will not be addressed by this guide, nor will I respond to them (and probably many other questions on the meaning of life are left for discussion in some other location). 
 * Why would I want to integrate !OpenSocial?
 * Why would I want to run my own social network site?
 * How can I build my own social network site? (hint: just use Ning)
 * How can I integrate !OpenSocial / Shindig into my specific site running software X on platform Y?
 * What are the release-schedules / expected dates for feature X or product Y?
 * Can I legally use Shindig for X? (hint: Shindig is released under the apache license, so have your lawyers go over that).
 * What are the next steps after I finish this guide? (as soon as I know, I'll put it in the guide)
 * How can I make money off !OpenSocial (either as container or gadget)?

=== Quick FAQ at the beginning ===
==== What does the word ''Shindig'' mean anyway ====
 * From [http://en.wiktionary.org/wiki/shindig wiktionary]: ''Noisy party or festivities''
 * From [http://en.wikipedia.org/wiki/Shindig wikipedia]: ''Shindig refers to any sort of clever party, covered dish gathering, box social, (archaic) a brawl. Also it can refer to a dance party with lots of music''
==== What is the aim of the Shindig project ====
 * From [http://mail-archives.apache.org/mod_mbox/incubator-general/200711.mbox/%3c2A075A49-502E-4F9B-8C7D-D66F80873554@apache.org%3e the original proposal]: Shindig will develop the container and backend server components for hosting !OpenSocial applications;Shindig will develop a !JavaScript container and implementations of the backend APIs and proxy required for hosting !OpenSocial applications.
 * From my own experience: Shindig is the reference implementation for the gadget server. Most likely, most sites that will support !OpenSocial in the near future will do so on the basis of the Shindig server.
==== Whats the scope of Shindig ====
For the best answer, see these emails on the shindig list:
 * http://mail-archives.apache.org/mod_mbox/incubator-shindig-dev/200801.mbox/%3c4795F411.90404@demay-fr.net%3e
 * http://mail-archives.apache.org/mod_mbox/incubator-shindig-dev/200801.mbox/%3c9fec51e20801220718k393670e3hbb0d50d712dc7115@mail.gmail.com%3e
 * http://mail-archives.apache.org/mod_mbox/incubator-shindig-dev/200801.mbox/%3caa285c160801221219h7b643ffbnbcb5ab032e7989ce@mail.gmail.com%3e
Extract: We'll provide very basic, primitive reference data to satisfy the APIs, but any real production implementation will need to hook their own existing site up to the various gadgets and open social interfaces that we have implemented.
==== Do I need to have Shindig running to become an !OpenSocial container ====
No. Google has announced (link needed) that they will (most probably) be hosting a set of Shindig servers for general third-party use. Gmodules.com will eventually move to Shindig as well, however this does not automatically mean that it can be used by third-party containers. The way things look now, there will probably be some advantages (in speed) to be running your own Shindig servers, however things may change here.
==== Which language should I get Shindig in ====
(note: this question is on programming language; since I don't think Shindig has any user-interface, I don't think that i18n is an issue here)

Currently (31 Januray 2008), there is a PHP and an Java version of Shindig. The Java version is further in development. As I recall from discussions on the mailing list, this is because it seems less useful to solve architectural problems simultaneously in two different languages.

In my opinion (although I'm not sure that everyone on the Shindig list agrees with me), there should not be a requirement that your container and the Shindig server are in the same language, or share code, share datasources, share anything but a handful of shared secrets. The only thing that you need in your own programming language is the !GadgetSigner (soon to be renamed !GadgetTokenSigner) and appropriate subclasses. This is not more than a couple of lines of code in most cases. (I think they can even be compiled to standalone Java programs, which are then run from the commandline, so they can be called from any environment).

This guide describes my experience with the Java version of Shindig, and a PHP container.
==== Is Shindig production ready ====
No. There are still some issues. If you're interested, check [https://issues.apache.org/jira/browse/;jsessionid=2DF627B6016E52FB70FF89120330A60DSHINDIG/component/12312112 the JIRA]. There are still some security and functionality issues. Furthermore, obvioulsy, you need to do some custom work to attach the server to your datasources, see [#FinalDatAllTheWay later in this document].  That being said, Shindig is being developed at an incredible pace, and I wouldn't be surprised if it ''is'' production ready by the time you read this.

Also see [http://mail-archives.apache.org/mod_mbox/incubator-shindig-dev/200802.mbox/%3caa285c160801311823o4c024f8fp632895e90380868a@mail.gmail.com%3e Kevin's remarks] on an earlier version of the statement above.

I believe Orkut and Hi5 are running on a (slightly modified version of the) Shindig server, so obviously it is possible to modify Shindig in such a way that it may be used for a public sandbox test. I'm not sure on which platform Ning and (soon to be) Myspace are running their respective sandboxes (if anyone knows, please let me know). Hyves is committed to have their sandbox test run on Shindig.
==== Where did you get all this info ====
I do not have inside sources to either Shindig, Google nor anyone else within !OpenSocial. All the information here was collected using public resources, discussions on public mailing lists and digging through the Shindig source code.
==== What is Hyves ====
[http://www.hyves.net Hyves], the company I'm exploring Shindig for, is a Social Network Site (SNS) in The Netherlands. The company is sited in Amsterdam, and we are by far the largest SNS in The Netherlands, and one of the larger !OpenSocial-container launching-partners.

In our home market, The Netherlands, 28% of the (full) population is a member, resulting in 4.5 million Dutch members (out of 5.5 million members total). We served 3.3 billion pages this January, from a server park that is one of the  largest in The Netherlands.

Hyves is committed to implement !OpenSocial (probably version 0.7, or a later version if available) in Q1 2008.

== General concepts ==
==== Terminology ====
In order to avoid confusion, I'd like to define some terms I'll be using in the guide. If any of these terms conflict with definitions used elsewhere in !OpenSocial or Shindig, please let me know so that I can correct them.
 Container::
   The server-side code that is responsible for the social network site. In Hyves's case, the container is in PHP.
 Container-page::
   The parent-page that hosts the iframes that contain the gadgets. (note that the !OpenSocial spec never defines that gadgets need to be hosted in iframes; perhaps more accurate would be to say: the parent page that hosts the gadgets. In the future, most likely, we'll be able to drop the iframes through Caja security. This is not in the scope of this document, see [http://mail-archives.apache.org/mod_mbox/incubator-shindig-dev/200802.mbox/%3c47d79a260801312027x449ab0f4jf7e1556178b636db@mail.gmail.com%3e Bruno's notes on this]).
 Container site::
   The social network site (the collection of rendered-HTML pages) you want to integrate !OpenSocial in.
 Gadget server::
   The server that translates the gadgets from XML to whatever format they need to be for displaying. It has some extra functionality that is being discussed [#Generaldescription below].
 Shindig server::
   The only (Open Source) implementation of a Gadget Server that I know of, and the one used for this guide.
 Shindig::
   The project that develops the Shindig Server.
 Gadget::
   The clientside code that runs as gadget.
 Gadget frame::
   The iframe the gadget is hosted in. (note that the !OpenSocial spec never defines that gadgets need to be hosted in iframes; for now in Shindig, they are)
 Gadget xml::
   The xml that describes the gadget.
 Gadget xml url::
   The url the gadget xml can be found on.
 Gadget functionality::
   The client-side functionality needed for a gadget (without the functionality for !OpenSocial). This would be all functionality to adhere to the [http://code.google.com/apis/gadgets/docs/spec.html gadget specification].
 Opensocial functionality::
   All functionality that is part of the [http://code.google.com/apis/opensocial/docs/0.7/spec.html OpenSocial specification].
==== General description ====
This description will give you a ''very brief'' overview on how !OpenSocial works. This is not meant as specification, it'll just give you a general feeling. (I should probably just link here to some better description of this, suggestions are welcome)
 1. You have a website that supports !OpenSocial (or just plain gadgets). Somehow (how is outside the specs) a user has added a gadget xml url (http://example.com/gadget.xml) to his page.
 1. When the page renders, the gadget is rendered in an iframe. Through javascript from Shindig (or by any other way you like), an iframe is drawn, with a source of !http://mygadgetserver.com/gadgets/ifr?url=http%3A%2F%2Fexample.com%2Fgadget.xml#st=1234abcdef
 1. The gadget server retrieves the gadget xml, and transforms it to a HTML document (inserting a lot of javascript in the process)
 1. The gadget will run in the iframe, just like a normal HTML document loaded in an iframe
 1. If the gadget wants to do some AJAX call, this is routed through the gadget server (that serves as proxy-server)
 1. If the gadget has !OpenSocial functionality, and wants to retrieve some !OpenSocial information, this call is routed through the gadget server to the container.
Note: again I emphasize that the description above is ''not'' a specification. It is a possible implementation of the specification, useful to help understand the (possible) process.

== Getting started with Shindig ==
I did all my work on the Shindig version of two days ago (revision 616512). I installed it on Gentoo Linux, with Java 1.5.0_10. (Kevin Brown just [http://mail-archives.apache.org/mod_mbox/incubator-shindig-dev/200802.mbox/%3caa285c160801311900s22d5cb18v1d95fdf79c941958@mail.gmail.com%3e wrote a post today] that you should use Java 1.6)  . It might or might not work with another Shindig version, it might or might now work on other OS'es and Java versions, it will most probably not work if you use anything else than the Java version on Shindig.
=== First date with Shindig: just try and see how far you can get ===
 1. Check out the Shindig sourcecode:
{{{
svn co http://svn.apache.org/repos/asf/incubator/shindig/trunk shindig
}}}
 1. Understand the toplevel directories:
   * features: This contains descriptions for features; Features are the things that get included when your gadget xml containers a <Require feature="XXXXXX" /> tag. Note that there are no opensocial-0.x subdirectories in here; how exactly support for opensocial-0.x should look like is still not entirely clear (and will be discussed much later in the guide).
   * java: The Java Shindig server code
   * javascript: Javascript and other static web-files that are not part of the gadget server.
      * container: Contains javascript, css, and html files that you could use on your container-site (and probably should for your first steps), and a lot of examples.
      * gadgets: Not sure what the exact function is, seems duplicated content from the features directory
      * samplecontainer: Not checked it out yet, but is sure to contain Cassie's excelent sample container, that runs !OpenSocial gadgets without any kind of backend (just a local xml file)
   * php: The PHP Shindig server. As I explained before, this server is far behind in development and I wouldn't recommend it at the moment to anyone trying their first steps at Shindig, no matter how much better your understanding of PHP is than Java. (With this I mean no disrespect towards the authors of PHP Shindig, as I [#WhichlanguageshouldIgetShindigin explained before], there is a good reason that the PHP Shindig server is behind in implementation.)
 1. Run your Shindig server. The java/gadgets/README is your friend here, it tells you all you need to do. On gentoo "emerge maven" was enough for maven to be installed. When retrieving the gadget in the final step (from http://localhost:8080/gadgets/ifr?url=http://www.labpixies.com/campaigns/todo/todo.xml), you should receive in your browser/as the result from wget a valid HTML page, starting with <html><head>, then a WHOLE LOT of javascript (about 2000 lines) -- actually I'd think that this javascript will be externally linked sometime in the future. That should be followed by the original gadget code, and finally end the </body></html>. If you see this result, congratulations, you have Shindig installed. (Note that the suggested url is actually not correct, it should be http://localhost:8080/gadgets/ifr?url=http%3A%2F%2Fwww.labpixies.com%2Fcampaigns%2Ftodo%2Ftodo.xml, but the Shindig server forgives us).
=== Second date: digging a little deeper ===
I believe it's worth digging around the implementation of any tool you are considering for inclusion into your infrastructure. I learned a lot by going through the Shindig code, which is extremely readable and well set up (to my opinion). I got a good idea of the main parts of the server, even though I've not done any serious Java for 10 years (and hardly anything I would qualify "serious" before that). There are two extremely useful posts on the Shindig mailinglist that will help you. (see below)

It should be noted that the code you have is nothing be itself. At the least it needs some libraries and an executer, which are pulled in by Maven.

I would suggest to anyone to spend at least 2-3 hours going through the sourcecode. Take these posts as your guide:
 * http://mail-archives.apache.org/mod_mbox/incubator-shindig-dev/200801.mbox/%3cab5e78ed0801222049n4765d16ar6ba13464776b99f3@mail.gmail.com%3e
 * http://mail-archives.apache.org/mod_mbox/incubator-shindig-dev/200801.mbox/%3caa285c160801230134h111e9a3eqeadea01806ec3a5@mail.gmail.com%3e

=== Third date: Taking it a little further ===
Now I wanted to see what it would be like to be a container. Being a container that supports Gadget functionality (so no !OpenSocial functionality yet, we'll get into that later), is extremely easy.
 1. Go to the javascript directory and follow the instructions in the README (step 1 to 3). As soon as you point your browser to the first sample (running from disk), you've created your very own gadget container! Now that wasn't too hard.
 1. I managed to get samples 1 and 3 running just fine, 2 and 4 result in javascript errors. I guess some of the examples didn't keep track with the code from Shindig. That's not much of a problem through, since all you need to see is available in samples 5, 6, 7.
 1. Samples 5, 6 and 7 is where the fun starts. You need to have your Shindig server running (perferably on localhost (from your browser's point of view) on port 8080), or you'll have to edit the html in those files. Now you have html served from your own webserver, retrieving gadgets through your own Shindig server. I have to admit I felt pretty good about myself at this point :)
 1. Now copy the *.js, *.css and ifpc_relay.html from the javascript/container dir to (the development environment) of your own network. Include the *.js and *.css files in a page on your own (development) container site (make sure to keep the same order of inclusion as in the sampleX.html file!). Don't worry if you can't put them into the <head> element, they'll work just fine within the <body>. Copy the <script>...</script> block and the <div>s from the sample7.html file (again, into the <body> is fine). Finally add a <script> block below the divs, containing the code:
{{{
  init();renderGadgets()
}}}
 1. Call the page you just created in a browser. Congratulations, you now have implemented Gadgets in your website -- or at least some of it's functionality! (I would like to receive some feedback on how much of the Gadget functionality has been implemented when you get to this point in the guide).
 1. Play a little with the number of gadgets, specUrl (perhaps even having them come from your database).
 1. Call all your colleages and start showing off that you built !OpenSocial into your container-site within hours (if you show them lot's of changing titles and gadgets changing in height, they won't notice that the !OpenSocial stuff is still missing).

=== Forth date: Sampling what it should feel like ===
At this point I was growing over-confident, and skipped this step. I believe now that I should have done it: looking into the samplecontainer (as actually step 4 of javascript/README suggests). As soon as I have done this, I will comment on it.
=== Final date: All the way ===
Now it's time to connect your own data-backend to the !OpenSocial javascript code. I won't comment too much on the details here since 1) I haven't done it myself yet, and 2) it's probably a different process for each container. There is one thing I've learned though.

On the first date, we found out that there is no features subdirectory for opensocial-0.x, still all opensocial gadgets begin with <Require feature="opensocial-0.x"/>. When you try to pull a gadget with that <require> tag through the Shindig server, an error will be thrown.

The idea is that you provide this feature yourself, writing the code that is specific to your container. Just today, Paul Lindner from Hi5 [http://mail-archives.apache.org/mod_mbox/incubator-shindig-dev/200801.mbox/%3cF7D1B4266BD42F4B83D92426BD7757C105AFD3D8@MI8NYCMAIL15.Mi8.com%3e shared how they are doing this] (which was pretty close to my first try, just a little better :) ):
 1. create a new subdirectory "opensocial-0.7" in the features directory
 1. create a features.xml file in there with content (this is simplified from the actual Hi5 features.xml)
{{{
<feature>
  <name>opensocial-0.7</name>
  <dependency>opensocial-reference</dependency>
  <gadget>
    <script src="mycontainer.js"></script>
  <gadget>
</feature>
}}}
 1. Create a mycontainer.js file in that same directory with all code needed on how to connect to your backend. The exact way on how to do this will follow, as soon as I'm confident on how to do this best.
 1. Update the features/features.txt file to include your feature (on unix/linux, you can use the small script in the features/README file for this).
 1. Now call a gadget that has a <Require feature="opensocial-0.7"/> through the Shindig server, and in the result you should see your own code! (This way you can implement other features as well, however understand that the "Write once, run anywhere" philosophy of OpenSocial can only be maintained if all servers support the same set of features.

The [http://mail-archives.apache.org/mod_mbox/incubator-shindig-dev/200801.mbox/%3cF7D1B4266BD42F4B83D92426BD7757C105AFD3D8@MI8NYCMAIL15.Mi8.com%3e Hi5 patch of the gadgets.js file], is something that looks like it's going in the right direction. I would suggest using it (if it hasn't made it's way into Shindig by the time you read this).

As I promised, this guide ends somewhere in mid-air. I just haven't gone any further myself, however I thought I'd share my progress so far. Good luck on trying it out!