TracNav

Media upload platform

Uploading of media through the API consists of two steps to upload, after which the status can be monitored, and something useful can be done with the media:

  1. Obtain an upload ip and token (media.getUploadToken)
  2. Upload the media
  3. Monitor how the upload and rendering is going, and obtain the media id
  4. Do something useful with the media; e.g. add it to an album (albums.addMedia)

Obtaining an upload ip and token

This is being done through the normal API through a call to media.getUploadToken. This method has no additional parameters, but needs to be called with a valid accesstoken.

The method returns an ip and a token; these need to be used in the next call. Note that the ip and token are only valid for uploading of a single media. If more media is to be uploaded, multiple calls to media.getUploadToken should be done to get multiple tokens. The tokens also have a very short validity timespan, so it is advised to request the token right before the upload can start.

Uploading the media

The upload is a normal HTTP (version 1.1) POST upload. The upload needs to be done to the ip address received , to the path /upload, with the url variable token=(url-encoded value of the token). In the multipart/form-data post body, needs to contain at least the field "file", which needs to contain the file. Other valid fields here are "angle" (which can be 0, 90, 180, or 270); works only for photos, "title" (the title of the media), "submit" (is ignored), "description" (the description) and "geodata" (geolocation, should be 2 floats (in degrees) comma-separated, without a space. E.g. "52.3729,4.8937") to this. Note that all these fields need to be in the post body, not in the url.

The geolocation from the EXIF data is being used if no "geodata" field is present. The orientation in the EXIF data is not being used.

There is a minimum upload speed of 5KB/s. If your upload is consistently below this speed, the connection may be terminated by the server. Note that this is only a problem when the average speed is below this threshold, and then still there is a lot of leeway.

If an error is detected before the upload is complete, the connection is dropped to save bandwidth and time on the client. An error report is sent over the connection before it is dropped, but many http-clients will have trouble reading this error (because it was sent outside of the http spec). You can always retrieve the status message to get the error.

Internally we use the following curl command (in php) to upload: 

"curl http://".escapeshellarg($oXml->ip)."/upload?token=".escapeshellarg(urlencode($oXml->token))." -F file=@".escapeshellarg($filename)." -F title=".escapeshellarg($title)."
-Fsubmit=Verstuur"

Status updates

As soon as the first bytes are received, calls can be done to the server to retrieve the status. The format of this call is an HTTP-GET to: 

http://#ip#/status?token=#urlencode(token)#

Token may be a comma-separated list of tokens -- obviously only tokens that are valid for this ip can be queried on the ip; The result is a JSON string, or JSONP in case a second url parameter "callback" is present. The reply below is an example of a JSONP call (with added whitespace for readability): 

_jsreqCb.cb_4({
  "data":{
    "F-32GMjOiVPJ0dUOZw241jM_EVU0ODcyMjVjZTMxYTIzOGUyAwAAAAEA0AABAPbcbEgQDlnTH3o=":[
      {
        "storing":{
          "expected_endtime": 1215094073.631984
        },
        "upload":{
          "endtime": 1215094004.615327,
          "starttime": 1215094003.034825
        },
        "rendering":{
          "expected_endtime": 1215094072.117697
        },
        "done":{
          "mediaid": null
        },
        "currentstate": "renderqueue",
        "error":{
          "errornumber": null,
          "errormessage": null
        },
        "renderqueue":{
          "expected_endtime": 1215094042.330371,
          "starttime": 1215094004.616161
        }
      }
    ]
  },
  "currenttime":1215094009.904814
})

On the toplevel, there are two keys: data containing all data on the tokens, and currenttime containing the current timestamp on the server. This is a UNIX/POSIX timestamp, and is important since all other times should be interpreted relative to it. Data contains a single entry for each of the comma-separated tokens in the request. Each token contains an array of data; in all API usage only the first entry of the array should ever be set. This contains the state of the upload.

Uploading contains of 5 stages, which will be passed in this order: upload (bytes are being received from the client), renderqueue (media is waiting for a slot to be rendered), rendering (the actual rendering is taking place), storing (the rendered image is being stored on the final storage; please do not confuse this with the Dutch word "storing"), done (this image is done rendering, and can be used). In addition, there is a state error, in case an error occurs. An error may be anything from an unsupported filetype to an internal server error, to a teminated upload. "to render" in this context is the process in which thumbnails from a photo are created, or an flv version of the video.

The currentstate key always points to one of the other keys, and contains the state the media is currently in. The non-final stages all have similar keys: starttime: the time at which the state was entered (this is only available after the stage has been entered), endtime: the time at which the stage was finished (only available when the stage has actually finished), expected_endtime: best guess of the endtime of the stage (available as soon as there is some educated guess we can make there).

The done state has a mediaid that will get set as soon as the media is done. This mediaid is directly usable in subsequent calls to the API. The error state has a message and errornumber. We will in time provide an accurate list of errors and solutions; for now hopefully the errormessage gives enough information. If not, please email us with the error and the ip/token used.

There is a limit on the number of state calls you are allowed to do: only one state call can be done per token every 2 seconds on average.

Add media to album

Now the mediaid can be used in other API methods, for example add it to an album.

Errors

Per errorcode, I'll give a short description below:

  • 1000: An unknown server error. You should not be getting these. Please report them to us, especially if you can reproduce them
  • 1: Tokens are only valid for a limited amount of time. Currently, tokens gotten through the API are valid for 1 hour, so you need to make sure that you start your upload within the hour
  • 2: In the call you used to acquire the token, you were provided with an ip address as well. The token is valid only for that one IP address; if you use it on some other server, you'll get this error
  • 3: Tokens are generally valid for only one upload (attempt). If you try to reuse a token, you'll get this error.
  • 4: We don't recognize the token. Either you made a typo, or (more likely) forgot to url-encode it. Urlencoded tokens generally end in %3D
  • 5: We only accept HTTP/1.1 calls. Please use an HTTP/1.1 client
  • 6: We cannot parse the uri as valid; check that you use correct encoding throughout the uri
  • 7: The only valid uri's are /upload and /status; all other calls will result in this error
  • 8: On the /upload uri we expect a POST, on the /status a GET. Anything else will throw this error.
  • 9: We expect the token= parameter to be present in the url, not in the POST body
  • 10: We do not recognize the type of file you're uploading. This file is therefor unsupported
  • 11: We do recognize the type of file you're uploading, but this user is not allowed to upload the file type
  • 12: The quota of this user was reached. Note that we use different quotas for video and pictures.
  • 13: We found an unrecognized parameter in the URL or POST body
  • 14: A parameter that we expected to be of type "file" is of type "string" or vice-versa.
  • 15: A parameter is present more than once
  • 16: We encountered an unrecoverable error in the file that was uploaded; therefor we cannot accept it
  • 17: The Hyves upload service is temporarily unavailable; we're working on making it available again soon
  • 18: One of the parameters has an unexpected value; for instance a parameter that only accepts numbers has a non-numeric value
  • 19: During upload the connection was closed from the client side
  • 20: There was a timeout reading the request. We have pretty large timeouts, however expect an average upload-speed of at least 5kb/second for larger files
  • 21: The format of the callback paramater is wrong; the same rules apply to callback variables as in the hyves-api
  • 22: You're not allowed to do more than 1 status call every 2 seconds on average for every token. If you exceed this limit, your status calls will be answered by this error

If we were to group the errors:

  • 1000: Unknown
  • 10, 11, 16: The file being uploaded is invalid
  • 12: The user reached his quota
  • 17: Temporary failure; you might want to try uploading the file again in a couple of seconds
  • 18: Error in the call; this is an error in your client software