Assets | Webstrates

Adding and accessing assets
Listing assets and listening for new assets
Deleting assets
Access files in ZIP assets
Creating searchable CSV assets

Adding and accessing assets

Webstrates supports the attachment of assets (files). Files can be attached to a Webstrate by performing a POST request with a file (or multiple files) to the webstrate’s path:

<form action="" method="post" enctype="multipart/form-data">
  <input type="file" name="file">
  <input type="submit" value="Upload">
</form>

The action attribute in the above is the empty string (""), meaning the form will submit to itself. When adding the above code to a webstrate at /myWebstrate/, and submitting the form with a file, the request will be made to /myWebstrate/ and the file added to myWebstrate. Submitting to another webstrate is also possible by changing the action attribute.

Note that forms must have enctype="multipart/form-data" to be recognized by the Webstrates server.

Wait, there's a function for that!

Instead of manually creating a form, one might use webstrate.uploadAsset(function callback(error, result), options). Upon calling the function, a file selection dialog will open. After selection a file, the file will get uploaded to the current webstrate and the callback function will be called. The callback's result value will contain the same JSON object as returned when making a manual POST request (see below).

Uploading an asset will attach it to the current version of the document, but may also be accessed by future versions, assuming no other asset has been added with the same name since.

For instance, uploading cow.jpg at version 1 to myWebstrate will make it possible to access it at /myWebstrate/cow.jpg, /myWebstrate/1/cow.jpg, /myWebstrate/2/cow.jpg etc., assuming those versions exist.

If, however, another cow.jpg is uploaded at version 3, any requests to /myWebstrate/3/cow.jpg, /myWebstrate/4/cow.jpg and so forth will refer to the new cow.jpg, but any requests to previous versions will still refer to the old cow.jpg.

Accessing /myWebstrate/cow.jpg will naturally always point to the newest version of cow.jpg.

Copying and restoring Webstrates will also take the assets into account and perform as expected: Copying a webstrate will also copy over the assets to the new webstrate, so deleting the source webstrate won’t result in the assets in the new webstrate disappearing. Restoring a webstrate will bump the version of the older assets (e.g. the first version of cow.jpg) to the version of the restored webstrate.

See Restoring a document to understand why we bump the version when restoring a document.

When submitting an asset, the server will return a JSON object representing the asset, e.g.:

{
  "v": 128,
  "fileName": "cow.jpg",
  "fileSize": 138666,
  "mimeType": "image/jpg",
  "identifier": "eddc9c8937d6447c550433c5a3f20a65"
}

In the above example, we have uploaded an image cow.jpg with a size of 135 KB (138666 bytes) to version 128 of the document (the current version). The assets has also been given the unique identifier eddc9c8937d6447c550433c5a3f20a65 in the system. When copying/restoring webstrates, files may get different version numbers, making it hard to keep track of a specific version of an asset. The identifier solves this, as the identifier always refers to the same version of an asset.

Note that an asset is always attached to the newest version of a webstrate, because the philosophy of Webstrates is to never modify the history of a document, but only append to it. Adding an asset to a previous version would modify the history of the document. For the same reason, restoring a webstrate also doesn’t modify the history, but only appends to it. Assets cannot be deleted, except by deleting the entire document.

Listing assets and listening for new assets

All assets for a webstrate can be listed by making a GET request to /<webstrateId>/?assets or by accessing webstrate.assets, which contains a list of asset objects. Additionally, it is possible to get notified whenever an asset is added to the webstrate. This is done by listening for the asset event:

webstrate.on("asset", function(asset) {
  // Asset was added to the webstrate.
});

Deleting assets

While the Webstrates philosophy is to never delete history, it can sometimes be useful to exclude certain assets when copying a webstrate or downloading it. As a result, Webstrates makes a method available for just this purpose:

webstrate.deleteAsset(fileName, function(err, asset) {
  // Asset was deleted or an error occured.
});

Deleting an asset will not actually remove it from the server, but rather mark is as deleted at the current version. Assets can only be deleted at the current version of a webstrate to not conflict with the philosophy of not rewriting or deleting history.

For instance, deleting an asset (at version 10) of a webstrate, will add the property deleteAt: 10 to the ?assets list:

...
{
  "v": 5,
  "fileName": "cow.jpg",
  "fileSize": 154224,
  "mimeType": "image/jpeg",
  "fileHash": "aef9cd8d27a2dd39e3268f6bb5364de3",
  "deletedAt": 10,
  "identifier": "3b2b09c18c6c0f5ff308d2ef16a1b574"
},
...

When creating a ZIP from the webstrate – or copying the webstrate – at version 10 or later, the cow.jpg asset will not be included in either result. However, the asset will still be accessible at the prior versions of which it was also accessible before. For instance, if this cow.png asset was added to a webstrate test, it would still be accessible at /test/8/cow.jpg, but not at /test/cow.jpg, /test/10/cow.jpg or any later versions for that matter.

Access files in ZIP assets

Some JavaScript libraries require certain relative paths to exist, for instance when using RequireJS. In these cases, one file might attempt to require module/file.js, which can be troublesome as assets exist in a flat file hierarchy, i.e. no sub-folders. To solve this, users might upload the libraries as ZIP files and access the files from directly within the ZIP files.

For instance, when uploading a library library.zip with a folder module containing file.js, file.js can be accessed using /webstrateId/library.zip/module/file.js, which will let RequireJS (and similar module loaders) work.

To see the files in a ZIP file, ?dir can be appended to the URL at the desired path level, either directly after the name of the ZIP assets or after a path within the ZIP file. E.g. requesting /webstrateId/library.zip/module/?dir will return a list of all files in the module directory within the library.zip asset.

Creating searchable CSV assets

When working with a large dataset, it can be impractical to store it as a plain asset: Whenever the dataset is to be used, the client has to download the full dataset and possibly search through it. On top of that, CSV assets are just plain text, so they won’t contain any indexes, making the process of searching and filtering dreadfully slow and very demanding of the client.

When uploading an asset with searchable=true set in the POST request’s form data, upon arrival on the server, the CSV file will get parsed, converted to JSON and inserted into a MongoDB database where the data can the be queried/searched by the client.

Tip: The options argument to webstrate.uploadAsset(callback[, options]) is an object whose keys/values will be set in the POST request's form data, so to upload a searchable CSV asset, one might use: Use webstrate.uploadAsset(callback, { searchable: true }).

An uploaded CSV asset can be queried with webstrate.searchAsset(assertName, searchObject, function callback(error, result)).

E.g. calling

webstrate.searchAsset(assetName, {
  query: searchQuery
  sort: sortingObject,
  limit: resultLimit,
  skip: entriesToSkip,
}, (err, res) => {
  if (err) throw new Error(err);
  else console.table(res);
});

Will search the asset with name assetName with MongoDB search query searchQuery, sort the result by sortingObject and return resultLimit results, skipping the first entriesToSkip, and print them in a table in the console.

searchQuery and sortingObject are MongoDB search queries. All search parameters are optional. Limit defaults to 10 and can’t exceed 1000.

To learn more about the syntax, see the relevant MongoDB documentation:

Querying is limited to only allow the operators: $eq, $ne, $lt, $lte, $gt, $gte, $in, $nin, $and, and $or.

As an example, the query { $and: [ { name: 'Bob' }, { age: { $gt: 20 } } ] } will return all people named Bob who are older than 20 (age greater than 20). A sort query { city: 1 } will order the result by the users’ cities (starting with A). { city: -1 } will return the reverse order.

A note on downloading webstrates (using ?dl):

For each searchable asset, an empty file will be created with the same name as the original file, but with .searchable appended.

E.g. if a searchable CSV file called data.csv is uploaded, an empty file named data.csv.searchable will also be created. This file is used to let the Webstrates server know that make data.csv should be searchable if reuploaded. Because of this, prototyping across servers (or reuploading a downloaded webstrate) will persist the searchability of assets.